May 25, 2015
Introductions, Objectives and Overview
Reason for performance test
Slow applications produce user frustration
Any feature can be source of performance frustration
Inappropriate resource use can be disaster like failure.
Perception Load time ? transaction measure? (how is it different
from time measure) Resource utilization? Number of users load? Ability to run without interruptions?
Black-box perspective Response time Resource utilization Reliability
White-box perspective Code profiling Resource leakage Code coverage ( branch and statement )
Performance Perpetrator
Response Time is total time from once request made till completion.
Response time can be calculated based on queuing model M/M/1.
Formula is R= S/1-U. where R is response time S is No of server U is utilization
Response time and Resource utilization
• Example: The service time for a single component increases as the utilization increases beyond 60 percent. – If a transaction requires 1 second of processing by a
given component, at 45 percent utilization, it could be expected to take (1/(1-0.45)=) 1.66 seconds
– At 80 percent utilization time would be (1/(1-0.8)=) 5 seconds.
– When utilization for the resource reaches 90 percent, the transaction could be expected to take 10 seconds to make its way through that component.
CPU Utilization Trade offs - High Utilization graph of the CPU does not always indicate a performance issue.CPU performs all calculations that are
needed to process transactions. The more transaction-related calculations
that it performs within a given period, the higher the throughput will achieved for that period.
As long as transaction throughput is high and proportional to CPU utilization, this indicate that computer is being used to the fullest advantage.
If the response time for transactions increases to such level that it become unacceptable, this could be an indication that The processor might be swampedThe transaction process load is too high for
the System to manage. The CPU is processing transactions
inefficiently CPU cycles may be diverted
Network Bandwidth Utilization - Network bandwidth utilization should be captured to understand the possible bottlenecks because of large volumes of data is transfer
Database size and System performance - Performance degrades as the database size increases which do not take care of the increasing size.
Role of Hardware in Streaming applications - Same bit rate video shows different behavior when played from a USB 2.0 and 3.0 devices.
Performance of APIs - Performance of system is sum total of API responses + services
Other Factors
Consistent hardware Consistent software Consistent measurement method Benchmarks
Scholarly Perpetrator
Loading of unnecessary components Deadlocks/sleeps Memory leaks/Corruption Continuous increasing data size Unnecessary encryption /decryption Algorithms with out optimization Heavy Graphics Too much CPU or I/O bound
Common perception
Performance testing requires clearly identified performance goals like page load time of less than 4 seconds, page
fetch and rendering time of less than 2 seconds, saving file of certain size/complexity in certain time
Identify ranges for these goals with the operating conditions
For other scenarios(not in performance test plan ) identify typical use conditions and a subjective feel of acceptable parameters
Performance Destination
Reduce global variables Optimized string sizes Removal of extra strings Late initialization
Resolvent
Identification of features to be tested for performance
Set of performance goals for each feature –( response time, reliability and indicative ranges for resource utilization, )
Specification of Test environment creation of tests and scenarios Test execution and data collection Analysis Iterations of performance tests after
changes
Testing Process
All features are not equal priority Before performance testing begins
prioritize featuresMost used featuresCritical featuresNon-feature actions
Portent Prioritization
Performance Testing atUnit Level?? Integration Level??System Level??System Integration Level??
Which is better? First performance test cycle??Last performance test cycle??
Why?
Time for performance testing
Performance testing process should start from the stage which is determined by the risk associated with performance of the system.
If risk based testing method should adopted, performance risks should identified and prioritized at the start of project. If performance is deemed to be a high
priority risk area, performance testing is typically started earlier.
Client hardware benchmarking is also important in Performance testing. The hardware incapability may provide faulty
results, which leads confusion of performance issue of server.
Memory , CPU (cycles must be taken care when dealing with multi-thread applications), network bandwidth etc. should be benchmarked prior to start the performance test.
Benchmarking of client machine helps in getting a better idea of how much throughput could be attainable, and this could be useful in scaling process.
Benchmarking of client hardware
dynamic analysis performed on the code to monitor the code as the program executes.
Profiling is the collection of performance analysis data of a software at execution.
Profiling helps improve the software’s performance through code optimization.
Code profiling
Why Code Profiling?• Profiling is made to optimize the code with
issues• Profiling:
– Identifies Bottlenecks in Performance of the Code (program)
– Utilization Catches Memory and Resource Leaks
– Determines the subroutines/ functions consuming the longest time to execute
• Profiling can improve the overall performance of the code, and thereby the whole software.
What to look for while Profiling?Profiling tools basically look into the code
for specific areas of concern like: Resource/Memory Leaks Library Calls (Static as well as Dynamic) Functions execution time mostly etc.
A leak occurs when a program uses resources on a computer but never releases the resources.
The resources can include handles, physical RAM, the paging file, system resources, etc.
normally it is the result of a defect in a program that prevents it from freeing memory that it no longer needs
Symptoms of a memory leak include – system hangs, slow system performance, and program memory error messages.
Leaks
Leaks: Why an issue? It can diminish the performance of the
computer - By reducing the amount of available memory.
OR Too much of the available memory may become
allocated and the system/application may stop working correctly, fail, or slow down unacceptably.
• Memory Leak Scenarios – – The following code sample depicts a
Memory Leak
Memory Leak: “x” has not been freed
Conclusion it can demonstrate that the system
meets performance criteria It can compare two systems to find
which performs better It can measure what parts of the system
or workload cause the system to perform badly