Introduction to Performance Testing Part 1

Introduction toPerformance TestingPart 1(V1.1)

Author:Vjacheslav LukashevichEdited by:Eugene Muran

Part 2

Part 1•Test types•Measurements and scenarios•Testing team•Test tools•Planning and Implementing tests

•Tool Configuration•Test Run•Result analysis•Facts about development

Fact is that -…

No one cares about software

performance at beginning

Fact is that -…

And at the end we get something like this …

Or even worse…

SMALL REMARK:

ALL OF ABOVE MENTIONED

IS ALREADY IN THE PAST

Our days

When performance testing is a “must”

We expect large data volumesWe expect high loadInexperienced programmersCustomer requests it

Question!

What types of performance testing exist?

Test typesPerformance TestingLoad Testing

Stress TestingEndurance testingScalability testingRecoverability testing

How we name it - depends on test target!

Question!

What we will measure by the performance testing?

Application “speed” performance characteristics

Throughput – number of actions per time moment.

Response Time – time taken by action to complete

Application hardware characteristics

Primary resources:CPU (%)

Total CPU usage on hostCPU usage by target process

Memory UsageUsed Memory (on host or by process)Available Memory

NetworkNumber of bytes sent/received per second through network interface

Bottleneck

“Bottleneck” is the place where performance degradation occurs

Scalability

Ability to handle additional workload when:Adding hardware (memory, CPU, storage)Adding additional hosts (machines)

Performance test targets

Common targets:Get performance statistics (application speed)Find bugs (software stability issues)Find out bottleneck’sFind out capacityEstablish baselines

Other targetsScalability checkCrash testRecovery test

Each test is a scenario or a set of scenarios

Question!

Who is participating in performance testing?

Relationship Between Performance Testing and Tuning

Performance Tests

Performance Characteristics

Performance Tuning

Tuning process is done when:performance testing reveals system or application characteristics unacceptable to reduce the amount of resources being used

Performance tuning

Cooperative effortProduct vendors Architects Developers Testers Database administrators System administrators Network administrators

Performance testing activities

1. Identify the test environment

2. Plan and Design tests3. Configure the Test

Environment4. Implement the Test Design5. Execute the Test6. Analyze Results, Report,

and Retest

Identifying test environment:Hardware

In ideal case test environment must be as close to production environment as possible

Identifying test environment:Software –key components

Operation System(MS Windows, Linux,…)

Application Server(IBM Websphere, BEA Weblogic, Tomcat, …)

Database Server(IBM DB2,Oracle,MS SQL Server, …)

Identifying test environment:Software

For performance tests key software components in environment must be as close as possible to production environment –can impact overall system performance

VERSION#

->Patch VERSION#

(Build #)

Configuration

<Settings>

<<Java version>>

Performance testing scenarios

Scenario:Sequence of steps in applicationRepresents a use case

Question!

What scenarios we need to test?

Plan and Design tests:Determining Test Scenarios

Key Usage scenarios to include:most common, frequentbusiness-criticalperformance-intensivetechnical concern high-visibility, obviouscontractually obligatedstakeholder concern

Applying usage pattern for test design

Daytime user activity

0

500

1000

1500

2000

2500

3000

3500

4000

4500

Use

r Act

ions

per

hou

r

Options:1) Emulate exact usage pattern2) Run under maximum load (peak

load)3) Run under average load

Peak load

Most important - performance scenario must be Repeatable (!)

Plan and Design tests:Determining Test Scenarios

Repeatable test scenario

Dataprerequisitescleanup

Tests sequenceEnvironment state

Performance Test Plan

Test Plan contentTest environment Test dataTest scenariosPerformance Measurements

PERFORMANCE TEST PLAN

Test Plan: Test Data and Test Scenarios

Test prerequisites:DatabaseUsers Files (documents, images …)

Test ParametersScenarios listThink timeLoad SchemeDuration

TEST IMPLEMENTATION

Environment setup

Application deploymentDatabase setup Users/ Access RightsSystem clock synchronizationPerformance Monitoring services configuration

Stable version

For performance testing it is preferable to select stable, well tested using functional testing application version

To avoid the surprises for the performance testing it is preferable

to select stable, well tested by functional testing application version

Plan it and target for it!

Perform the smoke testing!

Before beginning the scripting, verify that the scenarios you are going to implement in your test script are working at all!

TESTING TOOLS

2 Main Purposes of test tools

Load Generation

Measurements/Statistics gathering

Best thing, if the tool can do this both tasks in parallel

2 ways to emulate web application

“Play” with browserLinks/Button clicksForm filling

Send POST/GET request to server

What tools we have for such testing?

LoadRunner(different protocols: POST/GET, web browser emulation/GWT/Mobile/SAP)

Jmeter (POST/GET)

Grinder(POST/GET)

*this is only some of available tooling

Capture/Replay

Pluses (+)Fast and easy test creation

Minuses (-)Unstable scriptsApplication change sensitivityUnnecessary requests in scriptsOverhead on server side for replayed scripts

Tests debug

Like any computer program, performance tests also must be tested

Possible checksView returned pages during replay in test tool (in LR - visual test results)Test tool logsServer logs

Content check

Check content, that is only available, when system operates correctly

Books

Performance Testing Guidance for Web Applications

http://perftestingguide.codeplex.com/downloads/get/17955

Web load for dummies