How to Start Performance Testing?

How to Start Performance Testing?

Nebojsa Videnov15 December 2015, Belgrade, SerbiaPHP Meetup

http://nos.twnsnd.co/

I wish, I wish,..

https://stocksnap.io/author/622

Agenda

Requirements

Vocabulary

Test Planning

JMeter

Takeaways

https://stocksnap.io/author/403 Jeffrey Betts

Requirements

https://stocksnap.io/author/652 Daria Nepriakhina

Badly formulated

“We expect 20 million users per year.”

“We anticipate 50,000 users a day.”

https://stocksnap.io/author/5574 Avel Chuklanov

How to format Requirements?

● What does web site needs to do well?● What is usage pattern of the proposed system?● What is daily peak demand? ● How much testing environment is similar to production environment?● Can you provide a breakdown of the frequency of certain activities?

(for example, 75% browse, 2% check their order status, and so on)● Where is the project in terms of the overall project performance goals?● ...

Questions.

Vocabulary

What is a hit?

- HTTP request- Whole page- Entire visit by a given user

https://stocksnap.io/author/402 Skitter Photo

What is a day?

Determining the period of time when most users arrive on the system becomes

critical for estimating peak load.

https://stocksnap.io/author/56 Sonja Langford

Throughput

- Transaction per unit of time (Transaction pair - HTTP request and response)

- Page rate- User rate

https://stocksnap.io/author/3350 Will Langenberg

Throughput plateau

Requests per second

Load

Throughput plateauSaturation point

10 30 8020 5040 7060

25

50

100

125Buckle point

Heavy load zone

Light load zone

THROUGHPUT

Response Time

https://stocksnap.io/author/4440 Luis Llerena

Standard Deviation

Standard Deviation is

measure of variability of data set.

https://stocksnap.io/author/3350 Will Langenberg

90% Line90% Line is the value below which 90% of the samples fail.

https://stocksnap.io/author/9429 Raymond Sam

Median

Number which divides the samples into two equal halves.

Half of the samples are smaller than the median and

half are larger.


Latency

Time between first request is sent until first byte of response reaches the client.

https://stocksnap.io/author/2784 Wil Stewart

Concurrent Load

https://stocksnap.io/author/16087 Marcin Czaja

Active Load


Pick Load

Peak Load refers to the maximum concurrent users within some

time period.

https://stocksnap.io/author/408 Fré Sonneveld

What is a user?

User is:1. User is a hit.2. User is all day visitor.

Once logged in for the whole day.user = whole set of activities by the user for the whole day.May or may not be very active.

3. User visit.visit = the most common usage of user

https://stocksnap.io/author/15749 Stephen Radford

Thinking Time

Interval between user requests

https://stocksnap.io/author/17101 Greg Rakozy

Steady stateThroughput

10 30 1020 5040 3040

25

50

100

125

Steady state

Ramp-downRamp-up

Load

Ramp-up periodThroughput

10 30 1020 5040 3040

25

50

100

125

Steady state

Ramp-downRamp-up

Load

Ramp-down periodThroughput

10 30 1020 5040 3040

25

50

100

125

Steady state

Ramp-downRamp-up

Load

Operation ceiling

Number of concurrent users that the application can withstand before becoming totally unresponsive

https://stocksnap.io/author/5300 Joseph Barrientos

Starting Performance Testing

http://nos.twnsnd.co/

Team for Performance Testing

Before you start

Develop goals and plan for obtaining the desired measurements.

Develop realistic scenarios

https://stocksnap.io/author/1949 Startup Stock Photos

1. Gather requirements

https://stocksnap.io/author/5257 Lukasz Kowalewski

2. Set your goals

https://stocksnap.io/author/16572 Chen YiZhou

3. Estimate

https://stocksnap.io/user/saumya04 Saumya Rastogi


Estimation - part 1.

- We anticipate 50,000 users a day?

- What is a user? User is a visit.

- What is a day? 8 hours

- Average user visit: 10 minutes

- Session timeout: 15 minutes

- Average pages per visit: 5

- Peak time factor: 5

- Requests per page: 6


Estimation - part 2.

50,000 users per day / 8 hour day = … ~= 1.7 users/second

Peak hours: 1.7 users/second * 5 ~= 8.5 users/second

During visit: 8.5 users/second * 10 minutes ~= 5,100 users5,100 users * 5 pages per visit = 25,500 pages per 10 minute visit6 requests per page * (25,500 / 10) / 60 =

THROUGHPUT = 255 requests/second

4. Performance testing tools

JMeter.

https://stocksnap.io/author/6696 Peter Belch

JMeter - Getting started

Test Plan

Test Group

Samplers

Logic controllers

Listeners

Assertions

Response Assertion

Duration Assertion

Size Assertion

Distributed testing

Plugins

Stepping Thread Group

Ultimate Thread Group

Others

Sending JSON using HTTP

HTTP Cookie Manager

CSV File Reading

Timers

Takeaways

https://stocksnap.io/author/63 Pawel Kadysz

JMeter Best Practices

https://stocksnap.io/author/402 Skitter Photo

- Generate reports after test run.

- Use CSV output rather than XML

- Use distributed (remote) testing

- Do not use XPath extractor

- Do not use Regular Expression Extractor with checked Body (unescaped)

- Use latest version of JMeter

- Use Cookie Manager for each Thread Group

- Read data from CSV files

- Use variables

- Use non-GUI mode

- …

- http://testerfenster.com/blog/jmeter-tutorials-best-practices/


http://testerfenster.com/blog/jmeter-tutorials-best-practices/

http://testerfenster.com/blog/jmeter-tutorials-best-practices/

Test script


-

- Provide a warm-up period to initiate the environment

- Obtain measurements within a steady-state period

- Verify error-free test execution

- Drive same workload between runs

- Capture measurement without impacting performances


Use performance terminology consistently.

https://stocksnap.io/author/5678 Piotr Lohunko

Your performance test is as good as data it produces.


Resources

Resources

Performance Testing Guidance for Web Applications Microsoft Corporation

Performance Testing with Jmeter - Second Edition Bayo Erinle

Performance Analysis for Java Web Sites Stacy Joines, Ruth Willenborg, Ken Hygh

https://blazemeter.com/blog/

http://www.testerfenster.comhttp://testerfenster.com/blog/jmeter-tutorials/

Thank You

Nebojsa Videnov

www.testerfenster.com

@TesterFenster

http://www.testerfenster.com

http://www.testerfenster.com

How to Start Performance Testing?

Engineering