Load Tests 101! Why and where to start?

(RE)LOAD TESTS 101

WHY AND WHERE TO START?

AURÉLIEN DELEUSIÈRETHE ONLINE COLDFUSION MEETUP

WHO AM I?

• Software engineer specializing in web technologies, consulting, development, architecture and troubleshooting

• Founder of Webellian based in France and Poland

• ColdFusion since 1996, Java, PHP (mainly for CMS), AngularJs, ExtJS…

WHY LOAD TESTING?

Load Testing

Performance

Resources

Scalability

Robustness

WHERE TO START?

• Depends on the architecture!

• Two approaches:

• Define the limit of a system in the absolute

• Validate the capability of a system to resist to an estimated traffic, anticipate spikes link to special events

SCENARIO “CALIBRATION”

• have a picture of the “background noise” of the scope tested

• have the best response time to calibrate results analysis

SCENARIO “HEALTHINESS”

• have a picture of the profile of resources consumption (JVM Memory and Garbage Collection, System CPU, I/O)

• validate the healthiness of the component before Stress Scenario

SCENARIO “STRESS”

• determine the maximum load capacity (MAX) where the target conditions are reach in term of response time and stability

• determine acceptable response time variation (in percentile)MAX

SCENARIO “ENDURANCE”

• validate the maximum capacity determined during Stress Scenario

• validate the healthiness of the component under target load

SCENARIO “BURST”

• validate the capacity of the component to support a sudden high pressure

• this case of sudden high pressure can happen when you put a server back in the pool, or when a special campaign is starting

ABOUT GATLING

• Very high performance testing Engine

• 1 thread ≠ 1 request

• Distributed multi-nodes injectors

• Scala descripted scenarios (≈ DSL)

ABOUT GATLING

Question: How much performant?

How many requests per second with a single injector (16 vcpu on AWS)?

A SIMPLE EXAMPLE

• Produce a chain “ABCDEFGHIJKLMNOPQRSTUVWXYZ”

• By concatenating ASCII characters from 65 to 90

• Tests done locally:

• MacBook Pro running Virtual Box guest Ubuntu

• ColdFusion 2016u3 (2016.0.03.300466) Developper Edition

• Lucee 5.0.0.254

REAL EXAMPLE: A QUOTATION SERVICE

• Service Layer is composed by several Component (S1, S2

…) behind a Load Balancer (LB) in a stateless fashion.

• The Service Layer is behind a Gateway in charge of Authentication, Transformation, Routing and Logging that is also composed by several instances (G1, G2) behind a Load Balancer.

• No database, configuration in memory from configuration (XML, DSL) files.

• What to be tested?...

way G3 G2

TRAFFIC

Datacenter 1 Datacenter 2

S1 S3 S5 S7 S2 S4 S6 S8

Agents

• Isolate a representative architecture to be tested…

• …in an environment identical to production

• Install the load tests Agent as close as possible to the component to be tested

• Install tools to monitor the resources (JVM memory details, GC, System CPU) like Fusion Reactor or Mission Control

• But you can also do test as a “black box”

Services S1 S2

Gateway G1 G2

TRAFFIC

FUSION REACTOR

MISSION CONTROL

The Load Tests are run against one single component to measure the intrinsic capability of one single instance.

Main objectives:• Measure the maximum capacity of one single component

• Measure the response time at component level

Services S1 S2

Gateway G1 G2

TRAFFIC

The Load Tests are run against a couple of component behind the Load Balancer.

Main objectives:• Determine the capacity factor between 1 instance and 2

instance that is generally is not linear

• Measure the response time to evaluate the latency introduced by the network devices

Services S1 S2

Gateway G1 G2

TRAFFIC

The Load Tests are run against the full Service layer through one single Gateway instance.

Main objectives:

• Measure the capacity of one single Gateway instance

• Measure the response time to evaluate the latency introduced by one Gateway instance

Services S1 S2

Gateway G1 G2

TRAFFIC

The Load Tests are run against the full stack.

Main objectives:• measure the response time with the full stack to validate the

order of magnitude for SLA

If possible validate this result with the full stack.

Services S1 S2

Gateway G1 G2

TRAFFIC

INITIAL TEST CAMPAIGN

CHRISTMAS 2016 SPIKE CAMPAIGN

1. GATHER EXISTING DATA

Experience from 2015:

• brutal ramp up at 6:00am from almost nothing to 2000 rq/min;

• peak around 7:00am around 1850 rq/min;

• plateau between 9:00am and 10:30am around 1500 rq/min;

• then 30% to 40% of sales more than usual during the next weeks.

Let’s focus on the 2000 rq/min

ILLUSTRATIVE

2. CHALLENGE BUSINESS EXPECTATIONS

Based on the current trends, the business expects an increase of 30%.

Let’s challenge our system with:• 2600 rq/min• A brutal ramp up from 0

3. FORECAST YOUR TRAFFIC

3. 2600 rq / min => 44 rq/sWrong! You have to correct the linear projection

“Feel” the volatility of what is your case.Here for a starting spike it should quite stable => 2x

Let’s challenge our system with:• 88 rq/s• A brutal ramp up from 0

4. SIZE YOUR SYSTEM

Original system design

way G3 G2

TRAFFIC

Datacenter 1 Datacenter 2

S1 S3 S5 S7 S2 S4 S6 S8

SERVICE CAPABILITY

Service 80 rq/s per server

GATEWAY CAPABILITY

Gateway 25 rq/s per server

4. Size your system

Target 88 rq/s

Service: 80 rq/s x 8 = 640 rq/s OKGateway: 25 rq/s x 4 = 100 rq/s OK

But….you have to expect a disaster!Service: 80 rq/s x 4 = 320 rq/s OKGateway: 25 rq/s x 2 = 50 rq/s NOT OK!

Required action: add 4 new nodes in Gateway layer.

way G3

TRAFFIC

Datacenter 1

S1 S3 S5 S7

Datacenter 2

S2 S4 S6 S8

1. Gather existing data2. Challenge business expectations3. Forecast your traffic4. Size your system5. Seat back in comfort on D-Day and bring the popcorns

• Measured results

Estimated: 2600

Estimated: 88

WHEN LOAD TESTING?

• Usually done on Major version releases

• Often by a specialist consultant

• Should be integrated into your automated tests

SPECIAL THANKS

• Mohamadou Mballo, a genius developer that just felt in love with CFML

• Stéphane Landelle father of Gatling for his support and the goodies to give away

• Yamina, my supportive girlfriend, that would be upset if she’s not listed on this page…

…and all of you making CFML so special…

You liked it? Tweet it!@adeleusiere

Load Tests 101! Why and where to start?

Software

Load-Carrying Performance and Hydrostatic Tests of ...

Pile Load Tests Including Quick-Load Test Method ...

Load-testing 101 for Startups with Artillery.io

Mineralogy 101 Chemical Tests

foundation excavation, underpinning, and field load tests

Messier Dowty Outstanding load tests

R-136 - Load-Deflection Tests on Corrugated Multi-Plate...

SINGLE PILES: DYNAMIC ANALYSIS, LOAD TESTS

Seismic Load Tests - EQC

Investigations and load tests in silty soils - Startsida -.....

Assessing Bridge Conditions by Performing Static Load Tests

Static Load Tests of Franklin Steel Signposts

Analyses of the Lateral Load Tests

Compilation of benchmark lateral load tests on piles

Al Mann - 101 Psychic Tests

Pile Load Tests