JMeter Intro

● Performance testing•Load Testing•StressTesting

● What is JMeter● JMeter basic elements● Building a Script●Running a Script●Recording a Script (Bootstrapping)

Sam Varadarajan, for [email protected]

Agenda

Performance Testing

What is Performance testing?

Performance testing is performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It is typically non-functional, meaning, we only care about load on the system, scalability etc. Actual functionality is not tested in this phase. (That will be done in Functional Test phase).

The real goal is to find that maximum load that the system can handle without performance degradation or crashes.

There are several types of performance testing, but 2 are important:• Load/Capacity Testing

• Stress Testing

We will be focusing mainly on Load Testing in this presentation.

Load / Capacity Testing

• Load testing is the simplest form of performance testing. A load test is usually conducted to understand the behavior of the system under a specific (expected) sustained load.

• The load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration.

• In simple terms, it is the maximum load the horse can carry without it tripping or the carriage toppling!

• During the tests we will collect the the response times of all the important business critical transactions. If the database, application server, etc. are also monitored, then this simple test can itself point towards any bottlenecks in the application software.

Stress Testing• Every system has a capacity limit. When the load

goes beyond the limit, the System starts responding very slowly and even produce errors. we have seen this in Databases, Network etc.

• It is more visible on a Web site, as number of users can drastically (often unpredictably) increase, depending on content. We saw this recently with HealthCare.gov! Sometimes, websites crash due to attacks such as denial of service attack.

• In simple terms, Stress on the system is that last straw on the camel's back that broke it.

• The purpose of the stress testing is to find the capacity limit of the system. We do this by increasing the load on the system beyond a safe limit, so the system will eventually crash. With the findings, we can adjust the parameters to stretch this limit.

General Load Testing ScriptLoad Testing itself can broken down into following steps:

• Create Test (Can be recorded using some tool)• Configure workload settings• Add measurement hooks• Data drive script (using variables and parameters to generalize it)• Add logging• Generate reports/charts from performance data collected.

Load Testing - ManuallyThis is how they did load testing before.Just ak a bunch of users to come to a training room. The users will start using the application according to a script and run simultaneously!We probably generated a very weak load and didn't really stress it either!

Load Testing Automated•All or most of these steps can be automated using several different programs/scripts. •There are testing tools that have integrated these into tools called Load Testing tools. •Mercury Load Runner and Rational Rose tools are commercial products that have been around for a long time. •With the advent of Web applications, several new tools have appeared in both commercial and open source arena. Following slide shows a list of Load testing tools available.

Load Testing ToolsTool Name Company Name Notes

Apache JMeter An Apache Jakarta open source project

Java desktop application for load testing and performance measurement.

BlazeMeter BlazeMeter Ltd. BlazeMeter is a JMeter compatible, self-service, load testing platform.

Blitz Spirent Communications Blitz is a service for load and performance testing of websites, mobile, web apps and REST APIs in the cloud.

Gatling Open Source JVM application with scenarios as code and portable HTML reports.

Loader.io SendGrid Labs Cloud based load testing service

Load Impact Load Impact - AB Cloud based performance testing SaaS tool.

LoadRunner HP Performance testing tool primarily used Licensed.

Load Test (included with Soatest)

Parasoft Performance testing tool

loadUI SmartBear Software Cross-platform load testing tool

Login VSI Login VSI, Inc. Performance testing software for Windows based virtualized desktops. Licensed.

NeoLoad Neotys Load testing tool for web and mobile applications.

OpenSTA Open System Testing Architecture Open source web load/stress testing application, licensed under the Gnu GPL.

Rational Performance Tester IBM Eclipse based large scale performance testing tool

Silk Performer Borland Application performance tool with cloud and local virtual agents. Licensed

Test Studio Telerik Test Studio is a load testing tool

Visual StudioUltimate edition Microsoft Visual Studio Ultimate edition includes a load test tool

WebLOAD RadView Load testing tool for web and mobile applications Licensed.[3]

(Courtesy: Wikipedia)

JMeter

What is JMeter?

• JMeter is a free, open-source performance measurement tool written in Java

• JMeter is built by the reputed Apache project community

• JMeter is available as a desktop Java application and can be run in GUI and batch modes. It can also be run in distributed modes.

• It has a completely extensible architecture. It allows us to develops various elements that can be reused in different parts of a test plan. It also allows plugins to be developed, so the tool can be further extended.

Why JMeter? It is an Open source tool. Can load and performance test many different server types:

Web - HTTP, HTTPS SOAP Database via JDBC LDAP JMS Mail - POP3(S) and IMAP(S)

User friendly GUI Design. Can be run in batch and distributed modes to handle more load. Full multithreading framework allows concurrent sampling by many threads and simultaneous

sampling of different functions by separate thread groups. Can mimick Web browser functionality including Caching and Cookies. Controllers are configurable and can monitor the server performance. Test results can be captured in various format like summary report, Graph, Aggregate report,

Aggregate graph, Results in tree and Results in Table. The results can be stored in files for later review as well.

Extensible architecture. Supports plugins, bean shell etc to add more functionality.

Installing Jmeter

• Install Java (1.5 or higher)• Download JMeter http://jmeter.apache.org/download_jmeter.cgi• Add path of java installation in environment path variables.

• A test plan describes a series of steps JMeter will execute when run.

• A complete test plan will consist of one or more Thread Groups, logic conrollers, sample generating controllers, listeners, timers, assertions, and configuration elements.

JMeter Test Plan

Elements of a Test PlanThe Test Plan is the wrapper for the entire test suite. It wraps around the following elements in an hierarchical fashion.

• Thread GroupThis is really the logic unit of a load test. There can be many thread groups in a Test Plan.

–Setup number of thread–Set up ramp up period–No. of times test executes

• Controllers–Sampler (Send Request to Server)–Logical Controller (Customize logic to send request); includes Loop Controller and One Time Controller.

• Listener Help to capture and display results in various formats. For e.g.,

–Graph Result–View Results Tree or Table format and many more.

• Timers–Add a delay to the next request

Continued….

Elements of Test Plans

• Assertions– Allow you to assert fact about responses received from HTTP request

• Configuration Elements– Allow you configure settings

• Preprocessor – Execute prior to sampler request

• Post Processor– Execute some action after sampler request

1. Config Element2. Pre-Processor3. Timer4. Sampler5. Post-Processor6. Assertion7. Listener

Order of Execution

Building a Test Plan

• Thread group elements are the beginning points of any test plan.

• All controllers and samplers must be under a thread group.

• Each thread represents a user setting a thread group to 500 simulates 500 users.

Thread Group

Thread Group Contd...

• Samplers tell JMeter to send requests to a server and wait for a response. They are processed in the order they appear in the tree.

Sampler

Sampler Contd...

• Listeners can be used to display (or store) performance information JMeter collects during tests.

Listener

Listener Contd...

Listener Contd...

• Logic Controllers let you customize the logic that JMeter uses to decide when to send requests.

• Logic Controllers can change the order of requests coming from their child elements. They can modify the requests themselves, cause JMeter to repeat requests, etc.

Logic Controller

• A configuration element works closely with a

Sampler. Although it does not send requests (except for HTTP Proxy Server ), it can add to or modifyrequests.

Configuration Elements

Configuration Element Contd...

Configuration Element Contd...

• Assertions allow you to assert facts about responses received from the server being tested.

• Using an assertion, you can essentially "test" that your application is returning the results you expect it to.

Assertions

Other Test Elements

● A Pre-Processor executes some action prior to a Sampler Request being made.

● A Post-Processor executes some action after aSampler Request has been made.

●A Timer will cause JMeter to delay a certain amount of time before each sampler which is in its scope.

Variables

JMeter allows parameters, properties and variables to keep the test scripts generalized.

Syntax for Variable:${VARIABLE}• Variables can be

declared and used in successive layers.

JMeter Test Plan Completed

JMeter Test Plan Completed

• Once you completed, you can save the test plan with a meaningful name.

• The script file generated is an XML file and will be saved with a .JMX extension.

• You can run this script using command line mode as well, so you don't have to load GUI.

Running a Test Plan

Running in Batch Mode

On command line, in the JMeter\bin directory, you can type:

jmeter -n -t C:\sam\load_test\scripts\LT600_laweb2_test_syt6.jmx -j C:\sam\load_test\log\jmeter_1212.overnight.log -l C:\sam\load_test\log\jmeter_1212.overnight.jtl

Where,

LT600_laweb2_test_syt6.jmx is the script I saved in the completion step above.

-n makes it non-GUI

-t name of the test script you want to use

-j for the JMeter.log (I change this to match my log file naming above)

-l for the JTL file - in XML format, has the same details we get in listeners above. We may have some use for this, but for now I am just saving it.

You can put the above command inside a batch file to save typing!

Appendix

Non-Testing Elements

Non-Testing Elements Contd...

HTTP(S) Script Recording Element

Recording a Script

• Setup a non-test element, HTTP(S) Test Script Recorder on the workbench.

• Setup all the inclusion, exclusion lists and other properties.

• Click on Start. Now, JMeter is running as a local proxy

server. If you adjust PC's proxy setting to use this as the proxy server, all requests will pass through JMeter.

Recording a Script Contd...Next you need to setup the browser to use our JMeter listener as a local proxy

Recording a Script Contd...


• Now that we have captured all the pages to be visited, we can clean it all up. This is a good place to start.

• Parametrize and add variables wherever possible and move the pages (samplers) to a Thread Group and build up our test plan.

Recording a script Contd...• An easier way is to use BadBoy software to capture the pages and export to

JMeter. Download and install Badboy software from http://www.badboy.com.au/download/send?svId=124


Real life Test Scenario

• Our App Servers kept crashing on heavy load days. We had to constantly restart the servers, bringing down all the web users along with it. After making some changes to the application, had to simulate the huge load on the servers to test if it will sustain the next tax season. After reviewing various tools, we chose JMEter.

Real life Test scenario Contd...

• When we upgraded our infrastructure* recently, we used JMEter to compare old environment and new environment for performance. We ran the same set of scripts against old and new environments to identify any bottlenecks.

* We upgrade to PowerBuilder 12.5, Oracle to 11g, Windows servers to Windows 2008.

Real life Test scenario Contd...

• Our Web application did an overkill of logging in the database. We had a maximum concurrent user limit set to 300 in Oracle. We wanted to increase that.

• To help with the tweaking, I ran JMeter scripts that pushed the EAServers to connect to the database continuously to make the database run out of handles - sort of stressing the database to its limits. We were able to tweak the setting and retest for the load condition.

• Eventually, I refactored the code to reduce these loggings. Of course, JMeter came to the rescue again!

Demo

Q&A