Where Value and Innovation Co-exist
Performance Testing
&
Apache JMeter
Narasimha Nallabothula
ValueLabs
Performance Testing COE
Where Value and Innovation Co-exist
Day 1: Agenda
What is Performance Testing?
Why Performance Testing is required?
Types of Performance Testing
JMeter Overview
JMeter Architecture and working process
Download & Setting up the JMeter Environment
Run the JMeter
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Performance Testing is defined as validating the
Speed
Scalability
Reliability
And stability
of a server, network and application
Discipline which uncovers what components of the
application/environment needs to be improved
before it goes to market
ValueLabs | www.valuelabs.com | Confidential
What is
Performance
Testing?
Where Value and Innovation Co-exist
OK, Now I understand what is
Performance testing.
But why Performance Testing
required to my application ?
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Key Types of Performance Testing
Performance Testing
Spike
Test
Load
Test
Stress Test
ValueLabs | www.valuelabs.com | Confidential
Volume
Test
Endurance
Test
Where Value and Innovation Co-exist
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Load Test
Load Test will be conducted to verify EUTs behavior under normal operational capacity.
o It would follow a ramp-up pattern where the load will be gradually increased. A steady
load state will be maintained for specified time and will be gradually ramped-down.
This test will help us to
o Measure and determine the response times and throughput rates to support peak
load
o Find out resource utilization of the servers under normal production load
o Identify functionality errors under load
Is my application is capable
to handle peak production
load? any errors will occur
under load?
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Stress Test
Stress Test will be conducted to determine the stability of a given system or component
beyond normal operational capacity, often to find a breaking point of the system
o Initially, a low load will be applied on EUT and it would be constantly ramped up till it
reaches a breaking point (EUT stops responding or a steep hike in response time is
observed or environmental errors are observed because of load)
This test will help us to
o Identify application issues that arise or become apparent only under extreme
conditions
o Determine the maximum sustainable users and throughput of the EUT/AUT
o Identify at what point the system start degrading/fail and reasons for the failure
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Spike Test
Spike Test will be conducted to determine the stability of a given system when there is a
increase/decrease in user load
This test will help us to
o Find out whether the application can handle significant changes in the load
Say for example, test will be initiated
with 100 concurrent user for a certain
period of time, suddenly the concurred
user load will be increased to 1000 and
then to 1500 users. Like wise the user
load will be decreased to 100 concurrent
users and verify the system returns to
normal operation and retaining the
performance as initiated.
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Endurance/Soak Test
Endurance Test will be conducted to verify if the system can sustain continuous
expected load for prolonged period of time
o
o
Endurance Testing is a subset of Load Test
Test will be performed with defined set of concurrent users for a prolonged
period of time
This test will help us to
Discover memory utilization issues and detect potential leaks (memory leaks) in
EUT under sustained use.
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Volume Test
Volume Test will be conducted to test the application performance under varying database
volumes
This test will help us to
o Measure response times, throughput rates and resource-utilization levels of EUT
under different volumes
o Determine how database acts under various circumstances
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Are there any tools
to test the
performance of my
application?
ValueLabs | www.valuelabs.com | Confidential
Yes! we have, I will tell you few tools
Open Source : Apache JMeter, Open STA
Commercial: HP LoadRunner, Silk
Performer, NeoLoad
Where Value and Innovation Co-exist
JMeter Overview
JMeter is an open source testing tool from Apache
Initially Stefano Mazzocchi developed the JMeter, later it is
expanded by Apache
JMeter is a Java Desktop application developed using Java
Swings
JMeter is purely developed by Java, hence it can be run on
any environment that have JVM
It is extendable and flexible to build your own components
JMeter also used to perform functional and automated tests
JMeter store its test plans in XML format. This means you
can customize a test plan using a text editor
It can be configured in any machine/environment easily
ValueLabs | www.valuelabs.com | Confidential
What is this?
Let me
explore
Where Value and Innovation Co-exist
JMeter Architecture And Working Process
Execution Environment
System Under Test
Virtual
Users
JMeter Console, Monitor and Load
Machine
ValueLabs | www.valuelabs.com | Confidential
Statistics and
Performance Data
Where Value and Innovation Co-exist
Download and Install process of JMeter
Pre-requisite
Install JDK 1.6 or above
Installation
Download the binary or source file from the following location
http://jmeter.apache.org/download_jmeter.cgi
Unzip the downloaded file in desired location
Set the windows environment variables
JAVA_HOME: <Its JDK home just like C:\Program Files\Java\jdk1.6.0_18>
PATH: Append <JDK bin folder path like C:\Program Files\Java\jdk1.6.0_18\bin; >
CLASSPATH: JAVA_HOME\lib
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Run / Launch JMeter
By invoking the below files
OS
File Name
Windows
Jmeter.bat
Linux
Jmeter.sh
Mac
Jmeter.sh
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
JMeter Main / Default GUI
The screen contains the following components
o Test Plan - A test plan describes a series of steps that JMeter will execute when run.
o WorkBench - The WorkBench simply provides a place to temporarily store test elements.
ValueLabs | www.valuelabs.com | Confidential
Where Value and Innovation Co-exist
Day 2: Agenda
Elements of the JMeter
Steps to build a script
Steps to replay the script
ValueLabs | www.valuelabs.com | Confidential
18
Where Value and Innovation Co-exist
Threads defines a collection of users
You can control
The no. of users simulated
The ramp-up and ramp-down
patterns
Duration of the test
Start and stop time test
It will execute before a Sampler
executes
Update variable in Sample
Modify the settings of a sampler
Ex: HTML URL Re-writing modifier
It will execute after a Sampler
executes
It is used to Process the response
data (To retrieve a value from the
response)
Ex: RegExp Extractors, Xpath
Extractors, JDBC Post processors
Samplers
Elements of JMeter
To set the configuration values of all
samplers like
Web Server's hostname
Database URL
Cookie Manager, CSV Dataset
Config etc
Threads (Users)
Config Elements
Timers
Pre Processors
JMeter thread sends request without
stopping between each sampler
Timer elements permit us to define a
delay between requests
If timers not defined, JMeter will
attack with nonstop requests
To control the flow of the JMeter
execution
To apply logics/conditions on JMeter
Samplers
To alter the order of the requests
Logical Controllers
Post Processors
Assertions
Listeners
ValueLabs | www.valuelabs.com | Confidential
It simulates a Users request for a
page (URL)
To deliver different types of requests
against to your server
Ex: HTTP Request Sampler, JDBC
Request, SMTP Request,
Assertions will allow you to validate
the response of the requests
To verify that your application is
returning expected results
You can verify some text, response
code, size of the response etc
To view Samplers result in the form of
tables, graphs, trees or simple text
To view and collect the performance
statistics
Where Value and Innovation Co-exist
Flow of a Web Request
Request
Response
End User
System Under Test
Flow of a Web Request with JMeter Proxy
Request
JMeter
Proxy
Response
End User
System Under Test
JMeter Proxy feature allows you to record the traffic / requests simply
o While users navigate/browse the pages through a standard browser, JMeter
will watch and record the traffic
ValueLabs | www.valuelabs.com | Confidential
20
Where Value and Innovation Co-exist
How to Record
(Contd)
Following are the steps to record a HTTP request
1. Right Click on WorkBench Add Non - Test Elements HTTP Proxy Server
ValueLabs | www.valuelabs.com | Confidential
21
Where Value and Innovation Co-exist
How to Record
(Contd)
The HTTP Proxy Server screen is as shown below
ValueLabs | www.valuelabs.com | Confidential
22
Where Value and Innovation Co-exist
How to Record
(Contd)
2. Open the browser, click on Tools Internet Options Connections LAN
Settings
a) Check the Use a proxy server for your LAN
b) Enter Address localhost
c) Port 8080 ( This is the port number that is configured in HTTP Proxy Server,
ensure that both are same)
d) Click on OK button
ValueLabs | www.valuelabs.com | Confidential
23
Where Value and Innovation Co-exist
How to Record
(Contd)
3. Select WorkBench, right click Add Logic Controller
Select Transaction Controller.
4. Rename Transaction Controller to appropriate page title
like Login page, Inbox etc.
5. Select the appropriate Target Controller in HTTP Proxy
Server while browsing the page during recording.
6. Click on Start button in HTTP Proxy Server.
7. Go to the configured browser and access the application
8. Ensure that the Target Controller will contains browsed
requests.
9. Once the recording is done click STOP button in HTTP
Proxy Server.
10. Save the WorkBench by right clicking on it.
ValueLabs | www.valuelabs.com | Confidential
24
Where Value and Innovation Co-exist
How to Replay
1. Right-click on the Test Plan element and select Add Thread Group . Now the Thread
group is created.
2. Thread Group is a pool of users that will execute a particular test case against your server.
Thread Group configurations will be explained during execution .
3. Right-click on the Thread Group element and select Add Config Element HTTP
Cookie Manager. Now the HTTP Cookie Manager is created.
4.
Copy the Transaction from WorkBench and paste to the
Thread Group as Child.
5.
In order to see the status(pass/fail during execution) of each
Transaction, the test plan should have Listener( Listener are
used to view, save, and read saved test results.).
ValueLabs | www.valuelabs.com | Confidential
25
Where Value and Innovation Co-exist
(Contd)
How to Replay
6. Now the test plan contains the Thread Groups, Logic Controllers, Sampler
generating controllers & Listeners.
7. Click on Run, by entering Number of Threads(Users) as 1 in Thread Group.
8. Check the status in View Results Tree, View Results in Table or Summary Report.
a) Status as Passed then it will appear as
b) Status as Failed then it will appear as
ValueLabs | www.valuelabs.com | Confidential
26
Where Value and Innovation Co-exist
Day 3: Agenda
Correlation (Regular Expressions)
Parameterization
And other script customizations
ValueLabs | www.valuelabs.com | Confidential
27
Where Value and Innovation Co-exist
Correlation
When you record a business scenario, JMeter generates a script that contains the actual values
used during recording. When you replay that script it will send the recorded values unless you
change them
Correlation is used to obtain data which is unique for each run
It is useful to extract some information (like session id, timestamps, url links etc) from the
response of a request
During Recording the script in JMeter
Request 1 : Login
Response for Request 1 along with session id 1234567
Request 2: Session ID 1234567
Response for Request 2
User (Browser)
ValueLabs | www.valuelabs.com | Confidential
Server
Where Value and Innovation Co-exist
During Replay the Script in JMeter
Request 1 : Login
Response for Request 1 along with session id 1112223
Request 2: Session ID 1234567
Session failed as 1112223 is expected by the server
JMeter Script
Server
Solution
Request 1 : Login
Response for Request 1 along with session id 999000
Capture the new session id from the above
response and use it in the next request(s)
Request 2: Session ID 999000
JMeter Script
ValueLabs | www.valuelabs.com | Confidential
Successful response to the Request 2
Server
Where Value and Innovation Co-exist
Correlation
1. Right Click the Request Add Post Processors Regular Expression Extractor
2. Regular expressions are used to search and manipulate text.
3. Allows the user to extract values from a server response using regular expression.
4. Regular Expression Extractor extracts the requested values, generate the template
string, and store the result into the given variable name.
5. The Regular Expression Extractor needs to be added in the previous request to
capture the dynamically changing data in the current request.
6. Determine the pattern where the desired text will find.
ValueLabs | www.valuelabs.com | Confidential
30
Where Value and Innovation Co-exist
Correlation
Reference Name: The name of the variable where the dynamically parameterized value get
stored, EX: Data
Regular Expression: This must contain at least one set of parentheses "()" to capture a portion
of the string, left and right boundary of the parameter to be correlated can also be given. The text
groups that will extracted are enclosed by the characters '(' and ')'. For Ex the expression is <td
id="ID">(.+?)</td>\s*<td id="Name">(.+?)</td>\s*, name of the expression is Data , using this
expression ID and Name is captured in the name like Data_1 (ID)& Data_2(Name) variables
ValueLabs | www.valuelabs.com | Confidential
31
Where Value and Innovation Co-exist
Correlation
Template: Each group of text extracted will be placed as a member of the variable Data,
following the order of each group of pattern enclosed by '(' and ')'. Each group is stored as
refname_g#, where refname is the string you entered as the reference name, and # is the group
number. $1$ to refers to group 1, $2$ to refers to group 2, etc. $0$ refers to whatever the entire
expression matches. For Ex the template for the above example is $1$$2$
Match No: Indicates which match to use. The regular expression may match multiple times
Default Value: If no matches are found, then the value given in this field will be substituted
ValueLabs | www.valuelabs.com | Confidential
32
Where Value and Innovation Co-exist
Parameterization
When you record a business process, JMeter generates a script that contains the actual values
used during recording
Lets say, JMeter generated the following recorded post parameters to login into the application
When you replay the script using multiple users and iterations, you do not want to repeatedly
use the same Username and Password values. Instead, you replace the values with a
parameter
The users substitute the parameter with values from a data source that specified
The data source can be either a file, or internally generated variables
ValueLabs | www.valuelabs.com | Confidential
33
Where Value and Innovation Co-exist
JMeter Parameterization
2. Static Value Parameterization
a) User Parameters
b) Function Helper Dialog (_StringFromFile, _CSVRead)
c) Config Elements CSV Data Set Config
ValueLabs | www.valuelabs.com | Confidential
34
Where Value and Innovation Co-exist
Parameterization
User Parameters
1. Right Click the Request Add Pre Processors User Parameters
ValueLabs | www.valuelabs.com | Confidential
35
Where Value and Innovation Co-exist
Parameterization
User Parameters:
1.
User Parameter are used to send the specify values for User Variables specific to individual threads
2.
Values can be accessed in any test component in the same thread group, using the syntax:
${<ParameterName>}.
3.
Make the number of users in Thread Properties to the number of users assigned in the User Parameter.
ValueLabs | www.valuelabs.com | Confidential
36
Where Value and Innovation Co-exist
Parameterization
Function Helper Dialog : _StringFromFile
1. Create a Parameter txt File in desired/specific location .
2. Click Options Function Helper Dialog in JMeter.
3. Choose the function __StringFromFile.
ValueLabs | www.valuelabs.com | Confidential
37
Where Value and Innovation Co-exist
Parameterization
Function Helper Dialog : _StringFromFile
1. Provide full path of the location of file.
2. Provide a name to the variable
3. Provide start file sequence Number as needed.
4. Click on Generate button.
5. Copy and paste the value generated to the value of the variable which has to be
parameterized in the request.
6. The data will be taken from the file while execution. This function can be used to provide
only one column of data.
ValueLabs | www.valuelabs.com | Confidential
38
Where Value and Innovation Co-exist
Parameterization
Function Helper Dialog : _CSVRead
1. Create a CSV File with the parameters in desired/specific location.
2. Click Options Function Helper Dialog in JMeter.
3. Choose the function __CSVRead.
ValueLabs | www.valuelabs.com | Confidential
39
Where Value and Innovation Co-exist
Parameterization
Function Helper Dialog : _CSVRead
1. Provide full path of the location of file.
2. Provide the column number as needed.
3. Click on Generate button.
4. Copy and paste the value generated to the value of the variable which has to be
parameterized in the request.
5. Column number will start from 0
6. The data will be taken from the file while execution.
Note: StringFromFile function can be used to provide only one column of data, using
_CSVRead & CSV Data Set Config has facility to send multiple column data during
execution.
ValueLabs | www.valuelabs.com | Confidential
40
Where Value and Innovation Co-exist
Parameterization
Config Elements CSV Data Set Config
1. Create a CSV File with the parameters in any location .
2. Right Click Test Group Add Config Element CSV Data Set Config.
ValueLabs | www.valuelabs.com | Confidential
41
Where Value and Innovation Co-exist
Parameterization
Config Elements CSV Data Set Config
1. Name : Descriptive name for this element that is shown in the tree. .
2. Filename : Name of the file to be read
3. File Encoding : Generally its blank for windows
4. Variable Names: List of variable names (comma-delimited)
5. Delimiter: Delimiter to be used to split the records in the file, generally keep, for CSV file
6. Recycle on EOF: Should the file be re-read from the beginning on reaching EOF? (default is true)
7. Stop thread on EOF?: Should the thread be stopped on EOF, if Recycle is false? (default is false)
8. Sharing mode: How to share the CSV file to current thread/ Current thread group / Current thread
9. Values can be accessed in any test component in the same thread group, using the syntax:
${<ParameterName>}
ValueLabs | www.valuelabs.com | Confidential
42
Where Value and Innovation Co-exist
Day 4 : Agenda
Steps to build a Performance Test Plan in JMeter
Run a performance Test in JMeter
Analyze and prepare the report
Demo on JMeter Distributed Testing
Sample Report walkthrough
ValueLabs | www.valuelabs.com | Confidential
43
Where Value and Innovation Co-exist
Test Execution
1.
Thread Group defines a pool of users that will execute a particular test case against your server .
2.
Following are the items in the Thread Group
a.
Name: Name of the thread group
b.
Action to be taken after a Sampler error: If a sampler error occurs select any one option below as per
requirement
i.
Continue - ignore the error and continue with the test
ii.
Start Next Loop - ignore the error, start next loop and continue with the test
iii.
Stop Thread - current thread exits
iv.
Stop Test - the entire test is stopped at the end of any current samples.
v.
Stop Test Now - the entire test is stopped abruptly. Any current samplers are interrupted if possible
c.
Number of Threads: Number of users to simulate
d.
Ramp-up Period: How the number of users to start the action.
e.
Loop Count: Number of times to perform the test case
f.
Start Time & End Time: This is used to when to start the test and when to end.
g.
Duration (seconds): Total duration of the execution.
h.
Startup delay (seconds): Time delay to start the test.
ValueLabs | www.valuelabs.com | Confidential
44
Where Value and Innovation Co-exist
Test Execution Single Scenario
Select Thread group Add Transaction Controller.
Rename the Transaction Controller to appropriate name if transactions are defined in WorkBench.
ValueLabs | www.valuelabs.com | Confidential
45
Where Value and Innovation Co-exist
Test Execution Single Scenario
Move/copy the Samplers to the respective Transaction Controller.
Add Listeners to the Thread Group.
Provide the following data in Thread group
Number of Threads
Action to be taken after a Sampler error.
Ramp-up Period.
Loop Count.
Click on Start under Run or Start Run image.
ValueLabs | www.valuelabs.com | Confidential
46
Where Value and Innovation Co-exist
Test Execution Multiple Scenario
Right click on Test plan Select Merge Select the another script.
ValueLabs | www.valuelabs.com | Confidential
47
Where Value and Innovation Co-exist
Test Execution Multiple Scenario
Add Listeners to the Test Plan.
Provide the following data for each in Thread group
Number of Threads
Action to be taken after a Sampler error.
Ramp-up Period.
Loop Count.
Duration
Start up delay
Click on Start under Run or Start Run image.
ValueLabs | www.valuelabs.com | Confidential
48
Where Value and Innovation Co-exist
Test Results
Thread Group 1 load criteria is as shown in the screen below
ValueLabs | www.valuelabs.com | Confidential
49
Where Value and Innovation Co-exist
Test Results
Thread Group 2 load criteria is as shown in the screen below
ValueLabs | www.valuelabs.com | Confidential
50
Where Value and Innovation Co-exist
Test Results View Results Tree
View Results tree is used for debugging for the parameters that are substituted and the correlated value is passed in
the post request.
Add the View Results Tree listener to the Test plan/Thread group.
ValueLabs | www.valuelabs.com | Confidential
51
Where Value and Innovation Co-exist
Test Results View Results in Table
View Results Table contains Start time, Label, Response time(ms) and Bytes transferred
Add the View Results Table listener to the Test plan/Thread group
ValueLabs | www.valuelabs.com | Confidential
52
Where Value and Innovation Co-exist
Test Results View Summary Table
Summary Report shows Average, Minimum and Maximum response times along with Std deviation , Error % and
throughput
Add the Summary Report listener to the Test plan/Thread group
ValueLabs | www.valuelabs.com | Confidential
53
Where Value and Innovation Co-exist
Thank you !
narasimha.nallabothula@valuelabs.com
Extension : 32167
