Load performance testing using JMeter

Software Reliability Testing or load performance testing is a field within software testing & performance testing services that tests a software’s ability to function in a given environmental conditions for a particular period of time. While there are several software testing tools for evaluating performance and reliability, I have focused on JMeter for this blog.

JMeter or The Apache JMeter™ application as it is popularly known is an open source software. It is a  hundred percent pure Java application designed for load testing, functional testing and performance testing.

Given below are some guidelines on using JMeter to test the reliability of a software product:

Steps to download and install JMeter for load performance testing

JMeter, as a simple .zip file,  can be downloaded from the below mentioned URL:

http://jmeter.apache.org/download_jmeter.cgi

Pre-requisite is to have Java 1.5 or higher version already installed in the machine.

Unzip the downloaded file in a required location, go to bin folder and double click on Jmeter.bat file.

If Jmeter UI opens up then the installation has been successful. If it does not open, then Java might not be installed/configured or the downloaded files may be corrupted.

The next step is to configure Jmeter to listen to browser. Open up the Command prompt under bin location of Jmeter and run Jmeter batch file. This will open up simple UI of Jmeter as already seen.

If you are accessing the internet via Proxy Server then this will not work when recording scripts. We need to by-pass the proxy server and to do this we need to start Jmeter by adding two parameters, –H proxy server name/IP address and –P port number. Adding these will look like the following image:

The next step is to configure the browser to `listen’ to Jmeter while recording the scripts. For this blog, I will be using Firefox browser

  1. Open Firefox browser
  2. Go to options->Advanced
  3. Select Network tab and click on Settings
  4. A connection settings tab will be opened under the `select manual proxy’ configuration and give HTTP proxy as localhost and port as 8080.

After having configured proxy settings, we need  to have Jmeter Certificate installed on the required browser.

The Certificate will be in Jmeter bin folder, under the file name “ApacheJMeterTemporaryRootCA.crt”. Given below are steps to be followed to  install the Certificate in Firefox browser

  1. Open Firefox browser
  2. Go to options->Advanced
  3. Select certificate tab and click on View certificates
  4. Under Authorities tab click on Import
  5. Go to bin folder of Jmeter and select the certificate under it (ApacheJMeterTemporaryRootCA.crt)
  6. Check “Trust this CA to identify website” option and click on Ok.

Note : If you do not find the certificate in the bin folder directory then you can generate it by running the `HTTP Recorder’. Every time  you run the recorder, the certificate will be generated and there is no need to replace the certificate every time.

Recording of Script for web-based applications:

Steps to record a sample test script:

  1. Open up Jmeter by running Jmeter.bat
  2. Right click on Test Plan->Add->Threads(Users)->Thread Group
  3. Right Click on Thread Group->Add->Logic Controller->Recording Controller
  4. Right Click on WorkBench->Add->Non-Test Elements->HTTP(s) Test Script Recorder
  5. Right Click on HTTP(s) Test Script Recorder->Add->Listener->View Results Tree

After adding the components, Jmeter UI would look like the following image:

  1. Click on HTTP(s) Test Script Recorder and again click on Start.  Jmeter Certificate will be generated with a pop- up informing the same. Click on `Ok’ button on the pop up.
  2. Open up the browser, for which Jmeter proxy is configured and go to the URL which is under Test and execute your manual test script with which you want to determine performance.
  3. Once you are done with all your manual test, come back to Jmeter UI Click on HTTP(s) Test Script Recorder and click on Stop. This will stop recording.
  4. Click on the icon for recording script to view all your recorded HTTP samples, as follows. You can then rename Recording script controller
  5. You can view details of the page recorded under “View Results Tree Listener”. By using these values you can  determine the assertions to put in your original run.
  6. Now Click on “Thread Group” and configure the users you want to run. Also Right click on Thread Group->Add->Listener->Summary Report and also add View Results Tree.
  7. Make sure to Tick check box “Errors” under View Results Tree, or else it will take up huge memory while running your scripts as it captures almost everything
  8. Now you can run your recorded scripts! Just click on Run/Play button on the top toolbar of Jmeter UI

Analyzing the test run result:

While the scripts are running, results/timing will be captured under “Summary Report Listener”. Summary report will look the following after your test run has been completed. You can save the report by clicking on “Save Data Table”

Below are details of each keyword:

  • average : Average is the average response time for that particular http request. This response time is in milliseconds.
  • aggregate_report_min : Min denotes the minimum response time taken by the http request. Time is in milliseconds.
  • aggregate_report_max : Max denotes  the maximum response time taken by the http request. Time is in milliseconds.
  • aggregate_report_stddev : How many exceptional cases were found which were deviating from the average value of the receiving time.
  • aggregate_report_error% : This denotes the error percentage in samples during run.
  • aggregate_report_rate : How many requests per second does the server handle. Larger is better.
  • average_bytes : average response size.  Lower the average number, greater is the performance.

Read Other Blogs on Load Testing

Getting Started with Load Testing of Web Applications using JMeter

JMeter Regular Expression Extractor Example

Before you delve in to the details of the tool, you can get a bigger picture on the importance of performance engineering that boosts the quality of digital experiences.

In this example, we will demonstrate the use of Regular Expression Extractor post processor in Apache JMeter. We will go about parsing and extracting the portion of response data using regular expression and apply it on a different sampler. Before we look at the usage of Regular Expression Extractor, let’s look at the concept.

1. Introduction – Apache JMeter

Apache JMeter is an open-source Java based tool that enables you to perform functional, load, performance and regression tests on an application. The application may be running on a Web server or it could be a standalone in nature. It supports testing on both client-server and web model containing static and dynamic resources. It supports wide variety of protocols for conducting tests that includes, HTTP, HTTPS, JDBC, FTP, JMS, LDAP, SOAP etc.

A quick look at some of the features:

  • It provides a comprehensive GUI based workbench to play around with tests. It also allows you to work in a non-GUI mode. JMeter can also be ported on the server allowing to perform tests in a distributed environment.
  • It provides a concept of template which are pre-defined test plans for various schemes or protocols that can be directly used to create your required test plan.
  • It enables you to build test plan structurally using powerful features like Thread Group, Controllers, Samplers, Listeners etc.
  • It provides debugging and error monitoring through effective logging.
  • It supports parameterized testing through the concept of variables.
  • It supports creation of different flavors of test plan that includes Web, Database, FTP, LDAP, Web service, JMS, Monitors etc.
  • It allows for remote testing by having different JMeter instances running as servers across nodes and accessed from a single client application.
  • It gives you real time test results that covers metrics like latency, throughput, response times, active threads etc.
  • It enables you to perform testing based on regular expressions and many more other features.

1.1. Regular Expression

Regular expression is a pattern matching language that performs a match on a given value, content or expression. The regular expression is written with series of characters that denote a search pattern. The pattern is applied on strings to find and extract the match. The regular expression is often termed as regex in short. Pattern based searching has become very popular and is provided by all the known languages like Perl, Java, Ruby, Javascript, Python etc. The regex is commonly used with UNIX operating system with commands like grep, ls, awk and editors like ed and sed. The language of regex uses meta characters like . (matches any single character), [] (matches any one character), ^ (matches the start position), $ (matches the end position) and many more to devise a search pattern. Using these meta characters, one can write a powerful regex search pattern with combination of if/else conditions and replace feature. The discussion about regex is beyond the scope of this article. You can find plenty of articles and tutorials on regular expression available on the net.

1.2. Regular Expression Extractor

Regular Expression (regex) feature in JMeter is provided by the Jakarta ORO framework. It is modelled on Perl5 regex engine. With JMeter, you could use regex to extract values from the response during test execution and store it in a variable (also called as reference name) for further use. Regular Expression Extractor is a post processor that can be used to apply regex on response data. The matched expression derived on applying the regex can then be used in a different sampler dynamically in the test plan execution. The Regular Expression Extractor control panel allows you to configure the following fields:

Apply to: Regex extractor are applied to test results which is a response data from the server. A response from the primary request is considered main sample while that of sub request is a sub sample. A typical HTML page (primary resource) may have links to various other resources like image, javascript files, css etc. These are embedded resources. A request to these embedded resources will produce sub samples. An HTML page response itself becomes primary or a main sample. A user has the option to apply regex to main sample or sub samples or both.

Field to check: Regex is applied to the response data. Here you choose what type of response it should match. There are various response indicators or fields available to choose. You can apply regex to plain response body or a document that is returned as a response data. You can also apply regex to request and response headers. You can also parse URL using regex or you can opt to apply regex on response code.

Reference Name: This is the name of the variable that can be further referenced in the test plan using ${}. After applying regex, the final extracted value is stored in this variable. Behind the scenes, JMeter will generate more than 1 variable depending on the match occurred. If you have defined groups in your regex by providing parenthesis (), then it will generate as many variables as number of groups. These variables names are suffixed with the letters _g(n) where n is the group no. When you do not define any grouping on your regex, the returned value is termed as the zeroth group or group 0. Variable values can be checked by using Debug Sampler. This will enable you to verify whether you regular expression worked or not.

Regular Expression: This is the regex itself that is applied on the response data. A regex may or may not have a group. A group is a subset of string that is extracted from the match. For example, if the response data is ‘Hello World’ and my regex is Hello (.+)$, then it matches ‘Hello World’ but extracts the string ‘World’. The parenthesis () applied is the group that is captured or extracted. You may have more than one group in your regex, so which one or how many to extract, is configured through the use of template. See the below point.

Template: Templates are references or pointers to the groups. A regex may have more than one groups. It allows you to specify which group value to extract by specifying the group number as $1$ or $2$ or $1$$2$ (extract both groups). From the ‘Hello World’ example in the above point, $0$ points to the complete matched expression that is ‘Hello World’ and $1$group points to the string ‘World’. A regex without parenthesis () is matched as $0$ (default group). Based on the template specified, that group value is stored in the variable (reference name).

Match no.: A regex applied to the response data may have more than one matches. You can specify which match should be returned. For example, a value of 2 will indicate that it should return the second match. A value of 0 will indicate any random match to be returned. A negative value will return all the matches.

Default value: The regex match is set to a variable. But what happens when the regex does not match. In such a scenario, the variable is not created or generated. But if you specify a default value then if the regex does not match then the variable is set to the specified default value. It is recommended to provide a default value so that you know whether your regex worked or not. It is a useful feature for debugging your test.

2. Regular Expression Extractor By Example

We will now demonstrate the use of Regular Expression Extractor by configuring a regex that will extract the URL of the first article from the JCG (Java Code Geeks) home page. After extracting the URL, we will use it in a HTTP Request sampler to test the same. The extracted URL will be set in a variable.

Before installing JMeter, make sure you have JDK 1.6 or higher installed. Download the latest release of JMeter using the link here. At the time of writing this article, the current release of JMeter is 2.13. To install, simply unzip the archive into your home directory where you want JMeter to be installed. Set the JAVA_HOME environment variable to point to JDK root folder. After unzipping the archive, navigate to /bin folder and run the command jmeter. For Windows, you can run using the command window. This will open JMeter GUI window that will allow you to build the test plan.

2.1. Configuring Regular Expression Extractor

Before we configure regex extractor, we will create a test plan with a ThreadGroup named ‘Single User’ and a HTTP Request Sampler named ‘JCG Home’. It will point to the server www.javacodegeeks.com. For more details on creating ThreadGroup and related elements, you can view the article JMeter Thread Group Example. The below image shows the configured ThreadGroup (Single User) and HTTP Request Sampler (JCG Home). Next, we will apply the regex on the response body (main sample). When the test is executed, it will ping the web site named www.javacodegeeks.com and return the response data which is a HTML page. This HTML web page contains JCG articles, the title of which is wrapped in a <h2> tag. We will write a regular expression that will match the first <h2> tag and extract the URL of the article. The URL will be part of an anchor <a> tag. Right click on JCG Home sampler and selectAdd -> Post Processors -> Regular Expression Extractor.

The name of our extractor is ‘JCG Article URL Extractor’. We will apply the regex to the main sample and directly on the response body (HTML page). The Reference Name or variable name provided is ‘article_url’. The regex used is <h2 .+?><a href="http://(.+?)".+?</h2>. We will not go into the details of the regex as this is a different discussion thread altogether. In a nutshell, this regex will find or match the first <h2> tag and extract the URL from the anchor tag. It will strip the word http:// and extract only the server part of the URL. The extractor itself is placed in a parenthesis () forming our first group. The Template field is set with the value of $1$ that points to our first group (the URL) and the Match No. field indicates the first match. The Default Value set is the ‘error’. So if our regex fails to match then the variable article_url will hold the value ‘error’. If the regex makes a successful match, then the article URL will be stored in the article_url variable.

We will use this article_url variable in another HTTP Request sampler named JCG Article. Right click on Single UserThreadGroup and select Add -> Sampler -> HTTP Request.

As you can see from the above, the server name is ${article_url} which is nothing but the URL that was extracted from the previous sampler using regex. You can verify the results by running the test.

2.2. View Test Results

To view the test results, we will configure the View Results Tree listener. But before we do that, we will add a Debug Sampler to see the variable and its value being generated upon executing the test. This will help you understand whether your regex successfully matched an expression or failed. Right click on Single User ThreadGroup and select Add -> Sampler-> Debug Sampler.

As we want to debug the generated variables, set the JMeter variables field to True. Next, we will view and verify test results using View Results Tree listener. Right click on Single User ThreadGroup and select Add -> Listener -> View Results Tree.

First let’s look at the output of Debug Sampler response data. It shows our variable article_url and observe the value which is the URL that we extracted. The test has also generated group variables viz. article_url_g0 and article__url_g1. The group 0 is a regular general match and group 1 is the string that is extracted from the general match. This string is also stored in our article_url variable. The variable named article_url_g tells you the no. of groups in the regex. Our regex contained only 1 group (note the sole parenthesis () in our regex). Now lets look at the result of our JCG Article sampler:

The JCG Article sampler successfully made the request to the server URL that was extracted using regex. The server URL was referenced using ${article_url} expression.

3. Conclusion

The regular expression extractor in JMeter is one of the significant feature that can help parse different types of values on different types of response indicators. These values are stored in variables that can be used as references in other threads of the test plan. The ability to devise groups in the regex, capturing portions of matches makes it even more a powerful feature. Regular expression is best used when you need to parse the text and apply it dynamically to subsequent threads in your test plan. The objective of the article was to highlight the significance of Regular Expression Extractor and its application in the test execution.

JMeter Blog Series: JMeter BeanShell Example

Here’s more about load and performance testing using Jmeter.

In this example, we will demonstrate the use of BeanShell components in Apache JMeter. We will go about writing a simple test case using the BeanShell scripting language. These scripts will be part of BeanShell components that we will configure for this example. Before we look at the usage of different BeanShell components, let’s look at the concept.

1. Introduction

Apache JMeter is an open-source Java-based tool that enables you to perform functional, load, performance, and regression tests on an application. The application may be running on a Web server or it could be standalone in nature. It supports testing on both client-server and web models containing static and dynamic resources. It supports a wide variety of protocols for conducting tests that include, HTTP, HTTPS, JDBC, FTP, JMS, LDAP, SOAP, etc.

A quick look at some of the features of Jmeter

  • It provides a comprehensive GUI based workbench to play around with tests. It also allows you to work in a non-GUI mode. JMeter can also be ported on the server allowing to perform tests in a distributed environment.
  • It provides a concept of templates which are pre-defined test plans for various schemes or protocols that can be directly used to create your required test plan.
  • It enables you to build a test plan structurally using powerful features like Thread Group, Controllers, Samplers, Listeners, etc.
  • It provides debugging and error monitoring through effective logging.
  • It supports parameterized testing through the concept of variables.
  • It supports the creation of different flavors of test plan that includes Web, Database, FTP, LDAP, Web service, JMS, Monitors etc.
  • It allows for remote testing by having different JMeter instances running as servers across nodes and accessed from a single client application.
  • It gives you real-time test results that covers metrics like latency, throughput, response times, active threads etc.
  • It enables you to perform testing based on regular expressions and many more other features.

1.1. What is BeanShell?

BeanShell is a scripting language written in Java. It is part of JSR-274 specification. It in some way is an extension to the mainstream Java language by providing scripting capabilities. It is an embedded interpretor that recognizes strongly typed Java syntax and scripting features like shell commands, loose types and method closures (functions as objects). BeanShell aids in quick development and test of Java application. One can use it for quick or rapid prototyping or quickly testing a small functionality or a process. The script can also be embedded in the Java code and invoked using the Interpreter API.

BeanShell can also be used as a configuration language as it supports creation of Java based variables like strings, arrays, maps, collections and objects. It also supports what is called as scripting variables or loosely typed variables. BeanShell scripts can also be written in a standalone mode in a external file which then can be loaded and executed by the Java program. BeanShell also provides the concept of UNIX like shell programming. You can give BeanShell commands interactively in a GUI shell and see the output instantly.

For more details on BeanShell, you can refer to the official website http://www.beanshell.org

1.2. JMeter Beanshell Components

JMeter provides the following components that can be used to write BeanShell scripts

  • BeanShell Sampler
  • BeanShell PreProcessor
  • BeanShell PostProcessor
  • BeanShell Assertion
  • BeanShell Listener
  • BeanShell Timer

Each of these component allows you to write scripts to conduct your test. JMeter will execute the scripts based on the lifecycle order of the components. For example, it will first invoke PreProcessor then Sampler and then PostProcessor and so on. Data can be passed between these components using thread local variables which has certain meaning and context. Every component provides you with pre-defined variables that can be used in the corresponding script.

The following table shows some of the common variables used by the BeanShell components:

Variable name Description
ctx It holds context information about the current thread that includes sampler and its results.
vars This is a thread local set of variables stored in a map used by BeanShell components in the same thread.
props These are variables loaded as properties from an external file (jmeter.properties) stored in the classpath.
prev It holds the last result from the sampler
data It holds server response data

2. BeanShell By Example

We will now demonstrate the use of BeanShell in JMeter. We will take a simple test case of sorting an array. We will define an array of 5 alphabets (a,b,c,d,e) stored in random order. We will sort the content of the array and convert it into string. After conversion, we will remove the unwanted characters and print the final string value. It should give the output as ‘abcde’.
We will make use of the following BeanShell components to implement our test case:

  • BeanShell PreProcessor – This component will define or initialize our array.
  • BeanShell Sampler – This component will sort the array and convert it into string.
  • BeanShell PostProcessor – This component will strip the unnecessary characters from the string.
  • BeanShell Assertion – This component will assert our test result (string with sorted content).

Before installing JMeter, make sure you have JDK 1.6 or higher installed. Download the latest release of JMeter using the link here. At the time of writing this article, the current release of JMeter is 2.13. To install, simply unzip the archive into your home directory where you want JMeter to be installed. Set the JAVA_HOME environment variable to point to JDK root folder. After unzipping the archive, navigate to <JMeter_Home>/bin folder and run the command jmeter. For Windows, you can run using the command window. This will open JMeter GUI window that will allow you to build the test plan.

2.1. Configuring BeanShell Sampler

In this component, we will sort the array. But before we sort the array, it needs to be initialized. You will see the initialization routine in the next section when we create the pre-processor component. Let’s first create the BeanShell Sampler component. We will write the code to sort the array after the initialization routine. Right click on Single UserThreadGroup and select Add -> Sampler -> BeanShell Sampler.

We will provide the name of our sampler as ‘Array Sorter’. The Reset Interpreter field value is retained as ‘False’. This field is only necessary when you have multiple BeanShell samplers configured or if you are running a sampler in the loop. The value of true will reset and create a fresh instance of BeanShell interpreter for each sampler. The value of false will create only one BeanShell interpreter that will interpret scripts for all the configured samplers. From the performance perspective, it is recommended to set this field to true if you have long running scripts with multiple samplers. The Parameter field allows you to pass parameters to your BeanShell scripts. It is usually used with external BeanShell script file, but if you are writing script in this component itself then you can use Parameters or bsh.args variable to fetch the parameters. The Parametersvariable will hold the parameters as a string value (retains spaces). The bsh.args variable will hold the parameters as string array. For this example, we are not passing any parameters to the script. The Script file field is used when you have a BeanShell script defined in an external file. It is important to note, this will override any scripts written inline in this component. We will retain the default value for all the above mentioned fields for all the BeanShell components. The finalScript textbox field allows us to write scripts inline in this component itself. It allows you to use certain variables in your scripts. As you can see there is no scripting code currently in this field. We will write the code after our array is initialized in the pre-processor component.

2.2. Configuring BeanShell PreProcessor

Beanshell PreProcessor will be the first component to be executed before your sampler. It becomes a good candidate to perform initialization routines. We will initialize our array, to be sorted, in this component. Right click on Array Sorter sampler and select Add -> Pre Processors -> BeanShell PreProcessor.

We will name the component as ‘Array Initializer’. Let’s see the code in the Script textbox field. First, we are declaring and initializing the array named strArray. It is a loosely typed variable. The values of the array are not in order. Then we make use of the vars variable to store the array by calling putObject() method. The vars variable will be available to all the BeanShell components that are part of this thread. We will fetch the value of vars variable in a ‘Array Sorter’ sampler and perform the sort. In the above section, we created the ‘Array Sorter’ sampler, now we will write the following code in that sampler to sort the array. Click on Array Sorter sampler, in the Script textbox field to write the following code:

First, we get the array using getObject() method of the vars variable. Then we will sort using the Arrays class of Java. The sort() method of the said class will take our array as a parameter and perform the sort. We then convert the array into string by calling Arrays.toString() method. Arrays is a utility class provided by the JDK to perform certain useful operations on array object. We will then put this sorted string as a response data through the use of SampleResult variable. Our sorted string will look like the following: [a, b, c, d, e].

2.3. Configuring BeanShell PostProcessor

The BeanShell PostProcessor will strip the unnecessary characters like ‘[],’. This component will act more like a filter. Right click on Array Sorter sampler and select Add -> Post Processors -> BeanShell PostProcessor.

We will name the component as ‘Array Filter’. The Script textbox field contains the code that strips the unnecessary characters from our string. If you recall, the string was stored as response data by the Array Sorter sampler. Now here we fetch the string using the function getResponseDataAsString() of the prev variable. Next, we use the replace() method of the String class to strip ‘[]’ and ‘,’ characters from the string. We store that string in the vars variable. This string will now be used by BeanShell Assertion component to assert the final result.

2.4. Configuring BeanShell Assertion

Using this component, we will assert the final result value as ‘abcde’. Right click on Array Sorter sampler and select Add ->Assertions -> BeanShell Assertion.

Using the vars variable, we will get the final string and store it in the finalString variable. Then we assert by checking if the final string does not contain the value ‘abcde’ then set the Failure variable to true and provide the failure message using the FailureMessage variable. The output of the test execution can be see in the command window from where you started the JMeter GUI. The below is the console output after running our tests.

3. Conclusion

BeanShell scripting language provides scripting capabilities to the Java language. In JMeter, you can use different BeanShell components to write the test scripts and execute the same. Each component is equipped with useful variables that can be used in the scripts to perform the control flow. The scripting feature adds a powerful and useful dimension to the JMeter testing tool. The objective of the article was to show the usage of common Beanshell components and how one can write test scripts to execute tests.

Know how well your application performs under load. Register for a free primary assessment.

Exit mobile version