Rajeev Hathi | Trigent Vantage

JMeter Regular Expression Extractor Example

Before you delve in to the details of the tool, you can get a bigger picture on the importance of performance engineering that boosts the quality of digital experiences.

In this example, we will demonstrate the use of Regular Expression Extractor post processor in Apache JMeter. We will go about parsing and extracting the portion of response data using regular expression and apply it on a different sampler. Before we look at the usage of Regular Expression Extractor, let’s look at the concept.

1. Introduction – Apache JMeter

Apache JMeter is an open-source Java based tool that enables you to perform functional, load, performance and regression tests on an application. The application may be running on a Web server or it could be a standalone in nature. It supports testing on both client-server and web model containing static and dynamic resources. It supports wide variety of protocols for conducting tests that includes, HTTP, HTTPS, JDBC, FTP, JMS, LDAP, SOAP etc.

A quick look at some of the features:

It provides a comprehensive GUI based workbench to play around with tests. It also allows you to work in a non-GUI mode. JMeter can also be ported on the server allowing to perform tests in a distributed environment.
It provides a concept of template which are pre-defined test plans for various schemes or protocols that can be directly used to create your required test plan.
It enables you to build test plan structurally using powerful features like Thread Group, Controllers, Samplers, Listeners etc.
It provides debugging and error monitoring through effective logging.
It supports parameterized testing through the concept of variables.
It supports creation of different flavors of test plan that includes Web, Database, FTP, LDAP, Web service, JMS, Monitors etc.
It allows for remote testing by having different JMeter instances running as servers across nodes and accessed from a single client application.
It gives you real time test results that covers metrics like latency, throughput, response times, active threads etc.
It enables you to perform testing based on regular expressions and many more other features.

1.1. Regular Expression

Regular expression is a pattern matching language that performs a match on a given value, content or expression. The regular expression is written with series of characters that denote a search pattern. The pattern is applied on strings to find and extract the match. The regular expression is often termed as regex in short. Pattern based searching has become very popular and is provided by all the known languages like Perl, Java, Ruby, Javascript, Python etc. The regex is commonly used with UNIX operating system with commands like grep, ls, awk and editors like ed and sed. The language of regex uses meta characters like . (matches any single character), [] (matches any one character), ^ (matches the start position), $ (matches the end position) and many more to devise a search pattern. Using these meta characters, one can write a powerful regex search pattern with combination of if/else conditions and replace feature. The discussion about regex is beyond the scope of this article. You can find plenty of articles and tutorials on regular expression available on the net.

1.2. Regular Expression Extractor

Regular Expression (regex) feature in JMeter is provided by the Jakarta ORO framework. It is modelled on Perl5 regex engine. With JMeter, you could use regex to extract values from the response during test execution and store it in a variable (also called as reference name) for further use. Regular Expression Extractor is a post processor that can be used to apply regex on response data. The matched expression derived on applying the regex can then be used in a different sampler dynamically in the test plan execution. The Regular Expression Extractor control panel allows you to configure the following fields:

Apply to: Regex extractor are applied to test results which is a response data from the server. A response from the primary request is considered main sample while that of sub request is a sub sample. A typical HTML page (primary resource) may have links to various other resources like image, javascript files, css etc. These are embedded resources. A request to these embedded resources will produce sub samples. An HTML page response itself becomes primary or a main sample. A user has the option to apply regex to main sample or sub samples or both.

Field to check: Regex is applied to the response data. Here you choose what type of response it should match. There are various response indicators or fields available to choose. You can apply regex to plain response body or a document that is returned as a response data. You can also apply regex to request and response headers. You can also parse URL using regex or you can opt to apply regex on response code.

Reference Name: This is the name of the variable that can be further referenced in the test plan using ${}. After applying regex, the final extracted value is stored in this variable. Behind the scenes, JMeter will generate more than 1 variable depending on the match occurred. If you have defined groups in your regex by providing parenthesis (), then it will generate as many variables as number of groups. These variables names are suffixed with the letters _g(n) where n is the group no. When you do not define any grouping on your regex, the returned value is termed as the zeroth group or group 0. Variable values can be checked by using Debug Sampler. This will enable you to verify whether you regular expression worked or not.

Regular Expression: This is the regex itself that is applied on the response data. A regex may or may not have a group. A group is a subset of string that is extracted from the match. For example, if the response data is ‘Hello World’ and my regex is Hello (.+)$, then it matches ‘Hello World’ but extracts the string ‘World’. The parenthesis () applied is the group that is captured or extracted. You may have more than one group in your regex, so which one or how many to extract, is configured through the use of template. See the below point.

Template: Templates are references or pointers to the groups. A regex may have more than one groups. It allows you to specify which group value to extract by specifying the group number as $1$ or $2$ or $1$$2$ (extract both groups). From the ‘Hello World’ example in the above point, $0$ points to the complete matched expression that is ‘Hello World’ and $1$ group points to the string ‘World’. A regex without parenthesis () is matched as $0$ (default group). Based on the template specified, that group value is stored in the variable (reference name).

Match no.: A regex applied to the response data may have more than one matches. You can specify which match should be returned. For example, a value of 2 will indicate that it should return the second match. A value of 0 will indicate any random match to be returned. A negative value will return all the matches.

Default value: The regex match is set to a variable. But what happens when the regex does not match. In such a scenario, the variable is not created or generated. But if you specify a default value then if the regex does not match then the variable is set to the specified default value. It is recommended to provide a default value so that you know whether your regex worked or not. It is a useful feature for debugging your test.

2. Regular Expression Extractor By Example

We will now demonstrate the use of Regular Expression Extractor by configuring a regex that will extract the URL of the first article from the JCG (Java Code Geeks) home page. After extracting the URL, we will use it in a HTTP Request sampler to test the same. The extracted URL will be set in a variable.

Before installing JMeter, make sure you have JDK 1.6 or higher installed. Download the latest release of JMeter using the link here. At the time of writing this article, the current release of JMeter is 2.13. To install, simply unzip the archive into your home directory where you want JMeter to be installed. Set the JAVA_HOME environment variable to point to JDK root folder. After unzipping the archive, navigate to /bin folder and run the command jmeter. For Windows, you can run using the command window. This will open JMeter GUI window that will allow you to build the test plan.

2.1. Configuring Regular Expression Extractor

Before we configure regex extractor, we will create a test plan with a ThreadGroup named ‘Single User’ and a HTTP Request Sampler named ‘JCG Home’. It will point to the server www.javacodegeeks.com. For more details on creating ThreadGroup and related elements, you can view the article JMeter Thread Group Example. The below image shows the configured ThreadGroup (Single User) and HTTP Request Sampler (JCG Home). Next, we will apply the regex on the response body (main sample). When the test is executed, it will ping the web site named www.javacodegeeks.com and return the response data which is a HTML page. This HTML web page contains JCG articles, the title of which is wrapped in a <h2> tag. We will write a regular expression that will match the first <h2> tag and extract the URL of the article. The URL will be part of an anchor <a> tag. Right click on JCG Home sampler and selectAdd -> Post Processors -> Regular Expression Extractor.

The name of our extractor is ‘JCG Article URL Extractor’. We will apply the regex to the main sample and directly on the response body (HTML page). The Reference Name or variable name provided is ‘article_url’. The regex used is <h2 .+?><a href="http://(.+?)".+?</h2>. We will not go into the details of the regex as this is a different discussion thread altogether. In a nutshell, this regex will find or match the first <h2> tag and extract the URL from the anchor tag. It will strip the word http:// and extract only the server part of the URL. The extractor itself is placed in a parenthesis () forming our first group. The Template field is set with the value of $1$ that points to our first group (the URL) and the Match No. field indicates the first match. The Default Value set is the ‘error’. So if our regex fails to match then the variable article_url will hold the value ‘error’. If the regex makes a successful match, then the article URL will be stored in the article_url variable.

We will use this article_url variable in another HTTP Request sampler named JCG Article. Right click on Single UserThreadGroup and select Add -> Sampler -> HTTP Request.

As you can see from the above, the server name is ${article_url} which is nothing but the URL that was extracted from the previous sampler using regex. You can verify the results by running the test.

2.2. View Test Results

To view the test results, we will configure the View Results Tree listener. But before we do that, we will add a Debug Sampler to see the variable and its value being generated upon executing the test. This will help you understand whether your regex successfully matched an expression or failed. Right click on Single User ThreadGroup and select Add -> Sampler-> Debug Sampler.

As we want to debug the generated variables, set the JMeter variables field to True. Next, we will view and verify test results using View Results Tree listener. Right click on Single User ThreadGroup and select Add -> Listener -> View Results Tree.

First let’s look at the output of Debug Sampler response data. It shows our variable article_url and observe the value which is the URL that we extracted. The test has also generated group variables viz. article_url_g0 and article__url_g1. The group 0 is a regular general match and group 1 is the string that is extracted from the general match. This string is also stored in our article_url variable. The variable named article_url_g tells you the no. of groups in the regex. Our regex contained only 1 group (note the sole parenthesis () in our regex). Now lets look at the result of our JCG Article sampler:

The JCG Article sampler successfully made the request to the server URL that was extracted using regex. The server URL was referenced using ${article_url} expression.

3. Conclusion

The regular expression extractor in JMeter is one of the significant feature that can help parse different types of values on different types of response indicators. These values are stored in variables that can be used as references in other threads of the test plan. The ability to devise groups in the regex, capturing portions of matches makes it even more a powerful feature. Regular expression is best used when you need to parse the text and apply it dynamically to subsequent threads in your test plan. The objective of the article was to highlight the significance of Regular Expression Extractor and its application in the test execution.

JMeter Blog Series: JMeter BeanShell Example

Here’s more about load and performance testing using Jmeter.

In this example, we will demonstrate the use of BeanShell components in Apache JMeter. We will go about writing a simple test case using the BeanShell scripting language. These scripts will be part of BeanShell components that we will configure for this example. Before we look at the usage of different BeanShell components, let’s look at the concept.

1. Introduction

Apache JMeter is an open-source Java-based tool that enables you to perform functional, load, performance, and regression tests on an application. The application may be running on a Web server or it could be standalone in nature. It supports testing on both client-server and web models containing static and dynamic resources. It supports a wide variety of protocols for conducting tests that include, HTTP, HTTPS, JDBC, FTP, JMS, LDAP, SOAP, etc.

A quick look at some of the features of Jmeter

It provides a comprehensive GUI based workbench to play around with tests. It also allows you to work in a non-GUI mode. JMeter can also be ported on the server allowing to perform tests in a distributed environment.
It provides a concept of templates which are pre-defined test plans for various schemes or protocols that can be directly used to create your required test plan.
It enables you to build a test plan structurally using powerful features like Thread Group, Controllers, Samplers, Listeners, etc.
It provides debugging and error monitoring through effective logging.
It supports parameterized testing through the concept of variables.
It supports the creation of different flavors of test plan that includes Web, Database, FTP, LDAP, Web service, JMS, Monitors etc.
It allows for remote testing by having different JMeter instances running as servers across nodes and accessed from a single client application.
It gives you real-time test results that covers metrics like latency, throughput, response times, active threads etc.
It enables you to perform testing based on regular expressions and many more other features.

1.1. What is BeanShell?

BeanShell is a scripting language written in Java. It is part of JSR-274 specification. It in some way is an extension to the mainstream Java language by providing scripting capabilities. It is an embedded interpretor that recognizes strongly typed Java syntax and scripting features like shell commands, loose types and method closures (functions as objects). BeanShell aids in quick development and test of Java application. One can use it for quick or rapid prototyping or quickly testing a small functionality or a process. The script can also be embedded in the Java code and invoked using the Interpreter API.

BeanShell can also be used as a configuration language as it supports creation of Java based variables like strings, arrays, maps, collections and objects. It also supports what is called as scripting variables or loosely typed variables. BeanShell scripts can also be written in a standalone mode in a external file which then can be loaded and executed by the Java program. BeanShell also provides the concept of UNIX like shell programming. You can give BeanShell commands interactively in a GUI shell and see the output instantly.

For more details on BeanShell, you can refer to the official website http://www.beanshell.org

1.2. JMeter Beanshell Components

JMeter provides the following components that can be used to write BeanShell scripts

BeanShell Sampler
BeanShell PreProcessor
BeanShell PostProcessor
BeanShell Assertion
BeanShell Listener
BeanShell Timer

Each of these component allows you to write scripts to conduct your test. JMeter will execute the scripts based on the lifecycle order of the components. For example, it will first invoke PreProcessor then Sampler and then PostProcessor and so on. Data can be passed between these components using thread local variables which has certain meaning and context. Every component provides you with pre-defined variables that can be used in the corresponding script.

The following table shows some of the common variables used by the BeanShell components:

Variable name	Description
`ctx`	It holds context information about the current thread that includes sampler and its results.
`vars`	This is a thread local set of variables stored in a map used by BeanShell components in the same thread.
`props`	These are variables loaded as properties from an external file (jmeter.properties) stored in the classpath.
`prev`	It holds the last result from the sampler
`data`	It holds server response data

2. BeanShell By Example

We will now demonstrate the use of BeanShell in JMeter. We will take a simple test case of sorting an array. We will define an array of 5 alphabets (a,b,c,d,e) stored in random order. We will sort the content of the array and convert it into string. After conversion, we will remove the unwanted characters and print the final string value. It should give the output as ‘abcde’.
We will make use of the following BeanShell components to implement our test case:

BeanShell PreProcessor – This component will define or initialize our array.
BeanShell Sampler – This component will sort the array and convert it into string.
BeanShell PostProcessor – This component will strip the unnecessary characters from the string.
BeanShell Assertion – This component will assert our test result (string with sorted content).

Before installing JMeter, make sure you have JDK 1.6 or higher installed. Download the latest release of JMeter using the link here. At the time of writing this article, the current release of JMeter is 2.13. To install, simply unzip the archive into your home directory where you want JMeter to be installed. Set the JAVA_HOME environment variable to point to JDK root folder. After unzipping the archive, navigate to <JMeter_Home>/bin folder and run the command jmeter. For Windows, you can run using the command window. This will open JMeter GUI window that will allow you to build the test plan.

2.1. Configuring BeanShell Sampler

In this component, we will sort the array. But before we sort the array, it needs to be initialized. You will see the initialization routine in the next section when we create the pre-processor component. Let’s first create the BeanShell Sampler component. We will write the code to sort the array after the initialization routine. Right click on Single UserThreadGroup and select Add -> Sampler -> BeanShell Sampler.

We will provide the name of our sampler as ‘Array Sorter’. The Reset Interpreter field value is retained as ‘False’. This field is only necessary when you have multiple BeanShell samplers configured or if you are running a sampler in the loop. The value of true will reset and create a fresh instance of BeanShell interpreter for each sampler. The value of false will create only one BeanShell interpreter that will interpret scripts for all the configured samplers. From the performance perspective, it is recommended to set this field to true if you have long running scripts with multiple samplers. The Parameter field allows you to pass parameters to your BeanShell scripts. It is usually used with external BeanShell script file, but if you are writing script in this component itself then you can use Parameters or bsh.args variable to fetch the parameters. The Parametersvariable will hold the parameters as a string value (retains spaces). The bsh.args variable will hold the parameters as string array. For this example, we are not passing any parameters to the script. The Script file field is used when you have a BeanShell script defined in an external file. It is important to note, this will override any scripts written inline in this component. We will retain the default value for all the above mentioned fields for all the BeanShell components. The finalScript textbox field allows us to write scripts inline in this component itself. It allows you to use certain variables in your scripts. As you can see there is no scripting code currently in this field. We will write the code after our array is initialized in the pre-processor component.

2.2. Configuring BeanShell PreProcessor

Beanshell PreProcessor will be the first component to be executed before your sampler. It becomes a good candidate to perform initialization routines. We will initialize our array, to be sorted, in this component. Right click on Array Sorter sampler and select Add -> Pre Processors -> BeanShell PreProcessor.

We will name the component as ‘Array Initializer’. Let’s see the code in the Script textbox field. First, we are declaring and initializing the array named strArray. It is a loosely typed variable. The values of the array are not in order. Then we make use of the vars variable to store the array by calling putObject() method. The vars variable will be available to all the BeanShell components that are part of this thread. We will fetch the value of vars variable in a ‘Array Sorter’ sampler and perform the sort. In the above section, we created the ‘Array Sorter’ sampler, now we will write the following code in that sampler to sort the array. Click on Array Sorter sampler, in the Script textbox field to write the following code:

First, we get the array using getObject() method of the vars variable. Then we will sort using the Arrays class of Java. The sort() method of the said class will take our array as a parameter and perform the sort. We then convert the array into string by calling Arrays.toString() method. Arrays is a utility class provided by the JDK to perform certain useful operations on array object. We will then put this sorted string as a response data through the use of SampleResult variable. Our sorted string will look like the following: [a, b, c, d, e].

2.3. Configuring BeanShell PostProcessor

The BeanShell PostProcessor will strip the unnecessary characters like ‘[],’. This component will act more like a filter. Right click on Array Sorter sampler and select Add -> Post Processors -> BeanShell PostProcessor.

We will name the component as ‘Array Filter’. The Script textbox field contains the code that strips the unnecessary characters from our string. If you recall, the string was stored as response data by the Array Sorter sampler. Now here we fetch the string using the function getResponseDataAsString() of the prev variable. Next, we use the replace() method of the String class to strip ‘[]’ and ‘,’ characters from the string. We store that string in the vars variable. This string will now be used by BeanShell Assertion component to assert the final result.

2.4. Configuring BeanShell Assertion

Using this component, we will assert the final result value as ‘abcde’. Right click on Array Sorter sampler and select Add ->Assertions -> BeanShell Assertion.

Using the vars variable, we will get the final string and store it in the finalString variable. Then we assert by checking if the final string does not contain the value ‘abcde’ then set the Failure variable to true and provide the failure message using the FailureMessage variable. The output of the test execution can be see in the command window from where you started the JMeter GUI. The below is the console output after running our tests.

3. Conclusion

BeanShell scripting language provides scripting capabilities to the Java language. In JMeter, you can use different BeanShell components to write the test scripts and execute the same. Each component is equipped with useful variables that can be used in the scripts to perform the control flow. The scripting feature adds a powerful and useful dimension to the JMeter testing tool. The objective of the article was to show the usage of common Beanshell components and how one can write test scripts to execute tests.

Know how well your application performs under load. Register for a free primary assessment.

JMeter Blog Series: THROUGHPUT EXAMPLE

In this example, we will demonstrate the concept of throughput as a performance indicator in test execution with JMeter. We will also briefly look into the usage of JMeter’s Throughput Constant Timer. Before understanding the semantics of throughput, let’s look at the concept.

1. Introduction

Apache JMeter is an open source Java based tool that enables you to perform functional, load, performance and regression tests on an application. The application may be running on a Web server or it could be a standalone in nature. It supports testing on both client-server and web model containing static and dynamic resources. It supports wide variety of protocols for conducting tests that includes, HTTP, HTTPS, JDBC, FTP, JMS, LDAP, SOAP etc.

A quick look at some of the features:

It provides a comprehensive GUI based workbench to play around with tests. It also allows you to work in a non-GUI mode. JMeter can also be ported on the server allowing to perform tests in a distributed environment.
It provides a concept of template which are pre-defined test plans for various schemes or protocols that can be directly used to create your required test plan.
It enables you to build test plan structurally using powerful features like Thread Group, Controllers, Samplers, Listeners etc.
It provides debugging and error monitoring through effective logging.
It supports parameterized testing through the concept of variables.
It supports creation of different flavors of test plan that includes Web, Database, FTP, LDAP, Web service, JMS, Monitors etc.
It allows for remote testing by having different JMeter instances running as servers across nodes and accessed from a single client application.
It gives you real time test results that covers metrics like latency, throughput, response times, active threads etc.
It enables you to perform testing based on regular expressions and many more other features.

1.1. What is a Throughput?

Throughput is one component of non functional requirement which falls under performance category and is measured as total no. of transaction or requests in a given time or TPS (transaction per second). It in a way reflects the capacity of the server. The ability of the server in terms of how much load it can take. It is one of the significant indicator that helps in evaluating the performance of application. Maximum throughput is often desirable, though the performance of the system itself cannot be based only on higher throughput. There are certain other indicators like response time, latency etc also needs to be considered when testing the application performance. From the networking perspective, throughput can also mean no. of bytes transfered or sent per second (b/sec or kb/sec). For a typical Web application, throughput is measured as number of requests sent to web server per second.

1.2. Throughput in JMeter

As we know, JMeter is a popular tool to perform the load testing of an application, one of the performance indicator we often verify is the ‘throughput’. We measure throughput by observing the test results. Test results are observed through listener components. The following listener components will allow you to measure or track throughput value:

Aggregate Graph
Aggregate Report
Graph Results
Summary Report

JMeter also provides a useful timer component to configure or set a constant throughput value to test the application load. Its called JMeter Throughput Constant Timer.

2. JMeter Throughput by Example

We will now demonstrate the usage of throughput in the load testing. We will conduct a sample load test with 100 concurrent users (threads) and evaluate the throughput value. We will perform the load test on the JCG home page (www.javacodegeeks.com). The concurrency will be controlled through the use of Ramp-up Period field of ThreadGroup element.

Before installing JMeter, make sure you have JDK 1.6 or higher installed. Download the latest release of JMeter using the link here. At the time of writing this article, the current release of JMeter is 2.13. To install, simply unzip the archive into your home directory where you want JMeter to be installed. Set the JAVA_HOME environment variable to point to JDK root folder. After unzipping the archive, navigate to <JMeter_Home>/bin folder and run the command jmeter. For Windows, you can run using the command window. This will open JMeter GUI window that will allow you to build the test plan.

2.1. Configuring Users and Ramp-up Time

We will define five ThreadGroup elements each with different ramp-up times. Each thread group will be configured with 100 threads (users). The ramp-up period for each thread group will be 0, 15, 25, 35 and 45 respectively. Ramp-up period indicates the time duration for each thread to start. For example, for 100 threads, if the ramp-up period is 100 seconds then each thread will start after (100/100) 1 second. Each thread group will have a HTTP sampler that will make request to the JCG home page (www.javacodegeeks.com). The below figure shows the ThreadGroup configured with 100 threads and ramp-period as 0.

The ThreadGroup name is ‘Use Case 1’. Number of Threads field is set as 100. It signifies 100 users and the Ramp-Up Period is 0. This means that all 100 threads will start at once without any delay. Similarly, you can configure the next four thread groups and name it in the format ‘Use Case [2..5]’. Configure the ramp-up period as 15,25,35,45 respectively for each of the thread group. Name the samplers for each thread group as UC[1..5]. As mentioned, each HTTP sampler will point to JCG home page. Make sure when you run or execute the test, each thread group is executed in sync i.e. one after another. In order to do this, select the Test Plan element and check the Run Thread Groups consecutively field from its control panel (see the image below).

2.2. Observing test results

We will observe the test results using Aggregate Report listener component. Right click on Test Plan element and select Add-> Listener -> Aggregate Report.

Run the test by clicking on the start icon. Let’s observe the througput value for each thread group that is executed.

The result of the first thread group which has the ramp-up period as 0. It means all the threads starts at once and therefore puts instant load on the server. This is not a practical scenario and should not be experimented when load testing in production environment. This has a fairly high throughput but the average request load time is very high (12005). Therefore, this does not qualify as a good performance output.

The result of the second and third provides a fairly balanced throughput and request load time. This is because we have configured a decent ramp-up period (not too low or high) for the threads to start. The result of the fourth and fifth thread group shows decline in the throughput as we had configured higher ramp-up period. So our more balanced performance output comes from second and third result. Remember we are only dealing with 100 users (threads), large number of users will require high ramp-up times. Measuring performance of an application can be a very tricky affair.

Some Points to ponder:

If you want to test with heavy load of users (say, in thousands), one can make good use of distributed testing aspect of JMeter. You could have instances of JMeter running on separate instances and perform distributed testing from a single client node.

One can make use of constant timer in JMeter to simulate delays between successive requests. You can provide a value of300ms which can start a request after 0.3 seconds causing an effective delay which is needed for better throughput.

Ramp-up period of zero is not a good option specially when testing with large no. of users (threads). It can quickly crash your server due to instant and rapid load as all threads start at once. For this example, we are only dealing with 100 users and therefore it is ok to simulate with this kind of 100% concurrency level. Ramp-up period also should not be too high as it makes the throughput indicator void. It means that the server has the capacity but there are not enough threads or user making the request in that short span of time. Ramp-up should be configured in a such a way that the next thread always starts before the prior thread is about to finish. This can give a balanced throughput.

2.3. Constant Throughput Timer

This is a unique timer where you can provide a throughput that is needed for your test. It is a kind of reverse approach to performance testing. Here you perform the load test by specifying the throughput first and try to ensure it fulfills your test criteria. This timer will allow you to specify a constant throughput value but it is not guaranteed that it will maintain that constant throughput during the test. Often you will see throughput coming close to the specified value when running the tests that lasts for longer duration. Apart from specifying the constant throughput value, it also allows you to select the mode of throughput calculation. These are algorithms that calculate delays in proportion with the total number of threads in all the thread groups or in the current thread group. Please note that this timer is only helpful when you perform a test that lasts long.

Let’s configure the Constant Throughput Timer. We will create a ThreadGroup element with 300 users and with the ramp-up time of 50 seconds and loop the test once. We will want to make sure the test takes load of 6 requests per second or 360 requests per minute of throughput. Add the HTTP sampler pointing to the JCG home page. Right click on HTTP Requestsampler and select Add -> Timer -> Constant Throughput Timer.

Set the Target throughput field value as 360. This value is no. of samples or requests per minute. It means we want to achieve a throughput of 6 requests per second. We will observe the test results with Aggregate Report listener component. Set the Calculate Throughput based on field value to ‘all active threads in current thread group’. It is one of the algorithm that calculates the delay period based on the number of threads in the current thread group.

Run the test and see the result.

As you can see, our throughput came close to the value of 6.0 requests per second. Note, if you set very high throughput value, there are chances your server may not able to accept the unusual load. So its alwayz recommended to start with lower value and gradually increase the throughput value until it reaches to a more rational acceptable form.

3. Conclusion

Throughput is one of the important performance indicator when evaluating the application performance. It signifies number of transactions or requests that can be made in a given period of time. It is a useful measurement to check the load capacity of the server. Though one should not purely depend on the throughput metrics and it needs to viewed in conjunction with latency and response times. JMeter provides different listener components to track or observe the throughput value. It also provides the timer feature to explicitly set the throughput value. The objective of the article was to demonstrate the usage of throughput as one performance indicator and its application with the Constant Throughput Timer component.

JMeter Blog Series: Random Variable Example

Here’s the beginning, load/performance testing using JMeter.

In this example, we will demonstrate how to configure Random Variable in Apache JMeter. We will go about configuring a random variable and apply it to a simple test plan. Before we look at the usage of Random variables, let’s look at the concept.

1. Introduction

It provides a comprehensive GUI based workbench to play around with tests. It also allows you to work in a non-GUI mode. JMeter can also be ported on the server allowing to perform tests in a distributed environment.
It provides a concept of templates which are pre-defined test plans for various schemes or protocols that can be directly used to create your required test plan.
It enables you to build test plan structurally using powerful features like Thread Group, Controllers, Samplers, Listeners etc.
It provides debugging and error monitoring through effective logging.
It supports parameterized testing through the concept of variables.
It supports creation of different flavours of test plan that includes Web, Database, FTP, LDAP, Web service, JMS, Monitors etc.
It allows for remote testing by having different JMeter instances running as servers across nodes and accessed from a single client application.
It gives you real time test results that covers metrics like latency, throughput, response times, active threads etc.
It enables you to perform testing based on regular expressions and many more other features.

1.1. Random Number Generation

Most programming languages today has an API that will generate or produce random numbers. The generator algorithm typically produce sequence of numbers which are arbitrary and do not follow any order, structure or format. The algorithm to devise randomness is based on some value also called as seed. The seed drives the sequence generation. Two same seeds will always have same sequence generated. The seed based approach is also termed as pseudo-random number generation.

1.2. Random Variable in JMeter

JMeter allows you to generate random number values and use it in a variable. You can do so through the Random Variable config element. The Random Variable config element allows you set the following parameters:

Variable name: You can provide the name of the variable that can be used in your test plan elements. The random value will be stored in this variable.
Format String: You can specify the format of the generated number. It can be prefixed or suffixed with string. For example, if you want the generator to produce alphanumeric values you can specify the format like SALES_000 (000 will be replaced with the generated random number).
Minimum and Maximum value: You can specify range within which the numbers to be generated. For example, the minimum number can be set as 10 and the maximum number can be set as 50. The generator will produce any number within that range.
Per Thread (User): You can specify whether random generator will be shared by all the threads (users) or each thread will have its own instance of random generator. This can indicated by setting false or true respectively.
Random Seed: You can also specify the seed value for your generator. If the same seed is used for every thread (Per Thread is set to true) then it will produce the same number for each thread.

2. Random Variable By Example

We will now configure the Random Variable config element. Finding test cases for random variables is always a tricky affair. You may have a test case that tests the random number itself, like whether it is in the proper range or the format of the number is valid or not. Another test case could be where you need to provide some random number as part of URL like say order ID (orderId=O122) or page numbers for pagination (my-domain.com/category/apparel/page/5). It may be best suited to perform load testing for such URL pages. We will use the configured variable in a HTTP Request Sampler as part of request URL. As part of this example, we will test Java category pages (1 – 10) of JCG website (www.javacodegeeks.com).
http://www.javacodegeeks.com/category/java/page/2/
The page number 2 on the URL will be fetched using random variable.

2.1. JMeter installation and setup

Before installing JMeter, make sure you have JDK 1.6 or higher installed. Download the latest release of JMeter using the link here. At the time of writing this article, the current release of JMeter is 2.13. To install, simply unzip the archive into your home directory where you want JMeter to be installed. Set the JAVA_HOME environment variable to point to JDK root folder. After unzipping the archive, navigate to /bin folder and run the command jmeter. For Windows, you can run using the command window. This will open JMeter GUI window that will allow you to build the test plan.

2.2. Configuring Random Variable

To configure Random Variable, we have to make use of Config Element option. Right click on Test Plan and select Add ->Config Element -> Random Variable.

We will give the name of the element as ‘Page Counter Variable’. The Variable Name is ‘page_number’. The page_numbervariable will be used in our test plan later. Keep the output format blank. We will set Minimum Value and Maximum Valuefield values as 1 and 10 respectively. It means the numbers so generated will fall between 1 and 10 (both inclusive). Keep the seed option as blank. Retain the value of Per Thread (User) field as False. It means if you configure multiple threads, all the threads will use this same random generator instance.
Next, we will create a ThreadGroup named ‘Single User’ with the Loop Count set as ’10’. We will use only 1 thread (user) for this example. You could experiment with multiple threads to simulate load test. Our main objective of the article is to show how we can configure and use random variable and therefore we will keep it simple to 1 user test. Loop count of value 10 will repeat the test ten times per user.

For our ThreadGroup we will create HTTP Request sampler named ‘JCG Java Category’.

It will point to the server www.javacodegeeks.com. Set the Path value as /category/java/page/${page_number}. You can notice here the use of our variable ${page_number}. As this test will be repeated 10 times (loop count), at runtime thepage_number variable will be substituted with random values between the range of 1 and 10.
You can view the result of the test by configuring View Results Tree listener. Run the test and you will see the following output.

As you can see, every request will generate random page values in the URL.

3. Conclusion

The random variable feature can be handy when you want to load test several pages with URL having parameter values that can be substituted dynamically at runtime. You could also devise other use cases for using random variables. The article provided a brief insight into the Random Variable feature of the JMeter.

A Primer on Microservices

Introduction

Microservices are components of an application developed as independent services. The concept of microservice is all about breaking down your application into independent set of services. Often when writing a monolithic application, the code base can grow quickly and in spite of being well-structured, debugging, performing changes, applying fixes and deploying can become challenging and nightmarish.

One approach obviously is to apply appropriate design patterns like ‘High Cohesion’ that makes every component of an application look more functionally independent and the application itself more cohesive. But this form of cohesiveness is more component driven than business. As you write more code, more dependencies are added and it can break the very fabric of cohesion.

Monolithic applications can be envisioned with the new architectural concept based on microservices. Application can be decomposed into small independent services based on domain functionality. The service code can be a set of business functions that speaks one domain vocabulary. The advantage of course is drawing a strict domain boundary and keeping the codebase small and restricted to that domain functionality. An application overall forms a perspective of small services than one large codebase.

“The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.” – As stated by Martin Fowler

One question often asked at various forums and presentations on microservices is: How small your services should be? A service written in a traditional construct-like language may have more lines of code than say a service written in a more modern scripting-like language. Given this fact, it also depends on how you wish to decompose your application. It’s a classical OOPS Abstraction phenomena! A chessboard of 64 squares, whether you perceive it as 4 large squares of 16 or 16 squares of 4 each. Your service should neither be too small or too big. It should just about focus on the domain functionality that is addressed in few lines of code. The codebase of the service should not grow after a goal of addressing a business functionality is achieved. The term ‘micro’ can be misleading. Breaking your application into a very small fine grained services (probably even breaking each module) could lead to complexity in development and deployment if there arises too much inter-dependencies.

Each service should be an independent entity capable of being deployed in isolation of other service bundles. Often the deployment itself should be distributed in nature and not tied to a single tier. A more prevalent example is of PaaS which offers a platform, to build and deploy your application, as collection of services. Encapsulation is the key when developing microservices. A service should consist of code which can be easily changed without affecting other services. If your service contains common code not tied to a service functionality, it could lead to tighter coupling with other services which can affect the modularity and make the deployment a more complex affair.

A well-defined communication pattern should be devised in order for service to talk to each other. One such pattern is the API based communication. The concept of API has become more prominent with the advent of RESTful services. Each service can be envisioned as a RESTful service which can talk to other service through a well-defined API using a URL. The communication pattern should be technology or language agnostic. Only the relevant aspects of the service should be exposed as API and the rest should be conveniently encapsulated.

Key Advantages of Microservices based Architecture

More technology and tool options

As discussed earlier, each service should be developed as an independent entity and therefore the choice of programming language or technology involved should not matter. We should be able to choose the most viable or appropriate technology option to write the service. This eliminates the culture of writing in only one programming language or follow a specific standard that could lead to a more tightly coupled architecture. Sometimes there are technologies that offers greater performance or flexibility and we should be able to use that to develop our service and not restrict to a particular technology.

Because each service is independent of each other, adapting to new technology or completely replacing the technology for that service is more easy and acceptable. Imagine an application developed using a framework ‘A’ which goes obsolete in few years. Now replacing ‘A’ with the new framework say ‘B’ would involve a complete rewrite of the application. If your application is divided into independent services, one can make a choice of technology or language that makes it more convenient to write the service and even more so if it has to offer greater performance benefit.

Mixing technologies also has its fair share of concerns. Technologies written for one particular environment and services developed around that environment could scale easily but if any one of its service uses the technology not part of that environment it could lead to difficulties in devising a right communication pattern. All in all, a polyglot style of programming should be the order of the day when envisioning a microservices architecture.

Ability to handle failure condition

Systems are made to fail. System or machine failures are handled through appropriate clustering and fault tolerance design. A monolithic application going down can have a wider impact. All the important functions are unavailable until the application is looped back to normal. Recovery from a failure condition often becomes too tedious, complex and slow. The microservices design helps you recover the application quickly and healthily.

As the services are autonomous and independent of each other, one service going down does not impact the whole system. For example, an ecommerce web application could have say two services running for its home page: 1. Listing of products 2. Mini dashboard showing the store activity. If say, the second service goes down, one can still view the product listing while the dashboard can simply flash a message ‘We will be back soon’.

On demand provisioning and scaling

The idea of microservices easily translates into cloud. Every cloud vendor will have an on-demand provisioning facility where you could provision for resource quickly when needed. The same concept applies to services. Each service can be provisioned and therefore auto-scaling can be applied only to that service. You only pay for what you provision. Imagine if you have to provision complete monolithic system and make it auto scaled, it can increase your cost three-fold. With microservices architecture, you only pay for service that you feel is important to provision and scale.

Ease of deployment

We discussed this earlier, a large monolithic application can be tedious to deploy. One change and the complete application undergoes build and deploy process. Often a large codebase takes hell lot of time to build. For many large complex system, it could take hours to build and deploy the change. So the obvious process is to aggregate all the changes and deploy the same at a later time (often scheduled). Now this approach could get error prone. If there is any error found during the test (or for that matter even during build), it has to undergo the bug fixing or rollback process and again undergo a complete build cycle before it is deployed. This affair can get frustrating for the developers and release managers.

With microservices, you can avoid building the system as a whole. You only build service that has undergone changes keeping the rest of the system untouched. It makes your deployment faster, smoother and cleaner. If there occurs any problem within that service, rolling back or fixing it becomes easy as it is done in isolation independent of other services.

Modular approach to development

When you have large codebase, the development process can take a toll. Often a large team is working on different pieces of code that could lead to issues with configuration management. It is always recommended to the divide the application into modules of importance and development team can then be formed to handle each module. Microservices architecture automatically fits into modularity approach. Because each service provides a specific domain functionality, we could have development team structured based on domain modules. It can bring in lot of productivity and efficiency in the overall development process.

Service reusability

Services by definition are self-contained and reusable components. With microservices, we build a service functionality that can be perceived and used by many external applications. The service itself is reusable. Apart from being inherent part of the system or application, it can also be exposed to the outside world. A traditional tightly coupled architecture may have components that can be reused only within the confines of that application. Microservices architecture goes a step forward and allows you to envision a service that can be made reusable enterprise wide and across organizations.

Easily replaceable

Every organization probably will have one large traditional legacy system that is waiting to be decommissioned. Imagine the cost and time associated with the migration. A microservices based architecture makes your system easily replaceable. The whole system is composed of mini services and each service can be easily replaced with the new implementation when required, thereby minimizing the cost, time and increasing the overall application manageability. Rewriting a service is more easy and manageable than rewriting the whole application. One can also quickly decommission an old service and easily replace it with a new service.

SOA and Microservices

Service Oriented Architecture (SOA) is a set of design principles where multiple services form an orchestration to fulfil a business goal. Microservices is one approach towards implementing SOA. The very concept of SOA emerged to tackle the challenges involved in the traditional tightly coupled architectures. The whole objective of SOA is software reusability. The concept of SOA is often too heavyweight to digest. Many organization till today struggle to understand and realize the SOA architecture. SOA itself needs introspection when it comes to messaging protocol like SOAP, service granularity, service interoperability and middleware solutions. Microservices on the other hand is a natural and easy way to achieve SOA. It’s a minified SOA where one takes gradual steps in forming a service based ecosystem.

Conclusion

The article talked about microservices as autonomous components developed as services. They are small domain specific functions as services that together makes up a complete system. They are negotiated using a well-defined API. Some of the advantages of microservice based architecture include ease of deployment, reusability, modularity, easy scaling and manageability.

JMeter Blog Series: Understanding Thread Group

In this example, we will demonstrate the significance of Thread Group in Apache JMeter. We will go about implementing one test plan using JMeter by configuring Thread Group and related elements. We will cover a basic Web application test case that will involve HTTP as a request protocol. Before we dive into the practical usage of Thread Group, let’s look at the concept.

1. Introduction

A quick look at some of the features

It provides a comprehensive GUI based workbench to play around with tests. It also allows you to work in a non-GUI mode. JMeter can also be ported on the server allowing to perform tests in a distributed environment.
It provides a concept of template which are pre-defined test plans for various schemes or protocols that can be directly used to create your required test plan.
It enables you to build test plan structurally using powerful features like Thread Group, Controllers, Samplers, Listeners etc.
It provides debugging and error monitoring through effective logging.
It supports parameterized testing through the concept of variables
It supports creation of different flavors of test plan that includes Web, Database, FTP, LDAP, Web service, JMS, Monitors etc.
It allows for remote testing by having different JMeter instances running as servers across nodes and accessed from a single client application.
It gives you real time test results that covers metrics like latency, throughput, response times, active threads etc.
It enables you to perform testing based on regular expressions and many more other features

1.1. What is a ThreadGroup?

In JMeter, you formulate or create a test plan. ThreadGroup is the most basic element of a test plan. To execute a test, you have to simulate test users who will perform or run the test. Test users are simulated using threads in JMeter. These threads are created as part of ThreadGroup element. These threads execute your tests. One can define how many threads you want to configure for your test using ThreadGroup element. The ThreadGroup element allows you to configure the following:

Number of threads that will initiate request and therefore execute test
Ramp up period for the threads to start
Number of times the test is executed per thread
Schedule a test run

1.2. What you can do with ThreadGroup?

Once the base ThreadGroup element is created, you can continue building the test plan by adding elements underThreadGroup. Some of the important elements that can be added under ThreadGroup are:

Samplers: Samplers are requests to the server made by each user (thread). You can configure requests based on the type of application that you are testing. For web application, you could use HTTP Request sampler, for FTP application, use FTP request sampler and so on.
Controllers: Controllers define the way in which the request or the sampler is processed. For example, you might want to execute a particular request 5 times, then you could use Loop Controller, or you may want to execute alternate requests, in which case you use Interleave Controller and so on. There are plenty of built-in logic controllers provided by JMeter to handle execution of requests or samplers.
Listeners: Listeners listens to the resulting data produced by a test plan during execution. It then produces output that is presented on the screen in a certain way. For example, Graphs Results listeners will show the test results plotted as graphs, View Results listener shows the response and the time taken as a tree and so on. With listeners, you can also configure, where to output your test results, in what format (Text, XML, CSV etc) and what metrics (response message, response time, latency etc) to show.
Config Element: This element is used to configure any default values for the samplers or add configuration details for specific type of sampler. The configuration details could be adding HTTP cookie manager or providing JDBC connection configuration and so on.
Timers: Timers allows you to pause a request before it is sent. When you are testing with multiple requests (threads), it is always a good idea to configure short time delay between each request. Timers can be applied to samplers (request) or controllers to allow you to control specific request execution.

2. ThreadGroup by example

We will now demonstrate the use of ThreadGroup by a building a sample test plan. The test plan that we will create will be for a web application.

2.1. JMeter installation and setup

Before installing JMeter, make sure you have JDK 1.6 or higher installed. Download the latest release of JMeter using the link here. At the time of writing this article, the current release of JMeter is 2.13. To install, simply unzip the archive into your home directory where you want JMeter to be installed. Set the JAVA_HOME environment variable to point to JDK root folder. After unzipping the archive, navigate to <JMeter_Home>/bin folder and run the command jmeter. For Windows, you can run using the command window. This will open JMeter GUI window that will allow you to build the test plan.

2.2. Configuring ThreadGroup

We will now build a test plan for a web application. To keep it simple, we will consider Java Code Geeks (JCG) website as a web application. The first step will be to create users (threads) and that will make http request to the web application. We will create three users that will make a request to two web pages on JCG. On the JMeter GUI left hand pane, select Test Plan, right click and select Add -> Threads (Users) -> ThreadGroup. This will create ThreadGroup element under Test Plan. Now we will configure the properties of ThreadGroup. Select the ThreadGroup that you created under Test Plan, you will see the ThreadGroup control panel on the right side.

We will provide the name of the ThreadGroup as ‘My Sample Test Users’ against the Name field. We will set the value ofNumber of Threads as 3. We will retain the default value for Ramp-Up Period as 1. It indicates that all the 3 threads will take 1 second to start. We will set Loop Count value as 2. It indicates that we will run the tests twice per user (thread).

2.3. Configuring Default Request Properties

As we are building the test plan for a web application, we will deal with HTTP request and configure its default properties. Users (threads) will invoke HTTP request to perform test execution. Select My Sample Test Users Thread Group, right click on it and select Add -> Config Element -> HTTP Request Defaults

Provide the Server Name as ‘www.javacodegeeks.com’. This indicates the base server or domain name where all the requests will be routed. Leave the rest of the fields with their default values.

2.4. Configuring HTTP Request Sampler

In the previous section, we configured default properties for HTTP request. In this section we will actually define HTTP request that will be invoked by the users (threads). If you recall, we mentioned that the user will make request to two pages. We will create first request for the home page of JCG (www.javacodegeeks.com) and the second request for the About page (www.javacodegeeks.com/about). To define the request, we have to use Sampler. Right click on the My Sample Test Users Thread Group and select Add -> Sampler -> HTTP Request

Change the Name field value to ‘JCG Home’. Leave the Server Name field blank, as if we have already set the server name in the HTTP Request Defaults configuration. Set the Path field value to ‘/’.

Similarly, add the second HTTP request. Right click on the My Sample Test Users Thread Group and select Add -> Sampler -> HTTP Request

Change the Name field value to ‘About JCG’. Set the Path field value to ‘/about’

Now when you execute the tests, three users (threads) will invoke two http requests one each for the JCG Home page and the About page.

2.5. Configuring Graph Results Listeners

This will be the last element we will add to our ThreadGroup element. The listener will track and show the graph of request data from performance point of view, as it gets executed. It will also log the request data to the specified file. Let’s look at how to configure this listener. Right click on My Sample Test Users Thread Group and select Add -> Listener -> Graph Results.

Select the file name to store the metrics by clicking on Browse button. You can also configure what needs to be shown and what should be the format of the log file by clicking on Configure button and select/deselecting different indicators.

Run the test plan by selecting Run -> Start from the menu.

3. Conclusion

The above example demonstrated the use of ThreadGroup and its related elements in JMeter. There are plenty of things one can achieve through the use of JMeter from the performance test perspective. The objective of the article was to highlight the significance of Thread Group in building the test plan.