JMeter in one of the industry leading open source performance tools many organizations use, to assess website/mobile application performance. It is very powerful in assessing load related performance issues, ongoing performance checks against benchmarks, as well as specific scenarios such as tests under peak load, enabling engineering teams isolate and address performance bottlenecks.
Often JMeter is a tool of choice given its ease of use, feature set and it being open source, saving license costs. JMeter is primarily used to generate HTTP Requests on websites, either by simulating browser activity or by calling back end REST APIs.
Here are some additional points to keep in mind in evaluating the tool:
- Ease of use – especially in generating load, processing and analyzing the responses of the requests made. No coding is required at the introductory level – all that is needed is to understand the load test scenario and the infrastructure under test
- Automatic management of parameters such as cookies to help with session state support
- Several configuration options to enable the required load generation (sample requests), analyze the results, including any preprocessing and post-processing of the results. For example, the Beanshell or BSF pre/post processors, response assertion, regular expression extractor, XPath extractor, CSV data set config all provide a lot of test flexibility
- JMeter cloud service – BlazeMeter – this is particularly useful for those that need to scale to a high load without provisioning their own systems
- Programmatic customization – For more lower level detailed programming and fine tuning control of load test actions or processing of results, one could write a JMeter plugin. It is also possible to encapsulate the logic into say a Java class and access it through Beanshell, in the pre/post processor or sampler. These require a deeper understanding of JMeter but is certainly not impossible
- Performance engineer needs to set-up the required load test generators/controller machines for simulating Vusers but when on Blazemeter the user has the ability to push large numbers of simulated Vusers by using/spinning the required load engines/generators
- A JMeter performance engineer has to typically extract results to analyze them over multiple runs; herein blazemeter has an edge with load test matrices that help analyze each test and compare multiple test results via a dashboard
Some of the key questions that rank high when a choice needs to be made include, how high a concurrent load do you need to generate on the system, how technically skilled are your personnel (yes sometimes this is also a factor to keep in mind in your choice, especially when you have an existing deep rooted performance team), how much programmable/script/code based control would you need from the tool, do you prefer more of a GUI based tool or a programmatic code/script based tool?
JMeter continues to mainly cater to those that want a GUI, moderate to low script/code based control/programmability, moderate to low load generation, and for users in the novice to expert range.