JMeter load test and Bamboo CI Server

Regular load testing is essential if you want avoid unexpected surprises and regressions. In this agile, fast-paced world, load testing is becoming more important in continuous integration. Load testing can be integrated into your application building process to allow for:

• Recognize performance regressions early
• Learn how new features can impact your application’s performance
• When versions of the underlying dependencies are changed, you can catch performance problems
• Embrace confidence

This blog post will further explore the integration process with a Bamboo continuous Integration tutorial. We will show you how load tests from Apache JMeter can be integrated into Atlassian’s Bamboo.

What is Atlassian Bamboo?

Atlassian Bamboo is a popular Continuous Integration, Release and Build server.

Because of its popularity, we will be focusing on load testing in your Continuous Integration process using Bamboo.

If your application uses bamboo, continue reading!

Bamboo with JMeter: The Challenges

It is important to understand that JMeter and Bamboo are not compatible out of the box when trying to integrate JMeter and Bamboo. This integration can be achieved by following the steps:

1. To trigger a JMeter test, add a bamboo build step.
2. You can view the build agent’s results on the Bamboo server.
3. These can be used to determine if the build is ready to go.

Although this is possible, it’s not ideal. Automating the analysis step is better because you will get instant notifications when a threshold is exceeded or an SLA violation occurs.

If you are using JMeter with bamboo, it is crucial that you install the Bamboo JMeter AGgregator plugin.

Bamboo Continuous Integration Tutorial: Installing The Bamboo JMeter Aggregator plugin

It is easy to set up the Bamboo JMeter AGgregator Plugin. The plugin can be set up in six easy steps.

1. The Atlassian Marketplace has the most recent version of the JMeter AGgregator plugin.

2. Stop the bamboo server.

3. Copy the atlassian-bamboo-jmeter-aggregator-x.x.x.jar to the /atlassian-bamboo/WEB-INF/lib folder of your Bamboo installation.

4. Start the Bamboo server.

5. Verify that the plugin was successfully installed using the Bamboo Plugin Manager. Click the “Administration” dropdown to select “Add-ons”.

6. Make sure you check the “JMeter AGgregator” in the list.

How to run a JMeter test with bamboo

It’s easy to add a JMeter run to Bamboo.

Bamboo performs tasks and monitors their status, just like any continuous integration system. Bamboo supports many task types. These include simple commands and shell scripts, to more complicated activities such as Visual Studio or Xcode builds.

JMeter can be launched using the non-GUI command-line mode. This is the most popular method.

Let’s first add an empty Command Line task for a JMeter test to your Bamboo project.

1. On the right-hand side, click the “Edit” button (the one with a pencil).
2. Click on the job where you’d like to add your JMeter test.
3. Click on the “Add Task” button.
4. Select a script from the list or enter “script” into the Search input.

Before we begin to fill in the script body, please remember these:

1. The Bamboo build agent should have the test result file located in its working directory. You will need to tell JMeter to save the test results in Bamboo’s Job folder.
2. You must save the test results file in XML format. JMeter defaults to saving its results in CSV format. You will need to modify it.

Add the following lines to your “Script body” section:

/bin/bash -c "/tmp/jmeter/bin/jmeter.sh \-Jjmeter.save.saveservice.output_format=xml \-n -t /tmp/tests/example.jmx \-l $bamboo.build.working.directory/example.jtl"

Let’s break it down:

1. /bin/bash is the system command interpreter.
2. /tmp/jmeter/bin/jmeter.sh = the path to the JMeter executable script.
3. -Jjmeter.save.saveservice.output_format=xml = the property which changes the JMeter result file format to XML. For more information about working with JMeter properties, see the Apache JMeter Properties customization guide
4. /tmp/tests/example.jmx – the path to your JMeter test script.
5. $bamboo.build.working.directory – the Bamboo variable which stands for the current build working directory.

Your task configuration should now look something like the one below. Make sure to adjust the settings above so that they match your JMeter test script locations.

It’s a good idea at this stage to run the first build in order to make sure it passes. You should expect to see something similar:

Let’s now configure the Bamboo JMeter AGgregator plugin to track performance trends and fail conditionally builds based on key metrics. The configuration of the plugin is done in the “Miscellaneous tab” of the Bamboo Job. This tab contains the JMeter test. This tab allows you to specify where to find the JMeter results files, and make arbitrary assertions.

I recommend that you set the value of “Build Log File” to **/*.jtl. When combined with the $bamboo.build.working.directory variable, it will configure Bamboo to store the JMeter test results along with the build working folder and you’ll be able to monitor performance trends across consecutive builds of your application.

To get your initial statistics, you can run your build plan again once you are satisfied with everything. You can find the results of the Bamboo JMeter AGgregator plugin in the “Load Testing Reports” tab of your build plan dashboard.

Clicking the link “Load Test Reports”, will take you to the JMeter Load Test Reports Page. You can view graphs for metrics and KPIs you choose, and track how they change from build to build.

Bamboo CI: Integration of JMeter Load Tests

You should now be able to add JMeter load testing to your Continuous Integration process using Bamboo.

A cron-based schedule can be used to schedule your tests so that they run every night. The Schedule editor is used to create this schedule. You can set the schedule to be daily (times per hour), weekly (days each week), monthly (days every month), or based upon a cron expression.

To schedule a plan building using acron expression:

• Select Build > All plans from the Bamboo header
• Select the plan you want to display in the list.
• Click on the Triggers tab and choose either an existing trigger, or add trigger.

• Choose the Scheduled trigger type.

• You can also enter a trigger description.

Select the Save trigger. Your Bamboo continuous integration tutorial is now complete with JMeter!