Load testing is usually done in a similar environment to the production environment. The application is subject to different load patterns. When performing custom load testing, you can gather performance metrics such as JMeter and aisle-won. This includes application response time and error rates, throughput and requests per second, memory and CPU statistics.
This blog will cover a tutorial on SoapUI load-testing. This blog will provide a deeper understanding of open-source SoapUI testing and discuss SoapUI load test best practices, strategies and considerations.
What is SoapUI Testing?
SoapUI can be used to test SOAP and Web Service functionality. You can also use it to test custom cases and provide complete coverage in one test environment.
You can extend functional API tests and create advanced test cases with a variety of features, such as:
- Click and Run: Create and run load tests in a matter of seconds.
- Multiple Load Strategies Simulated: Simulated multiple load patterns using built-in strategies
- Performance Validation: Use built-in assertions to evaluate service performance.
- Automation and Integration: Additional tools can be easily integrated via a command-line interface.
SoapUI Load Testing Tutorial: Getting Started
SoapUI load testing relies on your functional test cases. This tutorial assumes you have an existing SOAP project. This tool lets you create new SOAP and REST projects as well as import existing packaged programs. This tutorial will show you how to create a new SOAP project with calculator functionality.
Create a new SOAP project and test suite
Click the SOAP icon to access:
Fill in your Project Name and enter the Initial WSDL Path.
We will use the following boxes to create a load-testing suite in just a few clicks.
The Request checkbox is for the functional requirements of the test case, and the Test Suite is for the load testing suite that each test case requires. Complete the dialog box, and then press OK.
Generate the SoapUI Load Testing Suite
SoapUI offers more options to generate the TestSuite since we are creating it automatically. As you can see in the dialog box, SoapUI has many options. The four test cases in this project correspond to the Add, Divide and Multiply, Subtract, functionalities. You can also see that we chose to create a load test for each case separately.
You can also use the load testing suite to test different requests.
On the left menu pane, we see the CalculatorSoapTestSuite, where we added all functional cases to be tested individually. The Add TestCase contains a request for the Add Test Step.
Create Custom Load Testing Steps
In order to create a load testing suite in SoapUI with all cases, click on the CalculatorSoapTestSuite and press CTRL+N to create a new test case:
Click CTRL+N to open a dialogue box that will allow you to name your TestCase. Next, we create a test suite that contains all the calculator test cases. We name it AllTestCases.
Then, we add existing requests to the load testing suite. Click on the AllTestCases highlight in the menu bar to select a new SOAP request:
You can add the SOAP request by selecting the appropriate requests from the drop-down menu.
To validate performance, enter the Name and Add Assertions. This step will be discussed in more detail at a later date:
You can repeat the New TestRequest process in all Test Cases. The TestEditor (left click AllTestCases, hit Enter) will display all Requests as follows. To identify potential errors, you can also run the functional test by clicking the green Play button.
Make and run the SoapUI Load Testing
Now, we have created the Load Test in the AllTestCases suite. To create a new load testing, highlight the Load Tests section and hit CTRL+N.
The name of the new SoapUI Load Test will be automatically created. It will contain all Requests in the Test Case. This is the new interface:
Choose the Load Testing Strategy
Start by choosing the Simple strategy, which requires the following parameters.
- Threads – Simulated number of concurrent users. Although there is no ideal number, open-source SOAPUI is capable of running up to 100 Threads.
- Test Delay – The delay in milliseconds between each TestCase run.
- Random : This is the factor multiplied with Test Delay to randomize the delay between each case.
- Limit – There are three options: Total Runs (seconds), and Runs per thread. This parameter was set to 60 seconds for the entire load test run.
After the test is completed, logs are returned for different metrics.
- min: Minimum time required to load a step (in milliseconds).
- Maximum: Maximum time taken to load a step (in milliseconds).
- Average time taken to complete a load step (in milliseconds).
- Last: Time for the final load phase of the test (millisecond).
- cnt: Number of executions per load-step.
- tps: transactions per second.
- Bytes: Header data volume processed during the load step (bytes).
- bps: Header data volume processed at byte per second (bytes per second) during the load step
- Err: There were many assertions in the load step.
- Rat: % of requests were rejected (percent).
You can plot the results using graph statistics options. You can choose to view each Test Step individually or as a whole (Request: Add/Divide, Multiply/Subtract). Resolution refers to the number of data points that were plotted (250-1000 or the maximum number of simulated data points).
Load Testing Strategies for SoapUI
Now we can review all available load testing strategies using the SOAPUI open-source tool.
- Simple: Performs the test with a configurable delay, as demonstrated earlier.
- Burst : Performs traffic bursts with a specified duration and delay.
- Thread: From start to finish, the number of concurrent users is adjusted between fixed threads.
- Variance : Vary concurrent user traffic during a time period.
Considerations for Load Testing Strategies with SoapUI
When choosing your SoapUI load-testing strategy, there are several important things to keep in mind.
1. The Strategy script option isn’t available in the Free version. However, you can manually run multiple strategies and export data statistics to separate files for further analysis by using the icon below.
2. Read the Settings option to learn how metrics are configured and calculated. The time calculation can be based on actual time, which could be affected by load conditions and the performance of your local machine.
3. TPS (Transactions per Second) are calculated as (1000/avg)*threadcount) by default. This means that the avg for the Simple test strategy will be low because of the Test Delay. Instead, use the actual time passed to calculate the TBS/BPS. This is (time-passed/cnt).
4. Optional options Include Request Write and Response read will capture the exact time, requests and proxies during network communications between client and server.
5. To limit extreme errors, you can use sample intervals or maximum assertions. These values should be determined based on the acceptable thresholds for load testing metrics violations.
Performance Validation
The load test is finally terminated by performing a validation step that includes assertions. LoadTest Assertions can be configured to register errors when certain metric values reach. You can also configure assertions to terminate a test instance when a specified value is reached.
Click the LoadTest Assertions Icon at the bottom of the pane.
Choose the type of assertion you require, and set the threshold values accordingly. To evaluate the results, we deliberately set thresholds that are easily broken during the Load Test.
As expected, violation errors were registered. The graphs show the trajectory pattern and errors throughout the test interval.
Additional scripting
In the Setup Script (before load testing execution) or Teardown Script, you can add custom execution properties. These usage scenarios are possible:
- Initializing resources, such as shared data objects
- Removing metrics log data
- Reporting execution failure conditions, such as exceeding execution times.
- Create shared objects