Applies to ReadyAPI 2.8, last modified on August 16, 2019

About

LoadUI simulates virtual users (or VUs for short). Each user runs a scenario – a sequence of SoapUI test cases, test suites or individual requests. Depending on the test properties, virtual users can run only once during the load test run, or can restart after their scenario is over. The maximum number of concurrent virtual users your test can simulate is limited by your LoadUI license.

The way you set the number of virtual users your test will simulate depends on the Load Allocation property of the test. You can set it on the toolbar of the load test editor:

Load Allocation setting

Click the image to enlarge it.

The property can have one of the following values:

Load Allocation How you set the VU number

Relative

When using this allocation type, you specify the base number of virtual users and the proportion of these base users for each load test. Here is an example:

Relative load allocation

The base load is set to five virtual users:

  • Scenario 1 simulates 25% of base VUs, or one user.
  • Scenario 2 simulates 100% of base VUs, or five users.
  • Scenario 3 simulates 2000% of base VUs, or 100 users.

Equal

This allocation type means you use the same number of virtual users for all the scenarios in your test. In the example below, all the scenarios simulate 5 arriving virtual users:

Equal load allocation

Per Scenario

Select this type when you need to set the load individually for each scenario in your test. To do this, select a scenario, and then enter the desired number of virtual users in the property editor on the right:

Per scenario allocation

Possible issues

Simulation of virtual users consumes memory and processing power of your computer. If these resources are not enough for simulation, you will notice performance decrease or even crashes. Let’s see why this happens and how you can fix these issues.

What resources are important?
  • The amount of RAM available for the Java machine, which ReadyAPI is using. This amount depends on the total amount of RAM on your computer.
    The memory is needed to store request and response data, and to store virtual user results.

  • The processing power of the CPU.
    It is needed to simulate virtual users and to handle their data (each user works in a separate thread in the OS).

What factors affect resource consumption?
  • The number of the currently running virtual users. This number depends on the VUs (or Arriving users) and Load Type properties of your load test, and on the work time of each user scenario.

    • If Load Type is VUs, then the VUs property specifies the number of virtual users working with the tested server concurrently.

    • If Load Type is Rate, then the Arriving users property specifies the number of virtual users that “arrive” to the server every second, minute, or hour.

      If the arrival period is small (say, 1 second), and every user works for a long time (say, a minute), then the number of concurrent users will constantly increase. New users will arrive to the tested server while existing users will still be working on it. LoadUI will reach the memory or performance bounds rather quickly, especially, if the Arriving users property value is large.

  • The size of simulated scenarios. The more requests a scenario includes, the more memory its request and response data occupies.

  • The presence and number of target loops in simulated scenarios. Target loops increase the number of simulated requests and responses, which means they increase memory consumption.

  • File and hard drive operations. Reading and writing files are time-consuming operations as compared to data exchange in memory. The more file operations your functional test cases have, the longer virtual users work.

  • Other applications and services running on the computer where you run your load tests. They consume memory and CPU power, so LoadUI will get less resources for its work.

How to fix issues
  • First, we suggest that you examine the load test properties and check if they meet your needs or were set by mistake. For example, you may need to check if you really need that number of virtual users arriving to the tested server every second.

  • If the properties are correct, then the best solution would be to use multiple computers to generate the needed load. See About Distributed Load Testing for more information on this.

    Another possible option is to run your test on another computer that has more RAM and a more powerful CPU.

  • To generate load from your current computer, try one of the following (or both):

    • Increase the amount of RAM available for ReadyAPI on your computer.

      See instructions

    • Change the load test settings. To do this, check the work time of the test cases your virtual users run, the values of the Load Type and VUs properties of your load test. If the Load Type is Rate , select a reasonable length of the “arrival” period to avoid quick growth of the number of concurrent VUs on your computer.

      How to configure the arrival period

      Also, select an acceptable number of simulated virtual users. See below for more information on how to find this number.

How to find the maximum number of VUs a computer can run

We recommend using about 1,000 virtual users per machine in your load tests.

Note, however, this number is approximate. No formula will give you an exact answer, because the impact of the factors described above is not strictly defined. To find an answer, you have to use the trial-and-error procedure. Run your load test with the desired number of virtual users. If the test fails, decrease the number and run again. If the test passes successfully, increase the number and try again.

Here are some notes on configuring and running this test:

  • Select a reasonably long test duration. It’s quite possible that running a test for a short time will not reveal the issue. On the contrary, long tests are likely to cause issues:

    Setting the test duration
  • Select the metrics to monitor during the test run. We recommend selecting TPS on the Global Metrics chart, and Running VU, Queued > Value and Throughput > TPS on the Statistics page:

    Metrics to monitor on the Global Metrics chart

    Click the image to enlarge it.

    Metrics to monitor on the Statistics page

    Click the image to enlarge it.

    Add metrics to the Statistics page

    TPS stands for Transaction per second. It indicates the number of requests sent. Queued shows the number of queued requests. The Running VU metric shows the number of currently working virtual users.

    In case of memory and performance issues, you will see –

    • Growth of the Running VU value.
    • Growth of the Queued value. This value should be as low as possible, ideally, 0. A larger value indicates issues in the load test’s settings.
    • Decrease of TPS. In case of an issue, the TPS value will be significantly less than the number of currently running VUs.
  • Once you detect a performance issue, take a look at the number of virtual users that are currently running. This is the smallest number of virtual users that cause performance issues on your computer.

    Decrease that number by 10% to get the number of concurrent virtual users that your computer can simulate. Use the resulting number as the maximum load on your computer in the future.

See Also

Load Test Inspector
Load Testing Scenarios
LoadUI Licenses

Highlight search results