Set Virtual User Number

ReadyAPI simulates virtual users (or VUs for short). Each user runs a scenario – a sequence of functional test cases, test suites or individual requests. Depending on the test properties, virtual users can run only once during the load test run, or can restart after their scenario is over. The maximum number of concurrent virtual users your test can simulate is limited by your ReadyAPI Performance license.

The way you set the number of virtual users your test will simulate depends on the Load Allocation property of the test. You can set it on the toolbar of the load test editor:

Click the image to enlarge it.

The property can have one of the following values:

Load Allocation	How you set the VU number
Relative	When using this allocation type, you specify the base number of virtual users and the proportion of these base users for each load test. Here is an example: The base load is set to five virtual users: Scenario 1 simulates 25% of base VUs, or one user. Scenario 2 simulates 100% of base VUs, or five users. Scenario 3 simulates 2000% of base VUs, or 100 users.
Equal	This allocation type means you use the same number of virtual users for all the scenarios in your test. In the example below, all the scenarios simulate 5 arriving virtual users:
Per Scenario	Select this type when you need to set the load individually for each scenario in your test. To do this, select a scenario, and then enter the desired number of virtual users in the property editor on the right:

Simulation of virtual users consumes memory and processing power of your computer. If these resources are not enough for simulation, you will notice performance decrease or even crashes. Let’s see why this happens and how you can fix these issues.

What resources are important?

• The amount of RAM available for the Java machine, which ReadyAPI is using. This amount depends on the total amount of RAM on your computer.
  The memory is needed to store request and response data, and to store virtual user results.

• The processing power of the CPU.
  It is needed to simulate virtual users and to handle their data (each user works in a separate thread in the OS).

What factors affect resource consumption?

• The number of the currently running virtual users. This number depends on the VUs (or Arriving users) and Load Type properties of your load test, and on the work time of each user scenario.

  • If Load Type is VUs, then the VUs property specifies the number of virtual users working with the tested server concurrently.

  • If Load Type is Rate, then the Arriving users property specifies the number of virtual users that “arrive” to the server every second, minute, or hour.

    If the arrival period is small (say, 1 second), and every user works for a long time (say, a minute), then the number of concurrent users will constantly increase. New users will arrive to the tested server while existing users will still be working on it. ReadyAPI will reach the memory or performance bounds rather quickly, especially, if the Arriving users property value is large.

• The size of simulated scenarios. The more requests a scenario includes, the more memory its request and response data occupies.

• The presence and number of target loops in simulated scenarios. Target loops increase the number of simulated requests and responses, which means they increase memory consumption.

• File and hard drive operations. Reading and writing files are time-consuming operations as compared to data exchange in memory. The more file operations your functional test cases have, the longer virtual users work.

• Other applications and services running on the computer where you run your load tests. They consume memory and CPU power, so ReadyAPI Performance will get less resources for its work.

How to fix issues

• First, we suggest that you examine the load test properties and check if they meet your needs or were set by mistake. For example, you may need to check if you really need that number of virtual users arriving to the tested server every second.

• If the properties are correct, then the best solution would be to use multiple computers to generate the needed load. See About Distributed Load Testing for more information on this.

  Another possible option is to run your test on another computer that has more RAM and a more powerful CPU.

• To generate load from your current computer, try one of the following (or both):

  • Increase the amount of RAM available for ReadyAPI on your computer.

    See instructions

    See instructionsSee instructions

    • If you run load tests from the ReadyAPI UI:

      1. Open the following file in a text editor:

         • Windows
         • Linux
         • macOS

         File: <ReadyAPI>\bin\ReadyAPI.vmoptions

         File: <ReadyAPI>\bin\ReadyAPI.vmoptions

         File: <ReadyAPI Contents>\vmoptions.txt

      2. Find the line that starts with -Xmx:

         …
         -Xmx8000m
         …

         It specifies the maximum amount of RAM to be available for the Java machine that ReadyAPI uses. The letter m means megabytes.

      3. Increase the number to give more RAM to the Java machine.

         Save the changes.

    • If you use the loadtestrunner.bat/.sh command-line tool:

      1. Open the following file in a text editor:

         • Windows
         • Linux
         • macOS

         File <ReadyAPI>\bin\loadtestrunner.bat

         File: <ReadyAPI>\bin\loadtestrunner.sh

         File: <ReadyAPI Contents>\java\app\bin\loadtestrunner.sh

      2. Find the line that defines the READY_XMX variable:

         • Windows
         • Linux
         • macOS

         rem set READY_XMX=4000m

         #READY_XMX=4000m

         #READY_XMX=4000m

         It specifies the maximum amount of RAM available for the Java machine that the runner uses. The postfix m means megabytes.

      3. Uncomment the line and set a larger number of megabytes that will be available for the Java machine.

         Save the changes.

  • Change the load test settings. To do this, check the work time of the test cases your virtual users run, the values of the Load Type and VUs properties of your load test. If the Load Type is Rate , select a reasonable length of the “arrival” period to avoid quick growth of the number of concurrent VUs on your computer.

    How to configure the arrival period

    How to configure the arrival periodHow to configure the arrival period

    1. In your load test editor, select Per Scenario in the Load Allocation drop-down list.

    2. Click the scenario title, and

    3. Change the time unit and the number of arriving users in the editor on the right:

       

       Click the image to enlarge it.

    Also, select an acceptable number of simulated virtual users. See below for more information on how to find this number.

How to find the maximum number of VUs a computer can run

We recommend using about 1,000 virtual users per machine in your load tests.

Note, however, this number is approximate. No formula will give you an exact answer, because the impact of the factors described above is not strictly defined. To find an answer, you have to use the trial-and-error procedure. Run your load test with the desired number of virtual users. If the test fails, decrease the number and run again. If the test passes successfully, increase the number and try again.

Here are some notes on configuring and running this test:

• Select a reasonably long test duration. It’s quite possible that running a test for a short time will not reveal the issue. On the contrary, long tests are likely to cause issues:

  

• Select the metrics to monitor during the test run. We recommend selecting TPS on the Global Metrics chart, and Running VU, Queued > Value and Throughput > TPS on the Statistics page:

  

  Click the image to enlarge it.

  

  Click the image to enlarge it.

  Add metrics to the Statistics page

  Add metrics to the Statistics pageAdd metrics to the Statistics page

  You should add metrics to the Statistics page before starting your load test:

  1. Go to the Statistics page of the load test editor. Click Add Chart, if you have no charts on the page.

     

     Click the image to enlarge it.

  2. In the subsequent dialog box, type the metric name (or its part) in the Search box, and then select the metric’s check box in the tree of the available values:

     

     Click the image to enlarge it.

  3. Search for and add other metrics.

     Click OK when you are done.

  TPS stands for Transaction per second. It indicates the number of requests sent. Queued shows the number of queued requests. The Running VU metric shows the number of currently working virtual users.

  In case of memory and performance issues, you will see –

  • Growth of the Running VU value.
  • Growth of the Queued value. This value should be as low as possible, ideally, 0. A larger value indicates issues in the load test’s settings.
  • Decrease of TPS. In case of an issue, the TPS value will be significantly less than the number of currently running VUs.

• Once you detect a performance issue, take a look at the number of virtual users that are currently running. This is the smallest number of virtual users that cause performance issues on your computer.

  Decrease that number by 10% to get the number of concurrent virtual users that your computer can simulate. Use the resulting number as the maximum load on your computer in the future.