Load tests are based on functional test cases. Usually, you create a functional test case simulating real user behavior. But, not every functional test can comply with load testing specifics completely. To adopt a functional test for load testing, follow these recommendations:
Use Test Case custom properties rather than Test Suite, Project, or Global ones.
When you start a load test, ReadyAPI generates multiple virtual users going through the test steps simultaneously.
Test Suite, Project, and Global properties are shared between all the virtual users. When you modify a property value in a test case, each virtual user will modify the value when simulating the test case. It causes the value to change multiple times during a load test, which leads to unpredictable behavior and errors.
Also, each virtual user exists in a separate thread that uses its own context containing information on test case properties. ReadyAPI reads data from the context much faster, that is why storing data in the test case custom properties improves performance of your load test.
Do not use the Run TestCase test step.
The Run TestCase test step is not designed for load testing. Because of test step specifics, load tests based on test cases with this test step are not stable and can work incorrectly. We do not recommend that you use this test step in load tests.
By default, each virtual user will use an individual copy of the data source. Set the Shared option of DataSource test step to force all users to use the same copy of the data source. This will exhaust it much faster. Set the Restart Shared option to restart shared data source after it was used and continue the load test. You can configure the relative options directly in the ReadyAPI Performance:
To stop the load test after data source has been exhausted, set the Fail on Empty option. See DataSource Test Step.
For the DataGen test step, you can choose which properties should be shared between virtual users. For example, if you use the generated property with the Number type to create unique IDs, select the Shared check box to make them unique. You can then use a different value from DataGen without selecting the Shared check box. This value will be the same for all the users. See DataGen Test Step.
Share the DataSink test step.
When you run a load test with DataSink, by default, each virtual user that ReadyAPI Performance simulates will have a separate copy of used DataSink. Depending on the storage type you use, this will have the following effect on the test:
JDBC or Data Connection – All test cases will write data to the same database in parallel.
File or Excel – All virtual users will write data to the same file. This may cause concurrency or IO errors. To avoid possible issues, you can share the data storage or configure the test case to write data to a unique file. To do the latter, you can add a context variable to the target file name:
The variable will have unique values for each virtual user (or thread) running the test case. Select the Append check box to add data to the end of the specified file and not to overwrite previous results with new results.
Property – The property will store unique values for each virtual user that runs the test case.
Groovy – You configure your scripts to handle the DataSink storage as you need.
|Note:||We do not recommend using Groovy data sinks in load tests because many parallel threads may handle writing to a file incorrectly that will cause errors.|
There can be situations when you want to change this default behavior and gather data from all virtual users running your load tests to a single DataSink storage. For example, you may want the virtual users to write data to the same Excel file during the load test run.
To do this, share your DataSink storage:
In ReadyAPI Test, open the functional test on which your load test is based.
In the Navigator panel, find the needed test case and its DataSink.
Open DataSink and click in the DataSink editor.
In the DataSink Options dialog, select Shared:
Now, when you run your load test containing a shared DataSink storage, all virtual users in the test will use the same DataSink storage instance.
Do not write a file from Groovy scripts.
ReadyAPI runs each virtual user in a separate thread. Many parallel threads may handle writing to a file incorrectly that will cause errors.
When you get the test case object in a script, use the
Each virtual user works with its own copy of the test case and its properties. The
testRunner.getTestCase() method returns the test case instance specific to this particular virtual user.
If you obtain the test case object in another way (for example, as a child object of the project object), you will get a test case instance stored in the project. Property and field values of this instance may differ from the values you expect to get.