About Security TestRunner

Applies to ReadyAPI 3.56, last modified on November 21, 2024

Use the command line runner to run ReadyAPI security tests from the command line.

Start the runner either from the command line or from the ReadyAPI user interface. Use the latter approach if you need to generate a command line and check its settings. See below.

Tip: In addition to automating runs of security tests, you can automate runs of functional tests and of performance tests.

To use the command line runner, you need an active ReadyAPI Test license. If ReadyAPI is run without a license, the command line runner will ask you to specify the path to a license file.

To get a ReadyAPI Test license, sign up for a free trial.

Location

The runner is located in the <ReadyAPI>/bin directory. The file name is securitytestrunner.bat (Windows) or securitytestrunner.sh (Linux and macOS).

Licensing

The security test runner is installed along with ReadyAPI and uses the same modules. It does not require any special license as it uses the existing ReadyAPI Test license you have.

Test runner specifics

ReadyAPI checks the folder specified in the Script Library folder every 5 seconds. If you set the script library folder dynamically, you need to wait for it to be loaded to use it. To do this, add a delay to your script after you command it to load the script library:

project.scriptLibrary = project.getPropertyValue("dataPath")
Thread.sleep(12000)

When you run tests using the command-line runner, ReadyAPI does not open a workspace, so the getWorkspace() method returns null. You need to modify your script to avoid using the workspace object. For example, if you use the workspace object to load another project, try loading it from a file:

If you run a test on a headless machine, add the -Djava.awt.headless=true JVM option. It prevents the test runner from interacting with UI elements. To learn how to set the option, see Modifying JVM Settings.

Parameters

The configuration dialog organizes the utility parameters in the following tabs:

Basic tab

ReadyAPI: Command Line Runner basic tab

Click the image to enlarge it.

Parameter Description
Test Suite and Test Case Specifies for which test suite and test case a security test should run.
The ReadyAPI security testing engine is based on the SoapUI engine, where security tests were part of test cases. The ReadyAPI security testing subsystem supports the SoapUI approach for specifying the security tests to be run. The test suite and test case names are available in the security test editor right after the name of your security test:
ReadyAPI: Test suite and test case names

Click the image to enlarge it.

An alternative approach to specify the security test is to use the Security Test box.
The test suite and test case parameters are analogues of the -s and -c command-line arguments correspondingly.
Security Test Specifies a security test to be run. If this parameter is not specified, the runner executes all the security tests in the specified test case.
This parameter is an analogue of the -n command-line argument.
Specify a security test to be run by using this parameter, or by using a combination of the test suite and test case parameters.
Enable UI Use this option to enable UI-related components. This is required if your test uses the UISupport class.
This parameter is an analogue of the -i command-line argument.
Security TestRunner Path Specifies the fully-qualified name of the runner file (securitytestrunner.bat or .sh). By default, the file is located in the <ReadyAPI> directory.
Save Project This is useful to make sure that all the recent changes to the project are saved before the run.
Add Settings Commands the runner to use workspace settings from the readyapi-settings.xml file that is located in the <User directory>/.readyapi directory. To specify another XML setting file, use the -t command-line argument. In the Launch Security TestRunner dialog, specify this argument in the Custom Args tab.
Project Password Specifies the project password, if your project is encrypted.
This parameter is an analogue of the -x command-line argument.
user-settings.xml Password Specifies the password for the XML setting file.
This parameter is an analogue of the -v command-line argument.
Ignore Errors Commands the runner to ignore errors that occur during the test run. If you select this check box, the runner will not stop the test if an error occurs. However, in this case, the runner will not log information on errors. If you keep this check box clear, the runner will stop when an error occurs and will log information about that error.
This parameter is an analogue of the -I command-line argument.
Save After Commands the runner to save the project after the test run finishes. This may be useful if your test stores data to the project.
This parameter is an analogue of the -S command-line argument.

Overrides tab

ReadyAPI: Command Line Runner Overrides tab

Click the image to enlarge it.

Parameter Description
Environment Specifies the environment configuration for the test run. This value overrides the environment configuration selected for your project in the ReadyAPI toolbar.
This parameter is an analogue of the -E command-line attribute.
Endpoint Specifies the endpoint to be used for requests during the test run. This value overrides endpoints specified in request properties in your project.
This parameter is an analogue of the -e command-line argument.
Host:Port Specifies the host and port to be used in requests during the test run. Usage format: host:port. To specify the host, use its IP address or name. This value overrides the hosts and ports specified in request properties in your project.
This parameter is an analogue of the -h command-line argument.
Username Specifies a user name for authorization tasks during the test run. This value overrides the user names specified in authorization properties of test requests.
This parameter is an analogue of the -u command-line argument.
Password Specifies a password for authorization tasks during the test run. This value overrides the password specified in authorization properties of test requests.
This parameter is an analogue of the -p command-line argument.
Domain Specifies a domain for authorization tasks during the test run. This value overrides the domains specified in authorization properties of test requests.
This parameter is an analogue of the -d command-line argument.
WSS Password Type Specifies the WS-Security password type. Select Text or Digest to set either the PasswordText or the PasswordDigest type respectively.
This parameter is an analogue of the -w command-line argument.

Reports tab

ReadyAPI: Command Line Runner Reports tab

Click the image to enlarge it.

Parameter Description
Print Report Commands the runner to include summary data into the generated reports.
This parameter is an analogue of the -r command-line argument.
Default value: enabled.
Export JUnit Results Commands the runner to create a JUnit-style report.
This parameter is an analogue of the -j command-line argument.
Default value: enabled.
Export All If this check box is clear, the report will contain information only about errors. If the check box is selected, the report will include messages of other types too.
This parameter is an analogue of the -a command-line argument.
Default value: enabled.
Root Folder The fully-qualified name of the directory that will store exported test reports. If the specified directory does not exist, it will be created.
Note: The default value is ${WORKSPACE}, and it is used for Linux and Mac OS agent machines that are run from Jenkins. For Windows agent machines, change the value to %WORKSPACE%.
Tip: If the directory exists, files of the reports in it will be overwritten.
This parameter is an analogue of the -f command-line argument.
Open Report Commands the runner to open the generated reports in your default browser after the test run finishes.
This parameter is an analogue of the -o command-line argument.
Select Report Type Specifies the type of the generated report. You have the following options:

  • Security Issues Report – Creates a security report in the .pdf format with information about all security issues that were detected. For more information, see Security Issues Report.

  • SecurityTest Report – Creates a printable report with information about what scans were performed and their results. See Security Test Report for more information.

  • Data Export – Generates a report in the XML format. See Data Export For Automation. If you select this type, the Report Format(s) parameter must specify XML.

This parameter is an analogue of the -R command-line argument.
Report Format(s) Specifies the format of the generated report. Possible values: PDF, XLS, HTML, RTF, CSV, TXT, XML. If you keep this edit box empty, PDF will be used. Do not specify periods in format names.
To specify multiple formats, separate them with commas, for example: PDF, CSV, XML (default).
This parameter is an analogue of the -F command-line argument.

For more information about reports and customization, see Reporting.

Properties tab

ReadyAPI: Command Line Runner Properties tab

Click the image to enlarge it.

On the Properties tab, specify variable values that will take effect during the test run.

Use strings of the following format: <variable-name>=<value>.

Separate multiple name-value pairs with spaces, or put each pair into a new line. If a variable name or value includes spaces, enclose the entire pair in quotes. For example:

file=C:\Work\File.txt
image_to_upload="C:\My Application\image.png"
Property Description
Global Properties Specifies global property values for the test run. The values you specify will override properties listed in the Global Properties settings of the ReadyAPI preferences.
This parameter is an analogue of the -G command-line argument.
System Properties Specifies system property values for the test run. The values you specify will override these properties’ values you have in the ReadyAPI Help > System Properties dialog.
This parameter is an analogue of the -D command-line argument.
Project Properties Specifies the project property values for the test run. The values you specify will override project-level properties.
This parameter is an analogue of the -P command-line argument.

Custom Args tab

ReadyAPI: Custom Args tab

Click the image to enlarge it.

On The Custom Args tab, add any extra arguments to the runner if you need. For example, to command the runner to use a setting file that is different from default readyapi-settings.xml, enter the following text into the Tool Args edit box:

-tC:\Work\test-settings.xml

Get a generated command line

Via the Get Command Line option

Specify the needed parameters in the dialog and click Get Command Line.

The Get Command Line button

Click the image to enlarge it.

ReadyAPI will generate the command line and show it in a separate dialog.

The dialog with the command line

Click the image to enlarge it.

To copy the text to the clipboard, select it with the mouse and click Copy to Clipboard.

Via the runner launch

After you specify the desired launch parameters in the dialog, click Launch to start the runner:

ReadyAPI: Command Line Runner dialog

Click the image to enlarge it.

You will see the test log:

ReadyAPI: Test log window

Click the image to enlarge it.

After the test run is over, you will see the generated command line at the beginning of the test log:

ReadyAPI: The command line in the test log

Click the image to enlarge it.

Select the text in the window and copy it to the clipboard.

See Also

Security Test Runner Command-Line Arguments
Automating Test Runs

Highlight search results