TestRunner GUI

Applies to ReadyAPI 3.51, last modified on March 19, 2024

To launch the runner from the ReadyAPI user interface, select the appropriate menu command.

Note: To use the command line runner, you need an active ReadyAPI Test license. If you do not have a license, sign up for a free trial.
  1. Select any node in Functional Tests and select Project > Launch TestRunner from the main menu.

    ReadyAPI: Launching the test runner from the Project menu
  2. Right-click the test case you need and select Launch TestRunner.

    ReadyAPI: Launching the test runner from the context menu

After you selected a menu command, ReadyAPI displays a dialog box, where you can configure the run parameters. You can then use that command string to run the utility from the command line.

Parameters

The configuration dialog organizes the utility parameters in the following tabs:

Basic tab

ReadyAPI: The Basic tab of the test runner

Click the image to enlarge it.

Parameter Description
Test Suite Specifies the test suite to run. <all> means the runner will execute all test suites in your project.
This parameter is identical to the -s command-line argument.
Test Case Specifies the test case to run. <all> means the runner will execute all test cases in the test suite, or, if you have not specified the suite, all test cases in your project.
This parameter is identical to the -c command-line argument.
Enable UI Use this option to enable UI components. Use it if your test uses the UISupport class.
This parameter is identical to the -i command-line argument.
TestRunner Path Specifies the fully-qualified name of the runner file (testrunner.bat or testrunner.sh). By default, you can find that file in the ReadyAPI directory.
Save Project Commands ReadyAPI to save the project before the test run. Use it to ensure all recent changes to the project will be available during the run.
Add Settings Commands the runner to use workspace settings from the readyapi-settings.xml file. You can find that file in the <User directory>/.readyapi directory. To specify another XML settings file, use the -t command-line argument. In the Launch TestRunner dialog, you can specify this argument in the Custom Args tab.
Project Password Specifies the encryption password, if you have encrypted the entire project or some of its custom properties. See Protecting Sensitive Data.
This parameter is identical to the -x command-line argument.
user-settings.xml Password Specifies the password for the XML settings file.
This parameter is identical to the -v command-line argument.
Ignore Errors Commands the runner to ignore errors occurred during the test run. If you select this check box, the runner will not stop the test if an error occurs. However, in this case the runner will not log information on errors. If you keep this check box clear, the runner will stop when an error occurs and will log information about that error.
This parameter identical to the -I command-line argument.
Save After Commands the runner to save the project after the test run finishes. This may be useful if your test stores data to the project.
This parameter is identical to the -S command-line argument.
Run in-process Enable this option to run the test runner in the same process as ReadyAPI. Otherwise, the runner will work in a separate process.

Overrides tab

ReadyAPI: The Overrides tab of the test runner

Click the image to enlarge it.

Parameter Description
Environment Specifies the environment configuration for the test run. This value overrides the environment configuration you have selected for your project in the ReadyAPI toolbar.
This parameter is identical to the -E command-line attribute.
Note: If the test runner is unable to find the specified environment, the test run will fail.
Endpoint Specifies the endpoint to use for requests during the test run. This value overrides endpoints you have specified in request properties in your project.
This parameter is identical to the -e command-line argument.
Host:Port Specifies the host and port to use in requests during the test run. Usage format: host:port. You can specify the host by using its IP address or name. This value overrides the hosts and ports you have specified in request properties in your project.
This parameter is identical to the -h command-line argument.
Username Specifies a user name for authorization tasks during the test run. This value overrides user names you have specified in authorization properties of test requests.
This parameter is identical to the -u command-line argument.
Password Specifies a password for authorization tasks during the test run. This value overrides the password you have specified in authorization properties of test requests.
This parameter is an analogue of the -p command-line argument.
Domain Specifies a domain for authorization tasks during the test run. This value overrides the domains you have specified in authorization properties of test requests.
This parameter is identical to the -d command-line argument.
WSS Password Type Specifies the WS-Security password type. You can select Text or Digest that corresponds to the WSS PasswordText and PasswordDigest types respectively.
This parameter is identical to the -w command-line argument.

Reports tab

ReadyAPI: The Reports tab of the test runner

Click the image to enlarge it.

Properties Description
Print Report Commands the runner to include summary data into the reports ReadyAPI creates.
This parameter is identical to the -r command-line argument.
Default value: enabled.
Export JUnit Results Commands the runner to create a JUnit-style report.
This parameter is identical to the -j command-line argument.
Default value: enabled.
Export JUnit Results with test properties Commands the runner to include JUnit XML reports with test properties to the report.
This parameter is identical to the -J command-line argument.
Export All If this check box is clear, the report will contain information only about errors. If you select this check box, the report will include messages of other types too.
This parameter is identical to the -a command-line argument.
Default value: enabled.
Root Folder The fully-qualified name of the directory that will store test reports ReadyAPI exports. If the directory does not exist, ReadyAPI will create it.
Note: The default value is ${WORKSPACE}, and it is used for Linux and Mac OS agent machines that are run from Jenkins. For Windows agent machines, change the value to %WORKSPACE%.
Tip: If the directory exists, reports’ files in it will be overwritten.
This parameter is identical to the -f command-line argument.
Coverage Report Commands the runner to create test coverage reports (in HTML format).
This parameter is identical to the -g command-line argument.
Open Report Commands the runner to open the reports ReadyAPI has created in your default browser after the test run finishes.
This parameter is identical to the -o command-line argument.
Select Report Type Specifies the type of the report ReadyAPI creates. You can use the following values:
  • Test Case Report – Creates a report in the format you can specify by the Report Format(s) parameter.

  • JUnit-Style HTML Report – Creates a JUnit-style report in the HTML format. See JUnit-Style HTML Reports For Automation. When you use this report type, ReadyAPI ignores the Report Format(s) parameter. The value is selected by default.

  • Data Export – Generates a report in XML format. See Data Export For Automation. To use select this type, specify XML as the Report Format(s) parameter.

This parameter is identical to the -R command-line argument.
Report Format(s) Specifies the format of the report ReadyAPI creates.
Possible values: PDF, XLS, HTML, RTF, CSV, TXT, XML (default).
If you keep this edit box empty, ReadyAPI will use the PDF format. Do not specify periods in format names.
To specify multiple formats, separate them with commas. For example: PDF,XML,CSV.
This parameter is identical to the -F command-line argument.
Slack Access Token Specifies the authentication token of your Slack bot user.
Tip: To get the token, open the Slack settings, switch to the OAuth & Permissions tab, and copy the Bot User OAuth Access Token.
This parameter is used in the -W command-line argument.
To learn more, see Slack Integration.
Slack Channel(s) Specifies a comma-separated list of Slack channels in the #channel-name format, or the IDs of the users the test results will be sent to. You can specify both channels and user IDs.
This parameter is used in the -W command-line argument.
To learn more, see Slack Integration.

For more information about reports and customization, see Reporting.

Properties tab

ReadyAPI: The Properties tab of the test runner

Click the image to enlarge it.

On the Properties tab, you can specify variable values that will take effect during the test run.

To specify variable values, use strings of the following format: <variable-name>=<value>.

Separate multiple name=value pairs with spaces, or put each pair into a new line. If a variable name or value includes spaces, enclose the entire pair in quotes. For example:

file=C:\Work\File.txt
image_to_upload="C:\My Application\image.png"
Properties Description
Global Properties Specifies global property values for the test run. The values you specify will override properties on the Global Properties page of the ReadyAPI preferences.
This parameter is identical to the -G command-line argument.
System Properties Specifies system property values for the test run. The values you specify will override properties’ values in the Help > System Properties dialog.
This parameter is identical to the -D command-line argument.
Project Properties Specifies the project property values for the test run. The values you specify will override project-level properties.
This parameter is identical to the -P command-line argument.

Tags tab

ReadyAPI: The Tags tab of the test runner
Properties Description
Test Suite Tags List of tags you have assigned to test suites within the project. When you check one or more tags, the test runner will include in the test run only the test suites containing all these tags. This parameter is identical to the -T command-line argument.
Test Case Tags List of tags you have assigned to test cases within the project or the test suite.
When you check one or more tags, the test runner will include in the test run only test cases containing all these tags.
This parameter is identical to the -T command-line argument.

For more information about tags, see Working With Tags.

Custom Args tab

ReadyAPI: The Custom Args tab of the test runner

Click the image to enlarge it.

The Custom Args tab allows you to add extra arguments for the runner.

For example, you can command the runner to use a setting file that is different from the default soapui-settings.xml file. To do this, you can enter the following text into the Tool Args edit box:

-tC:\Work\test-settings.xml

Licensing

TestRunner is installed along with ReadyAPI and uses the same modules. It does not require any special license as it uses the existing ReadyAPI Test license you have.

Known issues

SPNEGO-Kerberos authorization will cause tests you start from the test runner to fail. To avoid this issue, add the following code to the end of the first set JAVA_OPTS line in the ReadyAPI.bat file.

-Djavax.security.auth.useSubjectCredsOnly=false

See Also

Automating Functional Tests
Command-Line Arguments
Test Runner Exit Codes

Highlight search results