Automating API Tests

Applies to ReadyAPI 3.56, last modified on November 21, 2024

This page is primarily dedicated to automation using ID-based licenses in SLM. Furthermore, it also explores the capabilities of test runner utilities and provides information on how to utilize TestEngine and Docker Image to enhance test functionality.

Runner utilities provided with ReadyAPI

ReadyAPI comes equipped with versatile test runner utilities that allow seamless integration of functional tests into your development process. These utilities enable you to execute tests from your preferred continuous integration (CI) and build tools, streamlining testing and deployment workflows. There are runners for:

  1. Functional Tests – Automates the execution of functional tests to ensure the correctness of API endpoints and application behavior.

    TestRunner allows you to run ReadyAPI functional tests and export results.

    Location – Find the runner in the <ReadyAPI>/bin directory. The file name is testrunner.bat (Windows) or testrunner.sh (Linux and macOS).

  2. Security Tests – Facilitates the automated execution of security tests, verifying the API's resistance to potential vulnerabilities and attacks.

    Security TestRunner allows you to run ReadyAPI security tests from the command line.

    Location – Find the runner in the <ReadyAPI>/bin directory. The file name is securitytestrunner.bat (Windows) or securitytestrunner.sh (Linux and macOS).

  3. Performance Tests – Enables automation of performance tests, assessing API response times, scalability, and resource utilization under varying workloads.

    Performance TestRunner allows you to run ReadyAPI load tests and exports results as configured. You can use it to run load tests from the command line.

    Location – Find the runner in the <ReadyAPI>/bin directory. The file name is loadtestrunner.bat (Windows) or loadtestrunner.sh (Linux and macOS).

  4. Virtual Services – Automates the execution of virtual services, which allows developers to simulate API behavior for testing and development purposes.

    VirtRunner allows you to run virtual APIs (virtual services) from a command line.

    Location – Find the runner in the <ReadyAPI>/bin directory. The file name is virtrunner.bat (Windows) or virtrunner.sh (Linux and macOS).

You can start the runner from the command line or from the ReadyAPI user interface. The latter approach is useful, when you need to build the command line and check the settings.

Running automated tests headlessly

For situations where the testing environment lacks a graphical user interface (UI), ReadyAPI offers different options to manage licensing for automated tests. To run ReadyAPI in a headless mode or in CI/CD build runs, you need floating licenses (online or offline) with virtual machines and Docker.

You cannot use UI based automation scripts on headless machine.

Using ID-based licensing

Connecting to the SmartBear Hosted License Server (Online)

To run ReadyAPI in a headless mode or in CI/CD build runs, you need to specify the SmartBear account to be used for the execution by adding certain parameters to the command line.

Activate with Access Key – Get the access key from the SmartBear Licensing Portal and use it as a command-line parameter, either as -accessKey or -K.

Sample Command

For complete information, refer to this page.

Note: SmartBear License Management's license authentication and request flows were revised in November 2023. Authentication using the username and password is no longer supported for test execution via TestRunners, Jenkins or Azure plugins, and SmartBear-hosted licenses. Username and password are limited to on-prem-hosted licenses only. Access Key is the required option for authentication if using SmartBear-hosted licenses headlessly.
Connecting to a License Server Hosted in Your Network (Offline)

To run ReadyAPI in a headless mode or in CI/CD build runs in private network, you need to install or run an on-premises license server. Before using ReadyAPI in headless mode, the required license must be exported to the on-premises server using the server fingerprint file.

To configure and run headless commands, you can use test runners for Functional, Performance, Security, and Virtual Service testing. The main parameters to specify here are:

  • Server Address -D licenseApiHost=SLM_License_Server_Address

  • Server Port -D licenseApiPort=SLM_License_Server_Port

Once you add both parameters to connect to the on-premises server, you need to include an additional argument to authenticate the user using any of the following approaches.

  1. Add AccessForEveryone flag – If you enable Access for Everyone on SLM on-premises, you no longer need to add Access Key credentials to your command line.

    In that case, you can add the -D licenseApiAccessForEveryone=true flag to your CLI command.

  2. Add Access Key – We recommend you add the license access key as a parameter in the command line.

    Use the -accessKey command-line parameter (or -K key for short).

    Sample Command

  3. Add Email Credentials – Alternatively, once the correct license is assigned to the Test User, you can configure the test runners using your credentials (in email format).

    Use the -email or -U) and -password or -V for short).

    Sample Command

  4. Add Client Credentials – Once the SLM Onpremise License Server is configured for OIDC and the correct license is assigned to the Test User, you can configure the test runners using your credentials.

    Use the --clientId, -ci <Client ID> and --clientSecret, -cs <Client Secret>

    Specifies the Client ID and Client Secret respectively for SLM On-premise license Server when configured for OIDC. Please see here for instructions on configuring SLM on-premise for OIDC and how to access the credentials.

    Note: SLM on-premise 2.0 or later must be configured for the OIDC/Okta environment.

    Sample Command

For complete information, refer to this page.

Note:

When using email and password credentials with the testrunner.sh and testrunner.bat scripts, they default to search for the service account via the "mail" attribute instead of the standard "sAMAccountName." This means users can successfully log in with the Service Account through the Web UI but not with the testrunner.

Customers using LDAP might find that email addresses in Active Directory are stored under the "email" attribute rather than the "mail" attribute.

You can configure SLM On-prem to search for the email field by setting -Dldap.emailField=email in the vmoptions file located in the /bin folder of the installation directory. After changing the configuration, the server needs to be restarted. TestRunner sends email/password requests, which are matched to the field configured by -Dldap.emailField (default is 'mail'). This configuration is supported in version 1.3 of SLM On-prem.

Using Access for Everyone with Headless Testing in Private Networks

There are 2 authorization approaches:

  1. Add flag to CLI command – If you enable Access for Everyone on SLM on-premises, you no longer need to add Access Key credentials to your CLI command configuration for TestRunner, Performance Load Runner, Security Test Runner, and VirtRunner.

    In that case, you can add the -D licenseApiAccessForEveryone=true flag to your CLI command.

  2. Add Access Key to CLI command – If you enable Access for Everyone on SLM on-premises and also need to use Access key credentials to your CLI command configuration, follow the below steps:

    1. Request and install On-Premises Server on your headless machine.

    2. Log in with the following credentials:

      • username: anonymous.user
      • password: <any, but cannot be left blank>
    3. In the top right corner of the On-Premise License Server UI, select > Access Key and copy the access key in the subsequent dialog.
    4. Use the access key in your command line configuration as specified in your product's documentation.

Using file-based licensing

Please note that starting October 2023, ReadyAPI will no longer provide file-based licensing and transition to SmartBear ID-based licensing. The SmartBear ID-based license model offers greater reliability and convenience, and users are encouraged to transition over. You should immediately engage with your SmartBear Account Manager to understand the steps involved in transitioning and getting started. Please read here to learn more.

Note:

On the on-prem server, the DESKTOP token expires after 4 weeks, regardless of usage. To ensure uninterrupted work, replace the value of the authToken (i.e., "DESKTOP" token) with an Access Key. Additionally, update the authToken in both the smarbear.auth and .auth files found in product directory you are using.

For instance, update smarbear.auth, TestComplete/TestComplete.auth, and TestExecute/TestExecute.auth files accordingly.

Docker Image

A ReadyAPI TestRunner Docker image can be used to execute functional tests. Follow the steps below to set up and use ReadyAPI with Docker:

  1. Get the Functional Test Docker Image – Pull the ReadyAPI Functional Test Docker Image from Install Docker image with SoapUI TestRunner.

  2. Project Setup – Place the project in the preferred folder such as "C:\TestDocker\Projects"

  3. Assign Test License – Ensure that you assign the Test or Bundle license to the Test User as per the outlined guidelines.

  4. Request Access Key – Request the Get Access Key for the Test User on the on-prem server.

  5. Run the Docker Container – Execute the following command in the console.

docker run -v="C:\TestDocker\Projects":/project -e SLM_LICENSE_SERVER="<Address for onprem SLM license server>" -e API_KEY="<API key for floating SLM>" -e COMMAND_LINE="'-EDefault environment' -sTestSuite-Main -cTestCase-Main '/%project%/Project-func-test.xml'" -it smartbear/ready-api-soapui-testrunner:<tag>

For complete information, refer to this page.

TestEngine

TestEngine is a standalone runner of ReadyAPI functional tests. You can install it on a dedicated machine in your network and use it as a central hub that runs all the ReadyAPI functional tests in your team. Please see more on this topic here.

See Also

Git Integration
Composite Project Structure
GitHub Website

Highlight search results