Run ReadyAPI Tests in TestEngine from Azure DevOps
This topic describes how to run ReadyAPI projects in TestEngine as part of your Azure Pipeline by using the Run ReadyAPI projects in TestEngine extension.
Supported Azure DevOps versions
You can use the Run ReadyAPI projects in TestEngine extension with the following versions of Azure DevOps:
Azure DevOps Services (formerly Visual Studio Team Services)
Azure DevOps Server 2019
1. Prepare ReadyAPI project
Make your ReadyAPI project available to the agents.
Typically, you store your project in a source control system (like Git or Azure Repos) and then configure your build to get the project from source control and place it to the needed folder.
As an alternative, you can place your project in a shared network folder. Make sure your agent has access to that folder.
In addition, if your project uses external files as data sources for your tests (see Data Sources and Data-Driven Testing), make sure your build agent has access to these files.
2. Install the extension
Go to the Azure Marketplace.
Tip
You can open the marketplace directly from Azure DevOps by selecting Browse Marketplace in the top right corner:
Search for the Run ReadyAPI projects in TestEngine extension.
The way you install the extension depends on the Azure DevOps version you use:
3. Configure pipeline
To run ReadyAPI tests from Azure DevOps, perform the following steps:
3.2. Configure pipeline to run tests
Open your Azure DevOps project’s web portal and navigate to the Pipeline hub.
Open an existing pipeline:
– or –
Create a new one:
If your ReadyAPI project is stored in a source control system, configure your pipeline to copy it to the folder to which an agent has access:
To run ReadyAPI tests, add the Run ReadyAPI projects in TestEngine task and specify its parameters:
 |
Option | Description |
|---|---|
Path to project file or directory | Required. The path to your ReadyAPI project. You can also specify the path to a folder with several ReadyAPI projects. In this case, the task will send all of them to TestEngine. |
Test Type | Specifies whether you run a functional or security test. ImportantCurrently, the Run ReadyAPI projects in TestEngine extension does not support load tests. |
Test suite name | A test suite to run. To run all the test suites in the project, leave the field blank. NoteAvailable only if you run a functional test. |
Test case name | The test case to run. Leave the field blank to run all the test cases of the specified test suite, or, if you have not specified a test suite, all the test cases of your project. NoteAvailable only if you run a functional test. |
Security test name | Required. The security test to run. NoteAvailable only if you run a security test. |
Active environment | The environment configuration that the task will use for the test run. See Environments. |
Callback URL | The address where TestEngine will send a JSON report after the test finishes. |
Priority Job | Enable this option to skip all non-priority jobs in the queue (requires administrator rights). |
Server URL | Required. The address of TestEngine. Make sure it is available for the Azure agent. |
Username | Required. The name of a TestEngine user, on behalf of which Azure DevOps runs the project. |
Password | Required. The user’s password. TipTo protect your password, use Azure variables. |
Project file password | The encryption password. It is required if your project is encrypted. |
Endpoint | The endpoint against which the test will run. |
Tags | The tags of the test cases you want to run. |
Publish test results | Enable this option to publish test results in Azure DevOps. |
Save JUnit Report | Enable this option to save the JUnit report as a file to the |
Filename | It is shown only when the Save JUnit Report option is enabled. Specifies the name of the report file. |
 |
To save the build, select Save & queue > Save:
 |
4. Run build and view test result
To run the build:
Select it on the Pipelines page and then click Run pipeline.
– or –
Click
Queue on the Edit Pipeline page.
Switch to the Job page to view the build progress:
 |
The build will run the task that executes your ReadyAPI projects in TestEngine. It will also fetch and publish the test results if the Publish test results option is enabled.
After the build is over, you can view the test results. On the Summary page of the build report, view the Test Results section. It shows the number of failed and passed tests:
 |
Switch to the Tests page to view detailed test results:
 |
On the Test hub, you can view the test run statistics for your team project:
 |
Click a test run to view detailed information.