Run ReadyAPI Tests in TestEngine from Azure DevOps

1. Make your ReadyAPI project available to the agents.

   Typically, you store your project in a source control system (like Git or Azure Repos) and then configure your build to get the project from source control and place it to the needed folder.

   As an alternative, you can place your project in a shared network folder. Make sure your agent has access to that folder.

2. In addition, if your project uses external files as data sources for your tests (see Data Sources and Data-Driven Testing), make sure your build agent has access to these files.

1. Go to the Azure Marketplace.

   Tip:	You can open the marketplace directly from Azure DevOps by selecting Browse Marketplace in the top right corner:

2. Search for the Run ReadyAPI projects in TestEngine extension.

3. The way you install the extension depends on the Azure DevOps version you use:

   Azure DevOps Services

   Azure DevOps ServicesAzure DevOps Services

   1. Click Get it free.

   2. Select the organization for which you want to install the extension and click Install.

   To learn more, see Azure DevOps documentation.

   Connected Azure DevOps Server

   Connected Azure DevOps ServerConnected Azure DevOps Server

   1. Click Install.

   2. Select the team project collection to which you want to install the extension.

   3. Click Confirm.

   To learn more, see Azure DevOps documentation.

   Disconnected Azure DevOps Server

   Disconnected Azure DevOps ServerDisconnected Azure DevOps Server

   1. Click Download and save the extension file.

   2. Go to https://{your-azure-devops-server}/tfs/_gallery/manage.

   3. Click Upload new extension.

   4. Browse for the extension file you downloaded earlier. Click Upload.

   To learn more, see Azure DevOps documentation.

To run ReadyAPI tests from Azure DevOps, perform the following steps:

3.1. Create a pipeline

3.2. Configure pipeline to run tests

To run ReadyAPI tests, add the Run ReadyAPI projects in TestEngine task and specify its parameters:

Click the image to enlarge it.

Option	Description
Path to project file or directory	Required. The path to your ReadyAPI project. You can also specify the path to a folder with several ReadyAPI projects. In this case, the task will send all of them to TestEngine.
Test Type	Specifies whether you run a functional or security test. Currently, the Run ReadyAPI projects in TestEngine extension does not support load tests.
Test suite name	A test suite to run. To run all the test suites in the project, leave the field blank. Note: Available only if you run a functional test.
Test case name	The test case to run. Leave the field blank to run all the test cases of the specified test suite, or, if you have not specified a test suite, all the test cases of your project. Note: Available only if you run a functional test.
Security test name	Required. The security test to run. Note: Available only if you run a security test.
Active environment	The environment configuration that the task will use for the test run. See Environments.
Callback URL	The address where TestEngine will send a JSON report after the test finishes.
Priority Job	Enable this option to skip all non-priority jobs in the queue (requires administrator rights).
Server URL	Required. The address of TestEngine. Make sure it is available for the Azure agent.
Username	Required. The name of a TestEngine user, on behalf of which Azure DevOps runs the project.
Password	Required. The user’s password. Tip: To protect your password, use Azure variables.
Project file password	The encryption password. It is required if your project is encrypted.
Endpoint	The endpoint against which the test will run.
Tags	The tags of the test cases you want to run.
Publish test results	Enable this option to publish test results in Azure DevOps.
Save JUnit Report	Enable this option to save the JUnit report as a file to the $(System.DefaultWorkingDirectory) directory.
Filename	It is shown only when the Save JUnit Report option is enabled. Specifies the name of the report file.

Click the image to enlarge it.

3.3. Save pipeline

To save the build, select Save & queue > Save:

Click the image to enlarge it.

4. Run build and view test result

To run the build:

• Select it on the Pipelines page and then click Run pipeline.

– or –

• Click Queue on the Edit Pipeline page.

Switch to the Job page to view the build progress:

Click the image to enlarge it.

The build will run the task that executes your ReadyAPI projects in TestEngine. It will also fetch and publish the test results if the Publish test results option is enabled.

After the build is over, you can view the test results. On the Summary page of the build report, view the Test Results section. It shows the number of failed and passed tests:

Click the image to enlarge it.

Switch to the Tests page to view detailed test results:

Click the image to enlarge it.

On the Test hub, you can view the test run statistics for your team project:

Click the image to enlarge it.

Click a test run to view detailed information.