Run TestComplete Tests as Part of a Test Plan

Applies to TestComplete 15.0, last modified on November 17, 2021

Video Tutorial

 

Requirements and prerequisites

1. Prepare your TestComplete project

  1. If your TestComplete project still does not have test cases specified, specify them. To learn how to specify test cases, see the description of the Execution Plan editor.

  2. If your TestComplete project is not added to your team project’s repository, add it. We recommend that you do it directly from TestComplete IDE. This way, only the necessary project files will be added to the repository. See the instructions in the Integration With Source Control Systems section.

2. Link TestComplete test cases to test cases of a team project’s test plan

If your TestComplete test cases specified in the Execution Plan editor of the project are not linked to test cases of your team test plan, link them. For detailed instructions, see:

Link TestComplete Test Cases to Azure DevOps Test Cases

3. Create a build

Connect to your team project. If you still do not have a build for which you will run tests, create it. To learn how to create builds, see docs.microsoft.com/en-us/azure/devops/pipelines.

4. Create a release that will run your TestComplete tests

  1. Open the web portal of your team project. You can navigate to it directly in your web browser, or you can open it from Team Explorer in Visual Studio.

  2. Navigate to the Pipelines > Releases page of your team project.

  3. Select an existing pipeline and open it for editing or create a new pipeline:

    TestComplete integration with Azure DevOps: Create a new pipeline

    Click the image to enlarge it.

  4. Add an artifact that includes your TestComplete project files to the release:

    TestComplete integration with Azure DevOps: Select artifacts

    Click the image to enlarge it.

  5. Select a stage that will run your TestComplete tests or create a new stage.

  6. Add an agent job to the pipeline. Specify agents on which your tests will run. Make sure that the agents meet all the requirements.

    If needed, configure the job to distribute the tests among multiple agents. To learn how to do it, see Run tests in parallel using the Visual Studio Test task. Note: Your tests must be able to run independently from each other.

  7. Add the Enable support for TestComplete tests task. This task installs the TestComplete test adapter to your agent and enables the TestComplete test support:

    TestComplete integration with Azure DevOps: Adding the Enable TestComplete tests task

    Click the image to enlarge it.

    The task is available only if you have the TestComplete Test Adapter extension installed on your server. To learn how to install it, see Install TestComplete Test Adapter Extension.

  8. Select the application that you want to use to run tests: TestComplete or TestExecute.

    Note: If the selected application is not installed on the agent, the one that is installed will be used instead.

  9. Specify when you want to retrieve test logs: only for failed tests, for failed tests and for tests with warnings, or for all tests.

  10. Add the Visual Studio Test task to the stage.

    In the Select tests using drop-down box, select Test run.

    TestComplete integration with Azure DevOps: Configuring a Visual Studio Test task to run a test plan with test cases to which TestComplete tests are assigned

    Click the image to enlarge it.

  11. Specify other release properties, if needed.

  12. Save the changes.

5. Run test cases

  1. Navigate to the Test Plans page of your team project.

  2. Locate the test case that has a TestComplete test assigned to it as automation:

    TestComplete integration with Azure DevOps: Test case with TestComplete test assigned

    Click the image to enlarge it.

  3. Click Run > Run with options.

  4. In the Run with options dialog, do the following:

    • In the Test type and runner drop-down list, select Automated tests using release state.

    • In the Build text box, select a build for which you want to run tests.

    • In the Pipeline drop-down list, select the pipeline you have configured on step 4.

    • In the Stage drop-down list, select a stage that will run your TestComplete tests.

    Running a test case

    Click the image to enlarge it.

  5. Click OK to save the changes and start the run.

  6. To monitor the testing status, click View Test Run.

6. View test results

  1. After a test run is over, you can view the test results:

    TestComplete integration with Azure DevOps: Test summary

    Click the image to enlarge it.

  2. If a test has failed, you can double-click it to view the error message:

    TestComplete integration with Azure DevOps: Detailed test results

    Click the image to enlarge it.

  3. To get the test log, download the log in the .mht format from the Attachments section:

    Depending on the Save logs for setting value, logs could be available only for failed tests, for failed tests and for tests with warnings, or for all tests.

See Also

Integration With Azure DevOps and Team Foundation Server via TestComplete Test Adapter
Link TestComplete Test Cases to Azure DevOps Test Cases
Install TestComplete Test Adapter Extension

Highlight search results