Run TestComplete Tests as Part of Build Pipeline - Requirements

Applies to TestComplete 14.50, last modified on April 22, 2021
This approach to running TestComplete tests as part of a build pipeline is obsolete. We do not recommend using it. Starting from version 14.20, to run TestComplete tests as part of a build, you can use the TestComplete test adapter. To learn more, see Integration With Azure DevOps and Team Foundation Server.

In our explanation, we assume that you create tests on one computer, but run them on another computer (on a build computer or on test agents).

The computers where you create and configure your test projects and manage your team builds must have the following software installed:

Team Server

You can run TestComplete tests as part of a team build if you use one of the following:

  • Azure DevOps Services (formerly known as Visual Studio Team Services or VSTS).

    Note: Use self-hosted (private) agents to run TestComplete tests. Running TestComplete tests on Microsoft-hosted agents is not supported.

  • Azure DevOps Server 2019

  • Team Foundation Server 2010–2018.

    Note: Support for Team Foundation Server 2010–2013 is deprecated. It will be removed in one of the future TestComplete versions.
Computer Where You Prepare Visual Studio Test Projects

  • One of the following Microsoft Visual Studio versions with the Team Explorer component:

    • Microsoft Visual Studio 2015–2019 (Community, Enterprise or Professional edition).

    • Microsoft Visual Studio 2013 (Community, Ultimate, Premium, or Professional edition).

    • Microsoft Visual Studio 2010–2012 (Ultimate, Premium, or Professional edition).

  • TestComplete 14.

  • TestComplete 14 Visual Studio Integration Package.

    The Integration Package installer is shipped along with TestComplete. You can find it in the TestComplete folder after installing the product:

    <TestComplete 14>\VS Integration\VSIntegration.exe

    Run this installer after installing TestComplete . In the installation wizard, select the Visual Studio Integration feature corresponding to your Visual Studio version.

Computer Where Your Tests Will Run

  • Azure DevOps Server.

    – or –

    Team Foundation Server with the Team Foundation Build Service component.

    – or –

    Team Foundation Build Service.

    – or –

    Build and Release Agent for Windows.

    – or –

    Test Agent for Microsoft Visual Studio.

    The build agent or test agent must be running as an interactive process so that TestComplete can interact with the desktop and with the tested application UI.

    An interactive user session must be opened on the computer where the agent is running. You can open the session manually, or, if you use Team Foundation Server 2018 or later, you can configure the agent to run with the auto-logon enabled so that it opens a user session automatically when it starts.

  • One of the Microsoft Visual Studio versions supported by TestComplete (optional, if you use Team Foundation Build 2010 or 2012).

  • TestComplete 14 (or Test Execute 14).

  • TestComplete 14 Visual Studio Integration Package.

    The Integration Package installer is shipped along with TestComplete. You can find it in the TestComplete folder after installing the product:

    <TestComplete 14>\VS Integration\VSIntegration.exe

    Run this installer after installing TestComplete on the computer, where you will run tests. In the installation wizard, select the Visual Studio Integration feature and Build and Test Integration or Test Agent Integration feature corresponding to the software installed on your build computer or test agent.

  • (Optional) Publish TestComplete Test Results extension.

    If you use Team Foundation Server On-Premises and use Visual Studio Test task version 1 to run TestComplete tests, we recommend that you use the extension to publish test results. The extension will include the TestComplete test results in the test run statistics automatically.

    The extension is available only for Team Foundation Server 2015–2018 On-Premises.

    The extension is shipped with TestComplete. You can find it in the TestComplete folder after installing the product:

    <TestComplete 14>\VS Integration\SmartBear.testcomplete-build-task-14-50.NNN.vsix

    Open the Manage Extensions page of your Team Foundation Server Web Portal and upload the extension, then install it. For detailed instructions on how to do it, see Install Publish TestComplete Test Results Extension (For Team Foundation Server 2015-2019 On-Premises Only).

See Also

Run TestComplete Tests as Part of Build Pipeline (Team Builds)

Highlight search results