Create a build definition and configure it to run TestComplete tests. You can use the Visual Studio Test task to run TestComplete tests on your build computer or the Run Functional Tests tasks to run TestComplete tests on remote test agents:
Requirements
The computer, where you create a build definition, must have access to your team project:
-
You can access the project from Visual Studio Team Explorer. To do this, you must have Microsoft Visual Studio 2015 - 2017 (Community, Enterprise or Professional) with the Team Explorer component.
– or –
-
On the computer, where Visual Studio is not installed, you can access the project directly from your web browser by using Team Web Access:
For On-Premises Team Foundation Server
http://<Your_Server_Name>:8080/tfs/<Collection_Name>/<Project_Name>
For Team Foundation Services
https://<Your_VSO_Account_Name>.visualstudio.com/<Project_Name>
You can find information on Web Access in the Work in the TFS Web Portal article of the MSDN Library.
The computer, where your TestComplete tests will run, must have the following software installed:
-
If you use the Visual Studio Test task to run tests, Visual Studio Team Services Build and Release Agent for Windows.
– or –
If you use the Run Functional Tests task to run tests, one of the following versions of Test Agent for Microsoft Visual Studio:
-
Test Agent for Microsoft Visual Studio 2015 RTM and Update 1
-
Test Agent for Microsoft Visual Studio 2017 and Visual Studio 2015 Update 2
You use the Visual Studio Test Agent Deployment task to install the test agent (see below). To learn about requirements the remote computers must meet so that you could install the test agent on them, see the step description in the MSDN Library.
-
-
TestComplete 12 (or TestExecute 12).
-
TestComplete 12 Visual Studio Integration Package.
The Integration Package installer is shipped along with TestComplete. You can find it in the TestComplete folder after installing the product:
<TestComplete 12>\VS Integration\VSIntegration.exe
Run this installer after installing TestComplete on computers where tests will run. In the installation wizard, select a Test Agent Integration feature appropriate to your agent version.
Configuring a Build Definition
-
Open the web portal of your team project. You can navigate to it directly in your web browser, or you can open it from Team Explorer in Visual Studio.
-
Navigate to the Build hub (if you use Team Foundation Server 2015 Update 1) or to the Builds page of the Build & Release hub (if you use Team Foundation Server 2015 Update 2 or later):
-
Open an existing build definition or click
New to create a new build definition:
In the Create new build definition wizard, configure the definition to be created:
-
Select the needed definition type.
-
On the Settings page of the wizard, specify the build source.
-
Select the queue for the build and click Create.
-
-
On the Repository page of the definition, configure your build definition to get the needed files from your source control:
If You Use Team Foundation Version Control
Map the source control directories that store your Visual Studio test project and TestComplete projects to the appropriate local directories on the build computer:
If You Use Git or Subversion
Specify the repository and the branches that store your team project files:
Keep in mind that the test project stores the relative path to TestComplete tests. When mapping the TestComplete source control path, make sure to select a local folder to keep the relative path specified in the test project valid. -
On the Build page of the definition, specify the details of the build process (what functions the team build will perform):
Add a task that will build your Visual Studio test project:
In the Solution box, click the ellipsis button and browse for the Visual Studio test project that contains your TestComplete tests:
Add tasks that will run your TestComplete tests:
Running TestComplete Tests by Using Run Functional Tests Task
Share a folder on remote computers where tests will run. To that folder, the test files will be placed.
Configure your definition to copy your TestComplete tests from build output folder to computers where tests will run. For example, you can use the Windows Machine File Copy task to do this:
Note: Make sure that the path to TestComplete projects or project suites that the TestComplete test items store remains valid. Configure your definition to deploy Test Agent for Visual Studio on computers where tests will run:
Specify the computers, to which the agent will be deployed. Specify the login and password of the user account that will be used to deploy agents.
Important: Configure the agent to run UI tests. Otherwise, the agent will launch TestComplete (or TestExecute) in non-interactive mode, and it will not be able to access UI elements to simulate user actions.
Make sure that TestComplete (or TestExecute) and the TestComplete Integration package are installed on remote test agents before you run tests.
You can configure your definition to install them by using the Run PowerShell Script on Target Machines task. Configure the task to run a script that will install TestComplete (or TestExecute) and the Integration package via command line in silent mode.
To learn more about installing TestComplete from the command line, see Installing TestComplete in Silent Mode. To learn about installing TestExecute in silent mode, see the appropriate topic in the TestExecute documentation.
To learn more about installing the Integration package via command line, see Installing Integration Package.
Add a Run Functional Tests task to the definition:
Configure the added task to run your TestComplete tests:
In the Machines text box, enter the network names or IP addresses of test agents where tests will run.
In the Test Drop Location text box, enter the path where your tests reside.
In the Test Selection drop-down list box, select Test Assembly.
To include TestComplete test results into a build report and assign them to a test run, in the Test Run Title text box, enter the needed test run name.
Running TestComplete Tests by Using Visual Studio Test Task
Add a Visual Studio Test task to your definition:
Configure the added task to run your TestComplete tests:
In the Test Assembly text box, specify TestComplete test items to be run:
If you use task version 1, TestComplete test results will not be published to the server. To learn how to include TestComplete test results into build reports, see Publish TestComplete Test Results.
If you use task version 2, to include TestComplete test results into a build report and assign them to a test run, in the Test Run Title text box, enter the needed test run name.
You can find information about tasks in the Build and release tasks article in the MSDN Library.
-
If needed, specify other build properties. To learn more, see the Continuous integration and deployment section in the MSDN Library.
-
Save the changes.
See Also
Defining a Build Process
Running TestComplete Tests as Part of a Build and Viewing Test Results