Automated Testing With TestComplete and Team Build
Author: SmartBear Software
Applies to: TestComplete 9.10-9.30, 10
Microsoft Visual Studio Team Foundation Server includes tools and technologies for all members of software development teams: project architects, developers, managers and, of course, software testers and Quality Assurance personnel. It helps team members communicate throughout the software development process.
Team Foundation Build, which is one of Microsoft Visual Studio tools for managing application lifecycles, enables the development team to create software builds regularly and manage this process. It is important that software testers can run their automated tests as part of team builds, so it is possible to test their applications automatically after the build is over.
TestComplete includes special packages for integration into Microsoft Visual Studio and Microsoft Team Foundation Server. Currently, it supports the following versions of Visual Studio:
Microsoft Visual Studio 2013 (the Ultimate, Premium and Professional editions).
Microsoft Visual Studio 2012 (the Ultimate, Premium and Professional editions).
Microsoft Visual Studio 2010 (the Ultimate, Premium and Professional editions).
Microsoft Visual Studio Team System 2008 Team Suite or Microsoft Visual Studio Team System 2008 Test Edition.
Microsoft Visual Studio 2005 Team Suite or Microsoft Visual Studio 2005 Team Edition for Testers.
The integration packages enable you to extend Visual Studio test projects with TestComplete tests and include them in your application’s lifecycle managed by Visual Studio and Team Foundation tools. You can run TestComplete in Visual Studio, create work items for the tests in Team System’s issue-tracking system or run them as build verification tests.
These packages also enable you to use Visual Studio Team System unit tests as part of your TestComplete projects.
In this article we will provide step-by-step instructions of how to use TestComplete automated tests as part of team builds and consider the following points needed for creating and configuring team builds, integrating TestComplete tests with them and automate the testing of your applications:
In our explanation, we assume that the test project is created and configured on one computer, but is run on another computer - the one that performs the build. Furthermore, we will only provide explanations for Visual Studio 2013. For detailed instructions on configuring and running team builds in earlier versions of Visual Studio, please refer to TestComplete’s help system.
Note for Windows 8, Windows 8.1 and Windows Server 2012 users: Visual Studio integration uses the TestComplete COM interface. Currently, to be able to work via COM on Windows 8, Windows 8.1 and Windows Server 2012, TestComplete requires specific preparations. You may need to change the privileges of TestComplete or of your application in order to run tests. For detailed information on the preparations you need to do, see the Setting Up Manifest section in the Working With TestComplete via COM topic of TestComplete documentation.
Creating and Configuring Automated Test Projects
Once the TestComplete integration package is installed, a new TestComplete 10 Test type becomes available in Visual Studio test projects. The items of this type specify the TestComplete project or project suite to execute and its run settings. Using these items, you can run TestComplete tests in Visual Studio in the same way you run other Visual Studio tests.
Known Limitation: By default, a TestComplete 10 Test item stores the relative path to the appropriate TestComplete project suite. If you are going to execute tests on a remote computer, for example, on a build machine, you need to make sure that the specified relative path to the TestComplete project suite remains valid after the Visual Studio test project’s files are copied to a drop folder. You can achieve this either by placing TestComplete project files in a shared network folder, or by creating custom build steps that will place the project suite files in the appropriate folder to keep the path valid. In this article, we will describe the former approach. For instructions on how to create custom build steps, see the Creating and Configuring Test Project topic in TestComplete documentation.
The procedure of creating test projects and adding TestComplete automated tests to them differs depending on the Visual Studio version you use. Below, we explain how to add tests to Visual Studio 2013 projects. For information on how to create and configure test projects in earlier versions of Visual Studio, see the Creating and Configuring Test Project topic in TestComplete documentation.
To create a new test project in Visual Studio, do the following operations:
So, we have just created a new test project. Visual Studio will open and display its contents in the Solution Explorer.
To add the TestComplete test to your automated test project, do the following:
Right-click the test project in Visual Studio’s Solution Explorer and select Add | New Item from the context menu.
In the ensuing Add New Item dialog, select the TestComplete 10 Test type:
Specify the desired test name and the test project. Then press OK.
After adding the TestComplete automated test to your Visual Studio test project, you can modify its properties using a special test editor in Visual Studio. To view this editor, double-click the desired TestComplete test item in Solution Explorer or select Open from its context menu:
In this editor, you click the ellipsis button and specify the TestComplete project or project suite to be executed during the build testing. The editor will display the tree of test items that are defined in your project or project suite.
If your TestComplete project or project suite is located in a shared network folder, specify the path to it in the UNC (Universal Naming Convention) format, that is, the project path should point to the shared folder and include the client computer name.
Note: If your TestComplete project is located on a local computer, specify a relative path to the project. For information on how to keep this path valid after the test project files are placed in a drop folder on a build server, see the Creating and Configuring Test Project topic in TestComplete documentation.
The editor will display the tree of test items that are defined in your project or project suite. You can enable the Run test items | Selected below option and choose test items that you want to run during the build testing and clear check boxes for items that you do not want to run. Or you can enable the Run test items | Selected in project suite option to ignore the selection of test items below and execute only test items selected in the specified project or project suite.
Also, in the Preferred test runner section you can specify what automation testing tool should be used for executing TestComplete tests – TestComplete or TestExecute.
You have now integrated a TestComplete project with a Visual Studio test project.
Before you create a build and integrate the automated test project with it, it is recommended that you run the project to make sure that it runs properly.
To use your automated test project in team build, you should also implement some more preparatory operations. Since Visual Studio builds operate with files taken from the source control, you need to add the created Visual Studio test project to Team System’s source control.
Note: In this article, we explain how to add a project to the Team Foundation Version control. For information on how to add projects to a Git repository, see TestComplete documentation or the MSDN library.
Visual Studio will check in the solution files to the source control system.
The creation and configuration of the automated test project is now finished. The next step is to create and configure a team build project.
Creating and Configuring Team Build Projects
Now that you have created and configured an automated test project for a team build, this is a good time to create the team build and integrate the automated test project into it. Note that the process of build creation slightly differs from one version of Visual Studio to another.
We will explain how to define a build process that will execute your TestComplete tests after Team Foundation Build finishes building your application in Visual Studio 2013. To learn how to define a build process in earlier versions of Visual Studio, see Defining a Build Process in TestComplete documentation.
To define a build process that executes your TestComplete tests, follow the steps below:
The window tabs that require your input are marked with a warning icon.
Also, configure the build process to copy output files and logs to a shared folder on the server.
Share a folder on the server’s hard drive. This folder will store build results.
In Visual Studio, open the Team Explorer panel and connect to your team project.
In Team Explorer, click Builds, and then click New Build Definition.
The Build Definition window will appear:
In the General tab of the window, specify the build definition name and description.
In the Trigger tab, if necessary, you can specify the event that will cause this team build to be run. By default, team builds are run manually.
In the Source Settings tab, specify the Team Foundation Version Control directories of the team project for which you are creating the new build definition and map these source control directories to the appropriate folders on the build computer.
Note: For information on how to choose a build source when using the Team Foundation Git repository, see TestComplete documentation or the MSDN Library.
In the Build Defaults tab, specify the build controller that will be used to process the team build.
On the Process tab of the New Build Definition window, you must specify details of the build process (what functions the team build performs and how it performs them).
In the Build process parameters table, specify at least one solution or project to build:
In order for your team build to run automated TestComplete tests included in your test project, follow the steps below:
Expand the Automated tests node under the Test node in the Build process parameters table.
Click the ellipsis button in the Test source box. The Add/Edit Test dialog will appear.
In the Name box of the Add/Edit Test dialog, specify the name of the test run and in the Test Runner box of the dialog, select Visual Studio Test Runner.
In the Test assembly file specification box of the dialog, type **\*.tc10test to specify TestComplete tests.
Click OK to confirm the changes.
After you perform the above-mentioned actions, the Test source node will specify the TestComplete tests to be run and the test run details. You can modify these parameters if needed.
Running the Build and Analyzing Test Results
Now that you have created a new build definition, you can start the team build. The test project will be executed at the end of the build run. Visual Studio will publish the test results automatically after the automated test has finished.
To start the build, do the following:
After the build is over, Team Foundation Build automatically runs tests.
To view test results, do the following:
You have just learned how to add TestComplete tests to Visual Studio test projects, how to define a build process that executes these tests after Team Foundation Build finishes building your application and how to view the test results. We hope this information will help you easily create and execute your team build projects with TestComplete.
If you are new to TestComplete, download now and try TestComplete for free.
Switch to the Team Explorer panel.
Right-click the desired build definition in the panel and select Queue New Build from the context menu. This will invoke the Queue Build dialog:
In the dialog, specify the build definition, the build controller, the priority of the build in the build queue and the output folder where the files and modules will reside after the build is over. The values of these parameters were specified previously when creating the build definition, but you can change them now if you need.
Note: If TestComplete tests which are run from the team build interact with the desktop and perform UI testing, the Build Service on the build machine controlled by the specified build controller must be running as an interactive process. If the Build Service is running as a Windows service, TestComplete tests will not be able to interact with the desktop and the tested application’s UI and will fail during the build process.
Press Queue. Visual Studio will add a build request to the build queue. You can track the build progress on the Builds tab of the Team Explorer or in the Build Explorer panel.
On the Builds tab of the Team Explorer panel, double-click the build under the My Builds section.
Visual Studio will open the build report in the Build Results window:
In the Summary section of the build report, expand the N test run(s) completed node and click a View Test Results link corresponding to the needed test run. Visual Studio will add a row to the Test Results panel:
Double-click the added row in the Test Results panel. Visual Studio will open detailed automated test results with the attached TestComplete log file.
Click the link to the log in the Collected Files section to open the attached log: