Automated Tests and Automated Builds

Author: SmartBear Software
Applied to: TestComplete 8 - 10 and Automated Build Studio 6.0 - 6.60


Application building is an inherent part of a software development process. Typically, it includes a number of steps, like getting the latest version of source files from a version control system, compiling libraries and packages, linking, making the installation program, software testing and others. Building an application manually quite often becomes a very tedious, is prone to errors and is an inefficient process. It often takes a lot of time, especially if there are large, complex projects. To save time and money and to reduce human mistakes that are possible during manual building, software developers usually try to automate this process.

Automated builds are very important during software development and the use of automated tests during the build process is also important. To achieve maximum efficiency, automated tests are included in automated builds and ran during the build process. This allows you to detect failures in a new build, catch bugs in newly integrated code and report them to developers.

Running automated tests in your automated builds guarantees that every build is tested for bugs. If any bug in the code arises, it is best to detect and fix it as soon as it appears. Using automated tests in automated builds makes it possible to discover bugs in the shortest period, when it is easier to find the reason for the bug and fix it. This allows software developers to find and correct more errors before the application is released.

SmartBear’s TestComplete is a powerful automated testing tool and it is well suited for efficient testing of applications during the automated build process.

Automated Tests in Automated Build Studio

Many development tools use TestComplete via COM to automate software testing. For example, SmartBear's Automated Build Studio uses COM to integrate TestComplete’s automated tests and test results with its macros.

Automated Build Studio is a powerful build and release management system that provides features for automated software build, test and release processes and lets you optimize day-to-day building tasks and perform them automatically.

Automated Build Studio can be tightly integrated with TestComplete. With both of these tools, you get a powerful and flexible system for automating builds and software tests. You can perform automated builds and run TestComplete’s automated tests from Automated Build Studio. To execute TestComplete’s automated tests, Automated Build Studio provides special operations. These operations use COM technologies to communicate with TestComplete. Automated Build Studio includes appropriate versions of these operations for each version of TestComplete.

To test your application properly, you should add the appropriate TestComplete operation at the end of your build macro, after the main building operations. So, Automated Build Studio will apply TestComplete’s automated tests to the finished build to detect bugs in it.

TestComplete Operation Within Automated Build Studio Macro
Figure 1 - The TestComplete operation within the Automated Build Studio macro.

After you add the operation to your macro, you should specify its properties. Let’s consider, for example, how you can use the operation that corresponds to TestComplete 10 and specify its properties. Just double-click the operation in the macro or select Properties from its context menu to display the "TestComplete 10" Operation Properties dialog. In this dialog, the operation properties are organized into the following four pages:

  • Properties - contains properties common for all Automated Build Studio operations. On this page, you can specify the operation’s description, enable or disable the operation, specify actions that you want to perform if the operation fails (stop execution, run a submacro or no actions) and some others.

    Properties Page in TestComplete Operation Properties Dialog
    Figure 2 - The Properties page in the TestComplete Operation Properties dialog.

  • Project - contains the main properties of the operation. On this page, you can specify TestComplete’s project suite, projects, project items, test items, script units and routines that you want to run for automated testing during the automated build process. You can also specify whether TestComplete will be hidden during the test execution.

    Project Page in TestComplete Operation Properties Dialog
    Figure 3 - The Project page in the TestComplete Operation Properties dialog.

  • Result - on this page, you can specify variables that will receive the total number of error and warning messages in the test log. You can also specify whether the test results will be exported to a .mht file.

    Result Page in TestComplete Operation Properties Dialog
    Figure 4 - The Result page in the TestComplete Operation Properties dialog.

  • Messages - on this page, you can specify how Automated Build Studio will store the log generated by TestComplete. You can specify which messages will be stored and select where they will be exported to: the Log panel, a macro variable or a specified file.

    Messages Page in TestComplete Operation Properties Dialog
    Figure 5 - The Messages page in the TestComplete Operation Properties dialog.

The operation can also have child operations. They are run by Automated Build Studio after the specified TestComplete automated tests have been loaded. These child operations can change some settings of TestComplete or modify the automated test project’s settings. Once all of the child operations are executed, the operation starts running the automated test.

By default, Automated Build Studio does not import TestComplete’s test log. If you select the Output TestComplete messages check box, the operation will import only warning and error messages. To import messages of other types, select the corresponding check boxes. In the Output messages to section, you can specify whether the imported messages will be stored in the macro execution log, or in a variable or file. Note that Automated Build Studio imports only the message text. To see the entire test log, you need to open your test project in TestComplete.

The test log with TestComplete messages.
Figure 6 - The test log with TestComplete messages.

TestComplete’s automated tests can also be executed with another SmartBear’s tool - TestExecute. This tool allows you to execute TestComplete’s automated tests on machines without TestComplete installed. Automated Build Studio also includes the appropriate operation for launching automated tests from your build macros using TestExecute. This operation has a similar set of properties as the TestComplete operation has and it is used in the same way.

TestComplete’s Integration With Other Build Tools

TestComplete can also be integrated with two other automated build tools: MSBuild and Visual Studio Team System. You can use TestComplete automated tests in your MSBuild projects or run automated tests as part of your team builds in Visual Studio Team System. This makes your automated builds more powerful. For more information, see the Running TestComplete Automated Tests from MSBuild Projects and Automated Testing with TestComplete and Team Build technical papers.


In this paper we have just given an overview of how you can use TestComplete automated tests in your automated builds. The paper explains how to run TestComplete tests from Automated Build Studio macros and explores the approaches to using TestComplete when making automated builds. We hope this information will be useful and will help you improve your automated build projects and make them more powerful, reliable and efficient. If you haven’t tried either of these products, download TestComplete and Automated Build Studio today.