Continuous integration (CI) is a software development approach that implies merging changes made by developers several times a day and making frequent automated builds and tests. If your company uses continuous integration, then, most likely, it uses a special continuous integration system like Jenkins, Bamboo, Team Foundation Server, or other. That system builds your software regularly and runs automated tests after each build on one or more test computers. This topic explains how you can include TestComplete in your CI cycles.
Supported Continuous Integration Systems
By running TestComplete tests using the TestComplete command line, the SessionCreator utility, or the COM interfaces, you can integrate your tests with almost any continuous integration system. For example:
Azure DevOps and Team Foundation Server,
and many others.
Read about the general approach below.
In addition, for several CI systems, TestComplete offers extensions that simplify the test run:
It is important that you open an interactive user session on the computers where TestComplete will run. See below.
You can run TestComplete tests with TestComplete or TestExecute – a lightweight resource-friendly test runner.
The computers, where your CI system will run TestComplete tests, are often called agent machines. They must meet the following requirements:
They must have TestComplete or TestExecute installed.
Important: The agent machines must have an interactive user session started. These sessions imply that a user works with the operating system and applications through the GUI (windows, menus, controls and so on). Quite often, CI systems use GUI-less sessions on agent machines. TestComplete tests cannot work in these sessions, because the GUI does not exist, and TestComplete cannot simulate user actions on windows and controls.
If you have TestComplete (or TestExecute) version 14.10, you can use the SessionCreator utility to open user sessions on your agent machines automatically. Otherwise, you will have to open user sessions manually or by using third-party tools.
Your project or project suite files must be available on these computers.
You can store your project files in a source control repository and configure your CI system to take them from the repository to the computer where your tests will run.
– or –
You can store your project files in a shared network folder to which the agent machines have access.
TestComplete or TestExecute must have access to all the resources, applications and files required for testing. Also, TestComplete (or TestExecute) must have appropriate access permissions for these files and resources.
Note: In addition, if you use Zephyr Squad to plan and monitor your testing efforts, you can publish your TestComplete test results to your Jira projects. To do this, bind your TestComplete project to your Jira project and assign Zephyr tests to your TestComplete tests. To learn how to do this, see Integration With Zephyr Squad.
Install TestComplete or TestExecute on the computer where your automated tests will run.
Configure your CI system to prepare your testing environment. For instance, configure it to place your test project in a folder to which the agent machines have access.
Important: Open an interactive user session on the computer where your tests will run.
(For TestComplete 14.10) Use the SessionCreator utility that is shipped with TestComplete and TestExecute to open an interactive user session on your test agents and to run your tests. See the utility description.
As an alternative, you can open user sessions on remote machines manually or by using various third-party utilities.
Configure your CI system to start TestComplete or TestExecute on the needed computer and to run tests.
(For TestComplete 14.10) Use the SessionCreator utility that is shipped with TestComplete and TestExecute to run tests. See the utility description.
(For TestComplete versions prior to version 14.10) Use the TestComplete command line to run tests. See TestComplete Command Line.
If needed, configure your CI system to get TestComplete test logs.
Note: By default, TestComplete stores test results in the XML format. You can configure TestComplete to export results in the HTML and MHT formats or to pack them into an archive. To learn more, see Working With Test Results.
If needed, configure your CI system to clean your test environment after the test runs are over.
TestComplete Command Line
Integration With Jenkins
Running TestComplete Tests From MSBuild Projects
Integration With Azure DevOps and Team Foundation Server