Continuous integration (CI) is a software development approach that implies merging changes made by developers several times a day and making frequent automated builds and tests. If your company uses continuous integration, then, most likely, it uses a special continuous integration system like Jenkins, Bamboo, Team Foundation Server, or other. That system builds your software regularly and runs automated tests after each build on one or more test computers. This topic explains how you can include TestComplete in your CI cycles.
Supported Continuous Integration Systems
You can run TestComplete tests using the TestComplete command line or the SessionCreator utility, or you can run TestComplete tests via COM interfaces. This allows you to integrate your tests with almost any continuous integration system: Jenkins, Team Foundation System, Bamboo, TeamCity, Zephyr and many others. For some CI systems, TestComplete offers special extensions that simplify the test run. For others, you can try using the SessionCreator utility or the TestComplete command line.
At the moment, TestComplete documentation includes instructions for integrating with the following CI systems:
If you use another CI system, read about the general approach below and see your CI system documentation for information on starting programs from the command line.
It is important that you open an interactive user session on the computers where TestComplete will run. See below.
You can run TestComplete tests with TestComplete or TestExecute – a lightweight resource-friendly test runner.
General Approach
Requirements
The computers, where your CI system will run TestComplete tests, are often called agent machines. They must meet the following requirements:
-
They must have TestComplete or TestExecute installed.
-
Important: The agent machines must have an interactive user session started. These sessions imply that a user works with the operating system and applications through the GUI (windows, menus, controls and so on). Quite often, CI systems use GUI-less sessions on agent machines. TestComplete tests cannot work in these sessions, because the GUI does not exist, and TestComplete cannot simulate user actions on windows and controls.
If you have TestComplete (or TestExecute) version 14.10, you can use the SessionCreator utility to open user sessions on your agent machines automatically. Otherwise, you will have to open user sessions manually or by using third-party tools.
-
Your project or project suite files must be available on these computers.
You can store your project files in a source control repository and configure your CI system to take them from the repository to the computer where your tests will run.
– or –
You can store your project files in a shared network folder to which the agent machines have access.
-
TestComplete or TestExecute must have access to all the resources, applications and files required for testing. Also, TestComplete (or TestExecute) must have appropriate access permissions for these files and resources.
-
If you are planning to run TestComplete tests in Windows-based testing environments provided by CrossBrowserTesting.com, you must have an active CrossBrowserTesting account with TestComplete support. You must also provide valid account information so that TestComplete can connect to the CrossBrowserTesting cloud from your computers.
This approach to running tests in CrossBrowserTesting.com environments is obsolete starting with TestComplete 14.40. We recommend using the Device Cloud add-on instead. See About Cross-Platform Web Tests for details.
Note: In addition, if you use Zephyr for Jira to plan and monitor your testing efforts, you can publish your TestComplete test results to your Jira projects. To do this, bind your TestComplete project to your Jira project and assign Zephyr tests to your TestComplete tests. To learn how to do this, see Integration With Zephyr for Jira.
Integration Steps
-
Install TestComplete or TestExecute on the computer where your automated tests will run.
-
Configure your CI system to prepare your testing environment. For instance, configure it to place your test project in a folder to which the agent machines have access.
-
Important: Open an interactive user session on the computer where your tests will run.
(For TestComplete 14.10) Use the SessionCreator utility that is shipped with TestComplete and TestExecute to open an interactive user session on your test agents and to run your tests. See the utility description.
As an alternative, you can open user sessions on remote machines manually or by using various third-party utilities.
-
Configure your CI system to start TestComplete or TestExecute on the needed computer and to run tests.
(For TestComplete 14.10) Use the SessionCreator utility that is shipped with TestComplete and TestExecute to run tests. See the utility description.
(For TestComplete versions prior to version 14.10) Use the TestComplete command line to run tests. See TestComplete Command Line.
-
If needed, configure your CI system to get TestComplete test logs.
Note: By default, TestComplete stores test results in the XML format. You can configure TestComplete to export results in the HTML and MHT formats or to pack them into an archive. To learn more, see Working With Test Results. -
If needed, configure your CI system to clean your test environment after the test runs are over.
See Also
TestComplete Command Line
Integration With Jenkins
Running TestComplete Tests From MSBuild Projects
Integration With Azure DevOps and Team Foundation Server