Running Tests in Parallel

Applies to TestComplete 15.45, last modified on December 01, 2022

Initially, TestComplete is designed as a functional GUI testing tool: during the test run, it takes over the user session to simulate user actions over GUI elements. Several functional tests cannot share a user session, therefore, only a single test can run on the workstation. However, in TestComplete, you can create tests that will run in test environments controlled by an external framework rather than on your local workstation. These tests are:

These tests can be run in parallel. This would reduce the overall test run time and will help manage testing resources more effectively.

You can orchestrate parallel runs of these tests:

Results of parallel tests are combined into a single test log you can view in TestComplete.

Requirements

  • Your tests must be compatible with the remote environments in which they will run. For instance, you may need to configure your tests or your TestComplete project to access the environments. The way you do it depends on the test type (web or mobile) and the device cloud where the tests will run:

  • Your tests must belong to the same TestComplete project from which you will orchestrate the parallel runs.

  • Your testing environments must be accessible from the computer where TestComplete is installed.

Supported test types

Only the following tests can run in parallel:

  • Cross-platform web tests. These tests can include operations that simulate user actions over web elements and operations that connect to SmartBear Device Cloud or to a target Selenium Grid server and launch web browsers there.

  • Cloud-compatible mobile tests. These tests can include operations that simulate user actions over objects in mobile applications and operations that open a user session on a target mobile device managed by Appium or BitBar.

Supported features

  • Checkpoints.

  • Operations that get test input data from external files (data-driven tests).

  • aqNNN scripting objects, for example aqString or aqConvert.

  • Event handlers.

  • BDD features and scenarios.

  • OCR-based actions and checkpoints. Only in cross-platform web tests.

Unsupported features

  • Nested parallel tests. Parallel tests that control other parallel tests are not supported.

  • Running parallel tests is not supported on 32-bit operating systems.

  • Desktop tests. Tests that simulate user actions over Windows desktop applications cannot run in parallel. For example, the following is not supported:

    • Operations that simulate user actions over desktop applications.

    • Low-level procedures.

  • Non-cross-platform web tests. Tests that simulate user actions over local web browsers accessed via ways other than by using Selenium Grid cannot run in parallel. For example, the following is not supported:

    • Accessing web browsers by using the Sys.Browser object.

    • The Run Browser and Browser Loop operation.

  • Legacy mobile tests. Tests that simulate user actions over physical mobile devices connected to your local workstation cannot run in parallel.

  • Image-based tests.

  • Testing web services.

  • User Forms, Network Suites, Unit tests, manual tests, object-driven tests (ODT), ReadyAPI tests.

  • OCR-based actions and checkpoints are not supported in mobile tests running in device clouds (Appium-based mobile tests).

If you run a test that uses an object or an operation that is not supported by parallel runs, the test engine will report an error. Depending on the way your project handles errors, the test run may be terminated.

Run tests in parallel

Via the Execution Plan editor

You can specify the list of tests that you want to run in parallel and assign a list of remote environments where these tests will run on the Execution Plan of your project:

To run a test in several testing environments in parallel

  1. Create a test you will run in parallel and prepare it for running in remote environments. For detailed instructions, see the appropriate section:

  2. Add the created test to the list of test items of your project’s execution plan.

  3. In the Parallel Device Cloud Environments panel of the plan, select the device cloud provider you will use to run tests.

    For cross-platform web tests, it is CrossBrowserTesting.com or BitBar Web.

    For mobile tests, it can be either BitBar or a private Appium server.

  4. Specify the list of environments where you want to run the test.

When you run the project, TestComplete will connect to the selected device cloud provider, launch the specified environments there, execute the test in them in parallel, and then report the test results.

To run several tests in parallel

  1. Add a parallel group to the list of test items of your project’s execution plan.

  2. Add your tests to the parallel group.

  3. For each added test, in the Parallel Device Cloud Environments panel, specify one or several environments where you want to run the test.

When you run the project, TestComplete will connect to the selected device cloud provider, launch the specified environments, and execute each of the tests in each of the environments in parallel, and then report the test results.

Via the Parallel script object

You can orchestrate parallel runs of cross-platform web tests and cloud-compatible mobile tests from script tests by using methods of the Parallel object:

  • Parallel.RunWebTests(Tests, Environments, TestedURL, ServerURL) - Connects to the device cloud specified by the ServerURL parameter, launches web browsers specified by the Environments parameter, and then runs web tests in each of these browsers in parallel.

  • Parallel.RunMobileTests(Tests, Capabilities, ServerURL) - Connects to the device cloud specified by the ServerURL parameter, opens the testing session described by the Capabilities parameter, and then runs mobile tests in these sessions in parallel.

  • Parallel.RunTests - Runs the specified tests in parallel. This method does not connect to the device cloud and does not specify target environments for the test to run. You have to do it manually in each test the method runs.

Number of parallel tests

  • Your workstation computing capacity: its processor, RAM, and so on. The more parallel tests TestComplete is controlling, the more workstation resources are consumed.

  • You can limit the number of maximum tests running in parallel manually by using the LicenseCount parameter of the Parallel object’s methods or by using the Options > Engine > Parallel > Parallel test count option.

See Also

Running Tests
About Cross-Platform Web Tests
About Mobile Tests

Highlight search results