Starting from version 14.80, the standalone TestExecute Lite will not be used to run tests in parallel. Orchestrate your parallel runs directly in TestComplete projects by using the Parallel object or the execution plan. See below.
Tests that support parallel runs
-
Cross-platform web tests
-
Cloud-compatible mobile tests
You can find the detailed information on these tests in the TestComplete documentation. The online version is available on our website:
Requirements
To run tests in parallel, you must have:
-
Access to a device cloud that manages the testing environments.
-
Your tests must support parallel runs.
For detailed information on requirements that must be met, please see the appropriate section of the TestComplete documentation.
Running tests in parallel
-
In TestComplete, create tests that will run in parallel. The way you do it depends on the test type: web or mobile.
See the TestComplete documentation for details:
-
In TestComplete, define an execution plan that will manage parallel runs, or create a test that will manage parallel runs by using the
Parallelobject. -
Run the project with TestExecute the same way you run any other project:
-
Manually, from the TestExecute context menu.
-
Automatically, by using the TestExecute command line or by using any suitable third-party task management or scheduling tool.
-
It will connect to the testing environments and run tests there in parallel. After the test run is over, the test results will be collected on the TestExecute workstation.
More on parallel web tests
Number of parallel tests
A single TestComplete project can control any number of parallel runs. It is only limited by:
-
Your test environment provider.
-
Your workstation computing capacity: its processor, RAM, and so on. The more parallel tests the TestComplete project is controlling, the more workstation resources are consumed.
-
The Parallel test count option of TestExecute.
Supported features
Parallel tests can include the following:
-
Operations that simulate user actions over tested applications.
-
Operations that connect to target testing environments.
-
Checkpoints.
-
Operations that get test input data from external files (data-driven tests).
-
aqNNNscripting objects, for exampleaqStringoraqConvert. -
Event handlers.
-
BDD features and scenarios.
-
OCR-based actions and checkpoints.
Only in cross-platform web tests.
Integration with CucumberStudio (formerly known as HipTest) is also supported.
Unsupported features
-
Nested parallel tests. Parallel tests that control other parallel tests are not supported.
-
Desktop tests. Tests that simulate user actions over Windows desktop applications cannot run in parallel. For example, the following is not supported:
-
Operations that simulate user actions over desktop applications.
-
Low-level procedures.
-
-
Non-cross-platform web tests. Tests that simulate user actions over local web browsers accessed via ways other than by using Selenium Grid cannot run in parallel. For example, the following is not supported:
-
Accessing web browsers by using the
Sys.Browserobject. -
The Run Browser and Browser Loop operation.
-
-
Legacy mobile tests. Tests that simulate user actions over mobile devices connected to your local workstation cannot run in parallel.
-
Image-based tests.
-
OCR-based mobile tests.
Note: OCR-based actions are supported in web tests only.
-
Testing web services.
-
User Forms, Network Suites, Unit tests, manual tests, object-driven tests (ODT), ReadyAPI tests.
If your TestComplete project includes a project item that parallel tests do not support, TestExecute will be able to open such a project, but it will not be able to run unsupported items in parallel.
If you run a test that uses an object or an operation that cannot be used in parallel tests, the test engine will post an error to the test log. Depending on the way your project handles errors, the test run may be terminated.
Stopping tests on errors and exceptions
If an error occurs during the test run (for example, if a tested object is not found), the test will post a message about it to the test log. Its further behavior depends on the way the test was run:
-
If the test was run as a project’s test item, the further behavior will be specified by the test item’s On error property (you specify it on the Test Items page of your project in TestComplete).
-
If the test was run by its name or by tag, its behavior will be specified by the project’s Playback > On error property.
Depending on the appropriate property value, the test will do one of the following:
-
Ignore the error and continue the run.
-
Stop the current item and proceed with the next item (applicable, for example, when running a project or tests by a tag or tag expression).
-
Stop the entire test run.