Running Cross-Platform Web Tests

Applies to TestExecute 14.80, last modified on April 13, 2021

Note: Starting from version 14.80, the standalone TestExecute Lite will not be used to run tests in parallel. Orchestrate your parallel runs directly in TestComplete projects by using the Parallel object or the execution plan. See below.

You can use TestComplete to create cross-platform web tests. These tests are created on a local computer with one of the browsers that TestComplete supports directly (Internet Explorer, Edge, Chrome, Firefox) and then run in a wide range of web browsers and platforms (like Opera and Safari, Linux, Unix, Mac OS, mobile Android and iOS) managed by Selenium Grid.

Requirements

  • To run tests in various browsers and on various devices, you must have Selenium Grid that manages them. It can be:

    • A private hub with Selenium Grid configured. It can be Selenium Grid running on your local computer or on a remote computer in your network. To learn more about installing and configuring Selenium Grid, see selenium.dev.

    • Selenium Grid provided as a service by CrossBrowserTesting.

  • Your Selenium Grid must be accessible from the computer where TestExecute is installed.

Creating cross-platform web tests

You create cross-platform web tests on a local computer where TestComplete is installed. For detailed information, please see the TestComplete documentation.

Running cross-platform web tests in series

You run the tests the same way you run any other test created with TestComplete:

  • Manually, from the TestExecute context menu.

  • Automatically, by using the TestExecute command line or by using any suitable third-party task management or scheduling tool.

TestExecute will connect to your device cloud, command it to launch the needed web browsers, execute test commands there, and then collect test results.

Running cross-platform web tests in parallel

  1. In TestComplete, define an execution plan that will manage parallel runs, or create a test that will manage parallel runs by using the Parallel object.

  2. Run the project with TestExecute the same way you run any other project:

    • Manually, from the TestExecute context menu.

    • Automatically, by using the TestExecute command line or by using any suitable third-party task management or scheduling tool.

It will connect to the device cloud and run cross-platform web tests there. After the test run is over, the test results will be collected on the TestExecute workstation.

More on parallel web tests

Number of parallel tests

You can run any number of tests on the same workstation in parallel. The number is only limited by:

  • Your test environment provider.

  • Your workstation computing capacity: its processor, RAM, and so on. The more tests are running in parallel, the more workstation resources they are consuming.

Supported features

Only cross-platform web tests can run in parallel. These tests can include the following:

  • Operations that simulate user actions over web elements.

  • Operations that connect to the SmartBear device cloud and launch web browsers there.

  • Checkpoints.

  • Operations that get test input data from external files (data-driven tests).

  • aqNNN scripting objects, for example aqString or aqConvert.

  • Event handlers.

  • BDD features and scenarios.

Integration with CucumberStudio (formerly known as HipTest) is also supported.

Unsupported features
  • Nested parallel tests. Parallel tests that control other parallel tests are not supported.

  • Desktop tests. Tests that simulate user actions over Windows desktop applications cannot run in parallel. For example, the following is not supported:

    • Operations that simulate user actions over desktop applications.

    • Low-level procedures.

    • Operations that manage the Tested Applications collection.

  • Non-cross-platform web tests. Tests that simulate user actions over local web browsers accessed via ways other than by using Selenium Grid cannot run in parallel. For example, the following is not supported:

    • Accessing web browsers by using the Sys.Browser object.

    • The Run Browser and Browser Loop operation.

  • Mobile tests. Tests that simulate user actions over mobile devices connected to your local workstation cannot run in parallel. For example, the following is not supported:

    • Android Gesture collection.

    • Operations that access mobile devices connected to your workstation via the Mobile object.

  • Image-based and OCR-based tests.

  • Testing web services.

  • User-forms, Network Suites, Unit tests, manual tests, object-driven tests (ODT), ReadyAPI tests.

If your TestComplete project includes a project item that parallel tests do not support, an error will occur, and the test run will not start.

If your test uses an object or an operation that cannot be used in parallel tests, an error will occur. Depending on the way your project handles errors, the test run may be terminated.

Stopping tests on errors and exceptions

If an error occurs during the test run (for example, if a tested object is not found), the test will post a message about it to the test log. Its further behavior depends on the way the test was run:

  • If the test was run as a project’s test item, the further behavior will be specified by the test item’s On error property (you specify it on the Test Items page of your project in TestComplete).

  • If the test was run by its name or by tag, its behavior will be specified by the project’s Playback > On error property.

Depending on the appropriate property value, the test will do one of the following:

  • Ignore the error and continue the run.

  • Stop the current item and proceed with the next item (applicable, for example, when running a project or tests by a tag or tag expression).

  • Stop the entire test run.

See Also

Running Tests With TestExecute

Highlight search results