Running Cross-Platform Web Tests in Parallel

Applies to TestExecute 14.72, last modified on January 12, 2021

About cross-platform web tests

Starting from version 14.4, you can use TestComplete to create cross-platform web tests. These tests can validate web pages and web applications running in browsers and on platforms that TestComplete does not support directly. For example, such web browsers as Safari or Chrome for Android, or such platforms as macOS, or Unix, or mobile iOS.

To control those web browsers and platforms, Selenium Grid is used. This can be:

  • SmartBear device cloud solution — CrossBrowserTesting.com.

  • A private hub with Selenium Grid configured on your local computer or on a remote computer in your network. To learn more about installing and configuring Selenium Grid, see selenium.dev.

When you run a test, the test connects to your Selenium Grid, commands it to launch the needed web browser, sends test commands to it, and gets the results.

Why run tests in parallel?

Selenium Grid allows controlling multiple nodes and distributing tests among them. To manage Selenium Grid resources more effectively, it would make much more sense to run several TestComplete web tests at the same time. This would reduce the overall test run time and will help you decrease the cost of your testing efforts.

About parallel test runs

By default, TestExecute (and TestComplete) does not allow parallel test runs. The tool is designed to run functional GUI tests: during the test run, it takes over the user session to be able to simulate user actions over GUI elements. It can only run tests in a sequence. In addition, several instances of the tool cannot be running on the same workstation simultaneously.

To run cross-platform web tests in parallel, you can use a special test running tool called TestExecute Lite. A single instance of the tool can run a single test at a time. However, an unlimited number of instances can be running on a computer at the same time. By using instances running in parallel, you can execute several cross-platform web tests simultaneously.

Running cross-platform web tests in parallel

Click the image to enlarge it.

Each test will connect to your device cloud, launch a web browser there, send test commands to the browser, and get the test results.

How to run tests

Running tests in parallel from the GUI is not currently supported. You can manage test runs in the following ways:

Export test results

Unlike sequential test runs that store the test results in the project’s Log folder by default, for your parallel test runs you must specify how you want to keep the results explicitly:

  • In an external file (several formats are supported).

  • In the TestComplete project to which the test belongs.

For detailed information on how to configure your tests to store test results, see Exporting Test Results.

If you use the TestComplete Support plugin for Jenkins to control your test runs, the test results will be added to your Jenkins build results automatically. To learn more, see Viewing Cross-Platform Web Test Results in Jenkins.

More on parallel web tests

User sessions

Interactive user sessions are not required to run cross-platform web tests in parallel. You can run your tests in GUI-less and headless sessions.

Number of parallel tests

You can run any number of tests on the same workstation in parallel. The number is only limited by:

  • Your Device Cloud Parallel license. The number of concurrent instances cannot exceed the overall number of instances that your license allows.

  • Your workstation computing capacity: its processor, RAM, and so on. The more tests are running in parallel, the more workstation resources they are consuming.

Supported features

Only cross-platform web tests created with TestComplete can be run in parallel. It includes the following:

  • All the cross-platform web test features that TestComplete supports, including:

    • Simulating user actions over web elements and controls in web browsers.

    • Special operations for connecting to the device cloud provided by CrossBrowserTesting and launching web browsers there.

  • Checkpoints.

  • Getting test input data from external files (data-driven tests).

  • aqNNN scripting objects, for example, aqString, aqConvert, and so on.

  • BDD tests.

  • Integration with CucumberStudio (formerly known as HipTest). To learn how to run TestComplete tests bound to a CucumberStudio project, see Running BDD Tests in Device Cloud.

  • Event handlers.

Unsupported features
  • Desktop tests. Tests that simulate user actions over Windows desktop applications cannot run in parallel.

  • Non-cross-platform web tests. Tests that simulate user actions over local web browsers accessed via ways other than by using Selenium Grid cannot run in parallel. For example, if your web test launches a web browser by using the Run Browser operation or accesses a web browser by using the Sys.Browser object, the test engine will not run it and report an error.

  • Mobile tests. Tests that simulate user actions over mobile devices connected to your local workstation via the Mobile object cannot run in parallel.

Stopping tests on errors and exceptions

If an error occurs during the test run (for example, if a tested object is not found), the test will post a message about it to the test log. Its further behavior depends on the way the test was run:

  • If the test was run as a project’s test item, the further behavior will be specified by the test item’s On error property (you specify it on the Test Items page of your project in TestComplete).

  • If the test was run by its name or by tag, its behavior will be specified by the project’s Playback > On error property.

Depending on the appropriate property value, the test will do one of the following:

  • Ignore the error and continue the run.

  • Stop the current item and proceed with the next item (applicable, for example, when running a project or tests by a tag or tag expression).

  • Stop the entire test run.

Video tutorial

 

See Also

Running Cross-Platform Web Tests in Parallel

Highlight search results