Distributed Testing - Basic Concepts

Applies to TestComplete 15.45, last modified on December 01, 2022
The Network Suite functionality is deprecated. We don’t recommend using it for distributed testing. Consider using a CI/CD system for managing distributed tests. See Migrating Distributed Tests to CI/CD Systems for details. In case you need to run web tests on multiple environments in parallel, you can also try using your project’s Execution Plan.

TestComplete allows you to perform distributed testing, that is, run tests that consist of several parts executed on different workstations and interacting with each other.

Distributed and Non-Distributed Testing

Distributed and non-distributed testing represents a different approach of how to classify testing methods. Non-distributed tests are run on a single computer and normally do not interact with other computers.

On the contrary, distributed testing means that a test consists of several parts and each of them is performed on a separate computer.

However, the term “distributed” means not only a simultaneous run of tests on a number of machines, which is specific for parallel testing. Distributed testing also suggests that test parts interact with each other during a test run. This makes them coordinated and synchronized in the appropriate manner. Actually, synchronization is one of the most crucial points in distributed testing.

Another key point is administration and configuration of the test. For better analysis of results, statistics on running of all test parts should be accumulated in a single common result, while you should still keep the ability to analyze individual results of the desired test part.

In addition, you should be able to repeat the distributed tests consistently, in the way it is required for regression testing.

Distributed Applications

Distributed testing is normally needed for web or intranet sites, client-server or other multi-tier applications, where a server application interacts with more than one client. We can call applications of this type distributed applications.

Testing of a distributed application involves testing of its client and server parts. You can test them separately. Distributed and simultaneous testing gives you a way to test them all together.

For instance, a test makes a client instance send requests to the server, at the same time due to several clients working together you can assess how the server performs. Note that you can extend the test if while the server is processing client requests, you will profile the server or a client instance (or both) using a debugging/profiling tool, for example, AQTime.

TestComplete Network Suite

In order to support distributed testing, TestComplete provides the NetworkSuite project item. TestComplete network suites are intended to manage the runs of several test projects or project suites on different computers. Test projects can run in a series or simultaneously. You can handle the network suite via the TestComplete user interface or from your keyword tests or scripts, which makes the testing more flexible.

Notes:

  • To run test projects on computers where TestComplete is not installed, you can also use TestExecute - a resource-friendly utility, which was specially created to execute TestComplete projects.

  • It does not matter if master and slave computers have different bitness of TestComplete/TestExecute. If the 64-bit version is available on a slave machine, it will be used to run tests. Otherwise, the 32-bit one will be used instead.

The network suite functionality is implemented by a group of Network Suite plugins:

  • Network Suite Core
  • Network Suite Host List Editor
  • Network Suite Job List Editor
  • Network Suite Run State
  • Network Suite Task List Editor

These plugins are installed and enabled automatically during the TestComplete installation. If you experience issues with the network suite functionality, check whether the plugins are enabled in the File > Install Extensions dialog.

The Network Suite Core plugin provides TestComplete with the NetworkSuite project item, which can be added to a project. Only projects that include the NetworkSuite project item can run other projects or project suites or can be run by other projects. That is, the network suite gives a project the capability to control the run of several projects on different network computers.

The NetworkSuite project item lets you configure the following items:

  • Computers (hosts) that can be engaged in distributed testing controlled by the given project.

  • Jobs. A job is a collection of tasks that can be run simultaneously.

  • Tasks in each job. A task specifies what project (project suite) should be run on which host.

  • Synchronization points used to synchronize execution of projects running on network computers.

  • Variables that can be used to exchange data between projects participating in distributed tests and synchronize these projects.

    At run time, all projects running through a network suite share network suite variables defined in the master project. When you start the network suite of the master project, TestComplete replaces network suite variables of the slave projects with the variables defined in the master project.

Organizing Projects in Network Suites

A distributed test has a hierarchical structure. It includes one master project (the root element of this structure) and one or more slave projects. The master project manages distributed testing and runs other projects via the network suite. Slave projects are run on remote computers under the control of the master project.

Network suite structure

On this image, Project A is a master project that controls its slave projects - Project B and Project C. When you run the network suite test of Project A, it will run the tests defined in Project B and Project C on remote computers.

Note that both master and slave projects must contain the NetworkSuite project item. Otherwise, they cannot participate in distributed testing. The NetworkSuite item of the Project C and Project B slave projects is empty, its elements are ignored.

The test log of the root project will include the logs of all slave projects. For more information on distributed tests logs, see Network Suite Log.

Slave projects must reside on the remote workstations when the test starts. You can configure the master project to automatically download them from the master computer to the remote machines or you can copy them manually by using file managers or special functionality provided by TestComplete. For more information, see Copying Slave Projects to Remote Computers.

Synchronization

Distributed testing suggests that test projects executed on remote computers interact and exchange data with each other. TestComplete provides the following means of synchronization:

  • Network Suite variables - Are used to exchange data among projects involved in distributed testing.

  • Events related to distributed testing - Allow creating a script log that will react to changes in network suite variables or changes in the state of tasks, jobs or the entire network suite.

  • Critical sections - Ensure that a project works with a particular resource exclusively, without interfering with other projects.

  • Synchronization points - Ensure that several projects resume execution from a specified point.

For detailed information on synchronization, see Synchronizing Projects in Network Suites.

Known Limitations

In TestComplete, you can mark runnable items (for example, script routines, keyword tests, and so on) as test cases to include their results in a Summary report. Although you can mark your network suite’s jobs and tasks or the entire network suite as a test case, TestComplete will not track the results of actions performed on remote computers during the test run. These results will not affect the corresponding test case results. For example, if you mark a job as a test case, the test case status in the summary will be 'passed' regardless of whether there were errors during the job execution.

See Also

Distributed Testing
Distributed Testing - Requirements
Creating and Running Distributed Tests
Synchronizing Projects in Network Suites
Network Suite Variables

Highlight search results