Distributed Testing - Basic Concepts

Distributed and Non-Distributed Testing

Distributed Applications

TestComplete Network Suite

Organizing Projects in Network Suites

Synchronization

Known Limitations

Distributed and non-distributed testing represents a different approach of how to classify testing methods. Non-distributed tests are run on a single computer and normally do not interact with other computers.

On the contrary, distributed testing means that a test consists of several parts and each of them is performed on a separate computer.

However, the term “distributed” means not only a simultaneous run of tests on a number of machines, which is specific for parallel testing. Distributed testing also suggests that test parts interact with each other during a test run. This makes them coordinated and synchronized in the appropriate manner. Actually, synchronization is one of the most crucial points in distributed testing.

Another key point is administration and configuration of the test. For better analysis of results, statistics on running of all test parts should be accumulated in a single common result, while you should still keep the ability to analyze individual results of the desired test part.

In addition, you should be able to repeat the distributed tests consistently, in the way it is required for regression testing.

Distributed testing is normally needed for web or intranet sites, client-server or other multi-tier applications, where a server application interacts with more than one client. We can call applications of this type distributed applications.

Testing of a distributed application involves testing of its client and server parts. You can test them separately. Distributed and simultaneous testing gives you a way to test them all together.

For instance, a test makes a client instance send requests to the server, at the same time due to several clients working together you can assess how the server performs. Note that you can extend the test if while the server is processing client requests, you will profile the server or a client instance (or both) using a debugging/profiling tool, for example, AQTime.

In order to support distributed testing, TestComplete provides the NetworkSuite project item. TestComplete network suites are intended to manage the runs of several test projects or project suites on different computers. Test projects can run in a series or simultaneously. You can handle the network suite via the TestComplete user interface or from your keyword tests or scripts, which makes the testing more flexible.

The network suite functionality is implemented by a group of Network Suite plugins:

• Network Suite Core
• Network Suite Host List Editor
• Network Suite Job List Editor
• Network Suite Run State
• Network Suite Task List Editor

These plugins are installed and enabled automatically during the TestComplete installation. If you experience issues with the network suite functionality, check whether the plugins are enabled in the File > Install Extensions dialog.

The Network Suite Core plugin provides TestComplete with the NetworkSuite project item, which can be added to a project. Only projects that include the NetworkSuite project item can run other projects or project suites or can be run by other projects. That is, the network suite gives a project the capability to control the run of several projects on different network computers.

The NetworkSuite project item lets you configure the following items:

• Computers (hosts) that can be engaged in distributed testing controlled by the given project.

• Jobs. A job is a collection of tasks that can be run simultaneously.

• Tasks in each job. A task specifies what project (project suite) should be run on which host.

• Synchronization points used to synchronize execution of projects running on network computers.

• Variables that can be used to exchange data between projects participating in distributed tests and synchronize these projects.

  At run time, all projects running through a network suite share network suite variables defined in the master project. When you start the network suite of the master project, TestComplete replaces network suite variables of the slave projects with the variables defined in the master project.

A distributed test has a hierarchical structure. It includes one master project (the root element of this structure) and one or more slave projects. The master project manages distributed testing and runs other projects via the network suite. Slave projects are run on remote computers under the control of the master project.

On this image, Project A is a master project that controls its slave projects - Project B and Project C. When you run the network suite test of Project A, it will run the tests defined in Project B and Project C on remote computers.

Note that both master and slave projects must contain the NetworkSuite project item. Otherwise, they cannot participate in distributed testing. The NetworkSuite item of the Project C and Project B slave projects is empty, its elements are ignored.

The test log of the root project will include the logs of all slave projects. For more information on distributed tests logs, see Network Suite Log.

Slave projects must reside on the remote workstations when the test starts. You can configure the master project to automatically download them from the master computer to the remote machines or you can copy them manually by using file managers or special functionality provided by TestComplete. For more information, see Copying Slave Projects to Remote Computers.

Distributed testing suggests that test projects executed on remote computers interact and exchange data with each other. TestComplete provides the following means of synchronization:

• Network Suite variables - Are used to exchange data among projects involved in distributed testing.

• Events related to distributed testing - Allow creating a script log that will react to changes in network suite variables or changes in the state of tasks, jobs or the entire network suite.

• Critical sections - Ensure that a project works with a particular resource exclusively, without interfering with other projects.

• Synchronization points - Ensure that several projects resume execution from a specified point.

For detailed information on synchronization, see Synchronizing Projects in Network Suites.

In TestComplete, you can mark runnable items (for example, script routines, keyword tests, and so on) as test cases to include their results in a Summary report. Although you can mark your network suite’s jobs and tasks or the entire network suite as a test case, TestComplete will not track the results of actions performed on remote computers during the test run. These results will not affect the corresponding test case results. For example, if you mark a job as a test case, the test case status in the summary will be 'passed' regardless of whether there were errors during the job execution.