Distributed Testing Tutorial

Author: SmartBear
Applies to: TestComplete 9-10

This article explains how you can perform distributed testing using TestComplete.

The article contains the following sections:

About Distributed Testing

Unlike ordinary tests that typically run on a single computer and do not interact with other machines, distributes tests consists of multiple parts that work on different computers and interact with each other.

Distributed tests are applied for testing distributed applications, that is, the applications that are working with multiple clients simultaneously: client/server solutions, web applications and so on. Testing a distributed application involves testing its client and server parts. You can always test them separately, but distributed testing gives you a way to test them all together. For example, the tests running on client machines can make simultaneous requests to the server, and at the same time, the server part can track and assess the server’s performance.

The crucial points of distributed tests are synchronization, administration, and configuration. For better analysis of results, the tool that you use for running distributed tests should accumulate statistics in some single common result set, while providing the ability to analyze individual results of the needed test part.

TestComplete supports distributed testing and provides features for starting tests on remote computers, exchanging data between projects during the test run, and synchronizing test projects.

This article provides a step-by-step explanation on how to create distributed tests in TestComplete. This approach works for any supported application type: desktop, web and mobile. You just need to have the appropriate TestComplete module installed.

Distributed Testing in TestComplete

To create distributed tests with TestComplete, you use the Network Suite project item:

Network suite

This project item is used for creating, managing and running test projects simultaneously on multiple workstations. You include the Network Suite item into the test project that will control the distributed test run. We call this project master project. In addition, you should also include the Network Suite item into each test project that will run on remote computers. These projects are called slave projects.

A slave project can be a master project for test projects that function at deeper level. That is, with TestComplete you can run distributed tests that have a multi-level project hierarchy:

Test structure

However, in most cases distributed tests include only two levels: master and one slave.

The Network Suite contains the Hosts collection - a collection of computers on which slave test projects will run.

Task items are used for running tests on remote computers. They specify which project or project suite should run on which host. The Task items are organized into Jobs for easier management.

One more important element of the Network Suite project item is SynchPoints. This is a collection of synchronization points that are used in test projects that participate in network suite run. For information on using synchpoints and on other means that TestComplete provides for project synchronization, see the Synchronizing Distributed Tests article.

In TestComplete you can run individual tasks, jobs and the entire network suite:

  • When you run a task, TestComplete executes the test project that is specified by the task's properties on the computer that is specified by the task's properties. Other tasks are not executed.
  • When you run a job, TestComplete runs all tasks in it concurrently. That is, tasks within a job work simultaneously.
  • When you run the network suite, you run the entire hierarchy of jobs and tasks you defined in your project. Jobs are run one after another, and within a job, tasks run concurrently.

    When you run the network suite, the master project can also work with the tested applications (for instance, simulate user actions against the application's forms), or you can use it only to control the run.

Let's see how you can create distributed tests with TestComplete.

About the Distributed Test to Be Created

Suppose, we have a physical testing lab or a virtual testing lab with three computers: MasterComputer, RemoteComputer1, and RemoteComputer2.

Let's use these computers to create a distributed test. MasterComputer will hold a project that manages this distributed test. RemoteComputer1 and RemoteComputer2 will act as remote workstations, where tests will run.

Suppose, that projects to be run on RemoteComputer1 - Project1.mds and Project2.mds - are stored on the master computer. We will configure the network suite so that it will copy these projects to RemoteComputer1 before the distributed test starts.

Projects to be run on RemoteComputer2 will be stored directly on that remote computer, the project file paths are C:\MyDocuments\Project3\Project.mds and C:\TestComplete\Projects\Project4\Project4.mds.

The rest of this article explains how to prepare TestComplete projects for testing, how to copy slave projects from the master computer to the remote computers, how to create jobs and tasks and modify their properties, how to check if the test starts successfully, and how to analyze the results.

This tutorial will describe basic operations you perform when creating distributed tests. It does not explain the following features that TestComplete offers for distributed testing:

  • Scripting objects for managing network suite, jobs and tasks.
  • Special events that you can use to control distributed test execution.
  • Network suite variables that are used for sharing data between master and slave projects.
  • Special means for synchronizing projects that participate in distributed test.

The described steps do not depend on the type of the tested application: desktop, web or mobile. With TestComplete, you can create distributed tests for any supported applications.

What You Need to Know Before Creating Distributed Tests

  • Each task within a job must work on a separate computer. That is, two tasks cannot be run on the same machine. Also, a master and slave projects cannot work on the same machine. You have to use different computers.
  • To run a task on a remote machine, you need to have TestComplete or TestExecute on that machine. The TestComplete instance that you have on your computer doesn't simulate user actions on remote computers. Instead, it sends commands to the remote instances of TestComplete or TestExecute to open slave projects and to run these projects.
  • In distributed tests, you can use various host types: physical computers, virtual machines or cloud machines. All these machines must have TestComplete or TestExecute installed on them and must be accessible within your network.
  • To run a slave project on a remote computer, the project files must be on that computer. You can copy them there manually, or you can use Network Suite to do this automatically. In the latter case, the remote machine must have some shared folder with Read/Write permissions for the account, under which you are running TestComplete on your computer.
  • In order for TestComplete or TestExecute to be able to simulate user actions on a remote computer, you need to create an interactive user session on that computer. You can create the session manually and then tell TestComplete to use it, or you can configure your network suite to create sessions automatically. The last approach is recommended.

Step 1 - Preparing TestComplete Projects for Distributed Testing

Now, we will prepare our TestComplete projects that will take part in distributed testing.

First, we will create a new TestComplete project on MasterComputer. This project will be the master project that manages the distributed test.

To create a new project, follow these steps:

Create New Project wizard

  • If TestComplete is not running yet, launch it now. Once it is running, select New | New Project from the TestComplete File menu. This will call the Create New Project wizard.
  • On the first page of the wizard, specify the project name and location and click Finish.

    TestComplete will create a new project in the specified location and display its contents in the Project Explorer panel.

To be able to participate in the distributed test, the project must contain the NetworkSuite item. To add this item to the project, follow these steps:

Create Project Item Dialog

  • Right-click the project node in the Project Explorer panel and select Add | New Item from the context menu. The Create Project Item dialog will appear.
  • In the dialog, select Network Suite and click OK.

TestComplete will add the NetworkSuite item to the project and display it in the Project Explorer panel:

Network Suite Item

Step 2 - Specifying Computers to Be Used in Distributed Test

After you created a network suite, you need to specify remote computers, on which the distributed test will run. You add remote computer links to the Hosts collection of the Network Suite project item. We are going to use two computers in our test: RemoteComputer1 and RemoteComputer2. To add them to the master project, do the following:

  • Right-click the NetworkSuite | Hosts project item in the Project Explorer panel and select Add | New Item from the context menu.
  • Specify the name for the new host, RemoteComputer1, in the subsequent dialog and click OK.

Creating Host dialog

In our case, the RemoteComputer1 name is kind of an alias that will be used for addressing the computer in tests. Now we need to specify the physical or virtual machine that corresponds to this alias.

  • Double-click the Hosts node of the NetworkSuite project item. TestComplete will show the Hosts editor in the Workspace panel.
  • Switch to the Hosts editor, click the ellipsis button in the Address column for the new host, and select the needed computer from the dialog:

Selecting Host dialog

Now we need to specify the way TestComplete will open a user session on the remote computer and the user account to be used for the session. To do this:

  • In the Hosts editor, specify the User name, Domain name, and Password of the account under which distributed testing will be performed on the remote computer. Note that the specified account must belong to the Remote Desktop Users group on the remote computer.
  • In the Login mode column select the Automatic (RDP Session) mode from the drop-down list. This will command TestComplete to open a user session on the remote computer automatically when the test starts (at that, the remote computer will be locked).

    To open the session, TestComplete will use the user account whose properties you specified in the User name, Domain and Password columns.

    Note: in order for the remote user session to be opened successfully, you need to configure the remote computer in a special way. For more information on this, see the Requirements for Remote Computers and Projects topic in the TestComplete online help.

    Now the Hosts editor looks as follows:

    Hosts editor

    The other available session modes are Automatic (Console Session) and Manual. If you use Automatic (Console Session), TestComplete will open a user session on the remote computer automatically and then retarget the session to the remote computer's console. The remote computer will be unlocked.

    If you select the Manual mode, you have to logon to the remote computer under the desired account manually before running the distributed test.

The same way, add the second host to the project. Call it RemoteComputer2. Specify its Address and login information. The Hosts editor will look like this --

Hosts editor

If needed, you can reboot hosts from the Hosts editor in the master project. To do this, use the Reboot command on the editor's toolbar or context menu:

Host editor toolbar

Step 3 - Copying Slave Projects

Before distributed testing starts, the slave projects, RemoteProject1.mds and RemoteProject2.mds, stored on the master computer should be copied to RemoteComputer1 where they will be run.

You can copy them from within TestComplete IDE by using the Copy Project to Slave command, or you can configure your master project to copy the project automatically. Whichever approach you choose, first you need to perform the following steps:
  • Open the Hosts editor in the master project.
  • From the list of hosts, select the RemoteComputer1 host.
  • In the Source path column, specify the path to the folder on the master computer holding the slave projects you want to copy. Note that by default the Source path column holds the Path to the current project suite value. If you leave the default value, all the contents of the folder, where your current project suite is stored, will be copied to the remote computer.
  • In the Base path column specify the C:\Projects string. It is a destination folder on RemoteComputer1 to which we will copy the projects. We will also use this path as a base path for the tasks to be run.

If you want the projects to be copied automatically upon the test starts, open the Properties page of the Network Suite Editor and switch the Deploy mode to Automatic.

Automatic Deploy mode

If you want to copy projects manually from within TestComplete IDE, use the Copy Project to Slave command Copy Project to Slave command of the Hosts editor toolbar to copy the projects.

Copying projects

The projects are copied to the C:\Projects folder on RemoteComputer1.

Step 4 - Creating Distributed Testing Jobs and Tasks

Now, we can create jobs and tasks in the master project.

We will create two jobs: Job1 and Job2. During the test run, the jobs will be executed one after the other.

Each of the created jobs will include two tasks. All tasks of a job will be running simultaneously during the test. Suppose, Job1 will run Project1 on RemoteComputer1 (Task1) and Project3 on RemoteComputer2 (Task2), and Job2 will run Project2 on RemoteComputer1 (Task3) and Project4 on RemoteComputer2 (Task4).

Test structure

To create a new job, right-click the NetworkSuite | Jobs project item in the Project Explorer panel, select Add | New Item from the context menu, assign an appropriate name to the job, Job1, in the resulting dialog, and click OK.

Creating job

The same way, create another job and call it Job2.

The created jobs are listed in the Jobs editor. To view this editor, right-click the NetworkSuite | Jobs node in the Project Explorer panel and then select Edit from the context menu:

Jobs editor

The jobs are executed in the order they are listed in the Jobs editor.

The execution of jobs means that the test engine runs tasks added to this job. The tasks within a job are started concurrently. To add a new task to a job, right-click the job in the Project Explorer panel, select Add | New Item from the context menu, assign an appropriate name to the task in the ensuing dialog, and click OK.

Creating task

Create two tasks in Job1 and call them Task1 and Task2.

Then, create two tasks in Job2 and call them Task3 and Task4.

Now, let’s modify the tasks’ properties. To do this, follow the steps below:

  • Double-click Task1 in the Project Explorer panel. TestComplete will open the Tasks editor in the Workspace panel.
  • Switch to the editor and specify RemoteComputer1 in the Host column of the Tasks editor.
  • Specify \Project1\Project1.mds in the Project file name column. We do not specify the full project path because we have specified part of this path in the Base path property of the RemoteComputer1 host.

    TestComplete will link the Base path and Project file name values when locating the project (the full project path will be C:\Projects\Project1\Project1.mds).

    Note that by default the Project file name column contains the File name of the current project suite value. If you leave the default value, the name of the remote project suite to be executed on the remote computer will be considered the same as the name of the current project suite.

    Task Editor

  • Repeat these steps to set the following properties of the other tasks:

    Tasks\Properties Host Project file name
    Task2 RemoteComputer2 C:\My Documents\Project3\Project3.mds
    Task3 RemoteComputer1 \Project2\Project2.mds
    Task4 RemoteComputer2 C:\TestComplete\Projects\Project4\Project4.mds

 

  Besides these properties, you can also set other properties in the Tasks editor:

  • Host timeout – Specifies the number of minutes your master project will wait for remote computers to respond.
  • Action after run – Specifies what action should be performed on the remote host after the task execution is over. You can choose to close TestComplete, log off from the remote computer, reboot the remote computer or shut the remote computer down. You can also leave the remote TestComplete instance running on the host after the task execution is finished.
  • Remote application - Specifies which testing application, TestComplete or TestExecute, will be used on the remote workstation.
  • Copy remote log - Specifies whether the task logs should be copied from the remote computer to the master computer.
  • Tag – Contains an arbitrary string associated with the task.
  • Use Previous instance - Specifies whether the previous running instance of TestComplete will be closed on the remote computer or used for the task execution.

Step 5 - Running the Distributed Test

Before we run the distributed test, we can see whether the test will start successfully. To do this, right-click the NetworkSuite node in the Project Explorer panel and select Verify command Verify from the context menu.

For each enabled task of enabled jobs, TestComplete will check whether the host computers are available, whether TestComplete or TestExecute is installed and can be launched on these computes, whether the project specified by the task’s Project file name property exists, and other necessary actions.

If one of the automatic modes is selected for a remote computer, TestComplete also opens a user session on that computer when you are verifying (or running) network suite. To open a user session, TestComplete uses account information specified in the Hosts editor.

If the verification is successful, you can run the distributed test.

You can run a whole network suite or its separate jobs and tasks. They can be run from the TestComplete UI or from keyword tests or scripts using special keyword-test operations or special scripting methods and objects.

In our case, we will run the distributed test from the TestComplete UI. To do this, right-click the NetworkSuite project item in the master project and select Run from the context menu.

Running Network Suite

During the run, the Run State page of the NetworkSuite editor will display the information about the network suite, job, and task execution. This page also displays the remote desktop windows of the remote hosts allowing you to observe the test run on the remote computers directly from the master project.

Run State page

TestComplete will run the network suite until all of its enabled jobs are completed. Each job, in its turn, runs until all of its enabled tasks are completed.

Each job of the network suite coincides with the previous job and will run only after the previous job has finished. Thus, TestComplete will first execute the tasks of Job1, wait for the end of their execution, and then, run the tasks of Job2. Task1 and Task2 will be executed simultaneously as well as Task3 and Task4.

Step 6 - Viewing Results

After the distributed test is over, we can analyze its results displayed in the master project test log:

Test Log

The distributed test log includes the master project log and the logs of the slave projects executed on the remote computers.

The master project log is stored on the master computer. Depending on the Copy remote log property, the slave project logs can be copied to the master computer and stored in the master project log folder, or they can be stored on the remote computers, where the slave projects were executed.

The Log Items panel displays the hierarchical structure of the distributed test log. The top-most item corresponds to the master project log. Its child items represent the structure of the executed jobs and tasks. The remote test logs to which the tasks refer are displayed as the child nodes of the task nodes:

Test Log task nodes

To view the remote test log content from the master project, select the corresponding node in the Test Items panel:

Test Log test items

Conclusion

This article briefly explains TestComplete support for distributed testing and explains how you can create distributed automated tests. We hope this information will help you start using distributed testing in your company.

For more information about distributed testing, see our Distributed Testing video.

If you are interested in trying TestComplete for free, download and try it today.