Load Testing With TestComplete - Introduction

Applies to: TestComplete 8

Preface

Load testing consists of simulating a load on web servers and services to determine how web applications function under a massive load. TestComplete supports load testing of web servers and web services and can be used to perform relative testing types like stress testing and scalability testing. This article describes how you can create load tests with TestComplete.

About Load Testing

Web applications work in an open environment. They process requests sent to them by different clients. In most cases, the applications have to work with several clients at one time. The purpose of load testing is to determine whether the application functions properly under a massive load, to find the stress point that causes crashes or delays when processing requests and to determine whether the application is scalable enough to handle an increased workload.

Performing load tests manually is often impossible because companies usually do not have enough testers that can perform load testing manually, at the same time. Load testing is performed with automated testing tools like TestComplete. Below is a list of conditions that should be met by any load testing tool to perform quality load testing:

• It should provide features for creating load tests easily. It is very important that load tests can be recorded and then visually modified if needed.
• It should be able to work with different web servers and browsers. Ideally, it should not depend on the web server or browser type.
• It should be able to simulate a heavy workload by automating an arbitrary number of virtual users. Virtual users are called virtual because they don’t exist. Only the actions performed by these users exist. When virtual users are simulated the tested web server "thinks" that it is working with "real" users.
• The load testing tool should be able to simulate real-life load testing conditions as close as possible. It should be able to simulate users working with different parts of the application, working from different workstations, different web browsers, with different connection speeds and other real-life conditions.
• To help you perform load testing of your web application, the automated testing tool must provide access to requests and responses. For instance, it should let you modify request parameters or the request body.
• Finally, the load testing software must generate detailed result logs, support result comparison, help you find crash points and check the scalability.

Load testing tools that match these criteria will help you easily emulate real-life conditions for your web applications and simulate the desired behavior of "users" during load testing.

Load Testing with TestComplete

TestComplete provides support for load testing of web applications. The support is included into the TestComplete Enterprise edition. The load testing subsystem meets all of the criteria mentioned above.

Record Traffic for Load Testing

Load testing consists in simulating virtual users each of which sends requests to the tested web server and receives responses from the server. To simulate the traffic with TestComplete, you record user actions over the tested web application and these actions are then reproduced during load testing.

TestComplete allows you to record both requests and responses sent via HTTP, HTTPS or SOAP protocols.

HTTP is a standard protocol of the Web. It is used to access a vast majority of web sites.

HTTPS is a secure version of HTTP. This protocol is used to secure data sent from one computer to another via the Internet. For example, when an e-shop asks users for their credit card numbers and other daily duties performed with an e-commerce type web application.

Another type of request that can be recorded with TestComplete is SOAP. The SOAP protocol is used for communication between web services and their client applications. Since TestComplete is able to record and play back SOAP traffic, you can use it to perform load testing of web services.

TestComplete records and stores traffic in the form of self-contained modules (tasks). Each task is a sequence of user actions to be simulated by one or more virtual users. TestComplete allows you to record as many tasks as you need.

TestComplete also provides a special editor that allows you to view and visually modify properties of recorded requests and responses.

Server and Browser Support for Load Testing

TestComplete records and plays back traffic, but not browser actions, so you can use it to test web sites regardless of the web browser you are using (Internet Explorer, Netscape Navigator, Mozilla, Opera, or any other), regardless of the web server type (Internet Information Services, Apache, and so on) and regardless of the platform (Windows, Unix, Linux) on which the web server is working.

Visual Creation of Load Tests and Virtual User Simulation

Recorded load testing tasks are played back via virtual users. To simulate virtual users, you create load tests in TestComplete.

The number of virtual users that can be added to a load test is specified by a special virtual user license, which is purchased separately from a TestComplete license. Without a virtual user license TestComplete does not allow you to use virtual users in your load tests. You can purchase a license for 5, 50, 250 or unlimited number of virtual users on the Order TestComplete page of our web site. A license for 5 virtual users may be enough to test a simple web application. If you want to emulate more load on your web server, you can buy other versions that support more virtual users.

In TestComplete, load tests can be created and executed visually or in scripts. TestComplete provides special program objects to execute load tests created visually and to create load tests programmatically. Both of these approaches let you create the desired number of virtual users and specify their attributes (for example, start delay, web browser, connection speed, and others).

The visual approach is easier to use and allows even inexperienced users to quickly create load tests. The scripting approach lets you specify test attributes from scripts, but it requires that you have more programming experience.

A load test may contain dozens or even hundreds of virtual users that work simultaneously.

Virtual users can belong to the same load test but perform different load testing tasks. That is, you may simulate several users that work simultaneously but perform different actions on the server.

Real-Life Load Testing Conditions

For better emulation of real-life load testing on web servers, TestComplete offers a number of attributes that you can use to load test your web application:

• Workstation. You can use this attribute to organize virtual users on several workstations in your network. This doesn’t affect the sequence of actions performed by the virtual user or the load test that each user belongs to. Workstation only specifies what computer the virtual user will perform its load testing tasks on. To simulate virtual users on remote computers, you can install TestComplete on these computers or use TestExecute or the TestComplete HTTP Load Testing Remote Agent (RA). In other words, there is no need to install TestComplete on all remote workstations where RA or TestExecute is installed. TestExecute is a resource-friendly utility that is aimed at running TestComplete projects. TestComplete Enterprise includes one license of TestExecute. You can order additional load testing licenses from AutomatedQA’s web site. Remote Agent is a utility that provides communication between the computer where TestComplete is installed and remote workstations. It is supplied along with any virtual user license package.
• Browser. This attribute allows you to emulate virtual users working on different web browsers. This can help you determine whether the tested web server has any problems when processing requests sent by some browser. TestComplete can emulate the following browsers:
  - Internet Explorer v. 5.0 - 8.0
  - Firefox v. 1.0.1 - 3.5
  - Opera v. 8.0, 9.5
  - Netscape Navigator 6.0
  - Pocket IE
  - SmartPhone
  - Any browser that was used during traffic recording.
  If you create a load test visually, you can easily select the desired browser for virtual users:

Figure 1 – Selecting a browser for virtual users

• Start Delay. Using this attribute you can set the time delay (in milliseconds) between the start of virtual user simulation and the start of the load test (by default, the simulation of all virtual users that belong to the load test is started when the execution of the load test is started). This can help you to emulate a growing load on the tested server.
• Connection Speed. This attribute lets you specify the connection speed of the virtual user that works with the tested web server. TestComplete emulates bandwidth of the download stream. The value of this attribute can be selected for every virtual user:

Figure 2 – Selecting connection speed for virtual users

The bandwidth of the upload stream can not be simulated.

Cookie Handling During Load Testing

Cookies are data stored by the web server on the client computer. They are used by the server for user authentication and for maintenance of client-specific information.

TestComplete includes a special option, Cookie handling, that helps you manage cookies when the recorded traffic is simulated by virtual users. This option helps you avoid some typical problems that may happen during load testing: cookies may expire and this can make your load tests invalid; cookies may cause cross impact of virtual users on each other, and other related issues. TestComplete lets you avoid these problems.

Proxy Servers in Load Testing

With TestComplete you can play back load testing tasks even if the computer, on which TestComplete is installed, is behind a proxy. This feature is useful if TestComplete does not have direct access to the tested web server. TestComplete includes specific settings that control the proxy settings.

Authentication During Load Testing

Web servers may provide anonymous or authenticated access to the contents of the server. If an anonymous connection to the web server is used, this server can be tested without any restrictions. However, there are some restrictions when testing web servers with certain types of authenticated connections. TestComplete supports most of the authentication types:

• Basic authentication
• Digest authentication
• NTLM
• Kerberos authentication
• Negotiate authentication

It provides special properties in which you can specify authentication information that is used to connect to web sites during load testing.

Modifying Traffic During Load Testing

TestComplete provides access to properties and parameters of requests to be sent to the tested web server and allows you to modify the recorded traffic, if needed. For example, if your web application’s pages contain an order form, then for advanced load testing of this application you have to emulate several users inputting different data when completing the order data.

Web browser can send data in a URL string or within the request body depending on the request type (GET or POST type). TestComplete supports both request types and allows you to change the request data visually or from scripts using special events and program objects. Server response properties can be changed in the same way.

To provide virtual users with different data to be sent to the server during a load test run, TestComplete allows you to create special load testing variables. Each load testing variable can be associated with a list or with a range of values of a certain type and return different values for different virtual users. By setting the values of such variables as request parameters, the recorded traffic can be modified and your virtual users will be able to generate different requests to the server during the test run. For example, they can log in using individual user names and passwords. As a result, you will be able to get realistic test results.

Analyzing Results After Load Testing

Results of the load test execution are stored in TestComplete’s Load Testing Log. This page contains detailed information on the load test and specifically about the simulated virtual users, information on each connection that was created during the test run, and much more.

Figure 3 – The Load Test Log page of TestComplete

If you need to compare load testing results for several test runs, you can do this on the Load Testing Analysis page. This page shows the table including the summary results of each load test run (average process time, average request size, the total number of requests, and more) and a diagram that graphically represents the load test results.

Figure 4 –The Load Testing Analysis page

Load Testing Sample Project

Below is an example that demonstrates how to perform load testing of a web application with TestComplete. Let’s perform load testing of the Load Testing Demo sample web application that is located at http://localhost/loadtesting/loadtestingdemo.dll. This web application is installed automatically along with other TestComplete samples and can be found in the <TestComplete Samples>\Load Testing\Application folder. To install and invoke this sample, Microsoft Internet Information Services (IIS) 5.0 or later must be installed on your computer.

Below are instructions on how to perform load testing of this application with TestComplete.

Creating a TestComplete Project for Load Testing

Before recording traffic and creating load tests, create a new TestComplete project:

• Select File | New | New Project from the TestComplete’s main menu. The Create New Project wizard will appear.
• On the first page of the wizard, specify the project’s name and location and then click Next. For example:

Figure 5 – The Create New Project page of the Create New Project wizard

• On the Tested Application Type page of the wizard, choose the Web application type and click Next.

Figure 6 - The Tested Application Type page of the Create New Project wizard

• On the Web Testing Settings page of the wizard, choose Load testing of web servers and click Next

Figure 7 - The Web Testing Settings page of the Create New Project wizard

• On the Project’s Scripting Language page of the wizard, choose the scripting language you prefer to use in tests of your project and then click Finish. For example, let’s choose JScript:

Figure 8 - The Project’s Scripting Language page of the Create New Project wizard

After the Create New Project wizard is finished, TestComplete creates a project suite containing the new test project with the specified name and location. If you specified all the settings in the wizard as it is described above, the newly created test project already contains the HTTP Load Testing project item, a script unit and a keyword test. The HTTP Load Testing project item will hold all information needed to perform load testing.

Figure 9 - The HTTP Load Testing project item in the Project Explorer

The created test project is ready for load testing. You can now record HTTP traffic to test your web server.

Recording Traffic for Load Testing

To record load testing traffic:

• Double-click the LoadTesting | Stations project item in the Project Explorer panel. The LoadTesting project item editor will appear.
• On the General Settings page of the editor, click the Record Load Testing Task button:

  
  

  
  
  Figure 10 - Starting recording

  TestComplete shows then the Specify Load Testing Test dialog.
• In the dialog, specify the name of the load test and load testing task to be created (TestComplete records traffic as self-contained modules called tasks). Enter MyTest as the test name and specify local as the name of the new task. From the drop-down list, choose a web browser to run for recording a task. It is recommended also to clear the selected web browser’s cache, cookies and temporary files before recording starts. You can command TestComplete to do this by selecting the Clear browser data check box in the dialog.

  
  

  
  
  Figure 11 - The Specify Load Testing Test dialog

  Click OK. TestComplete will launch the selected web browser and display the Recording toolbar with the Record Load Test button pressed. This means that TestComplete has just started recording the traffic.

  
  

  
  
  Figure 12 - The Recording toolbar

• Open the page to be tested in the browser. In our example, the page’s URL is http://localhost/loadtesting/LoadTestingDemo.dll. Enter it into the Address box of the web browser.
• After the page opens, perform the desired actions over it. For instance:
  • Click Start Web Application on the main page to start working with the sample web application.
  • Go through the pages of the application by clicking Continue until you reach the last page. On each page you can see an image.
  • Click Finish after you get to the last page.
• Stop recording the task by pressing Stop on the Recording toolbar.

  
  

  
  
  Figure 13 – Stopping recording

Viewing the Recorded Load Testing Traffic

Now, let’s explore the contents of the recorded local task. This task is displayed as a child node of the LoadTesting | Tasks node in the Project Explorer. To view the contents of the task, simply double-click its node in the Project Explorer panel. TestComplete will open the Load Testing Task editor in the Workspace panel:

Figure 14 – The local task in the Load Testing Task editor

On the left of the editor you can see the contents of the recorded HTTP traffic. As you can see, it is organized into a tree. By default, this information is grouped by web pages loaded during test recording. Nodes that correspond to web pages contain child nodes that correspond to HTTP requests sent to the tested web server to obtain the contents of the appropriate web pages. Also, the editor can display the traffic information grouped by network connections that were created during test recording. In order to group information by connections, simply right-click the tree in the editor and select Group by Connections from the context menu.

The Load Testing Task editor contains three tabbed pages on the right: General, Request and Response. You can select a request in the traffic tree on the left of the editor and in the corresponding tabbed page view and modify the request’s properties or properties of its response. For example, expand the Page 0001 node, select the Request 0008 item and switch to the Request tabbed page, as it is shown in the figure above. On this page you will see a table containing values of the selected request properties. You can modify values, if needed. For instance, in the User-Agent field you can change the browser to be simulated. On the top of the Request tabbed page, you can see the request’s header. If it contains query string parameters (they are specified after the ‘?’ symbol), you can easily modify the parameter values in the Edit Query String Parameters dialog that is invoked when you click the ellipsis button within the edit box containing the request’s header.

On the Response tabbed page, you will see the contents of the response that was received from the tested server for the currently selected request:

Figure 15 - The Response page of the Load Testing Task editor

In the Response page you can view and modify response properties.

In our tutorial, we will not modify the recorded traffic. So, leave it unchanged.

Creating a Load Test

Now we can create a load test and simulate virtual users. TestComplete automatically created a new load test named MyTest after traffic recording was finished. You can see this test in the Project Explorer panel under the LoadTesting | Tests node:

Figure 16 – The MyTest load test in the Tests collection

Let’s open this load test for editing. To do this, double-click the MyTest node in the Project Explorer. TestComplete will display the load test editor in the Workspace panel:

Figure 17 – The MyTest load test in the Load Test editor

In the Virtual Users table you can view and modify information on virtual users to be simulated during the load test run and their properties.

As you can see, in our case the table contains only one row that corresponds to one virtual user (the number of virtual users is specified in the User Count column). This is the user that was created automatically by default.

Let’s change the load test. As you may remember, the minimum number of virtual users provided by a virtual user license is 5. Let’s create a test that contains 4 virtual users. These users will "work" with different web browsers.

• Click within the User Count column of existing row in the Virtual Users table.
• Specify 2 in the in-place editor and press ENTER to confirm the input.
• In the User Name column specify the string for identifying testing results of the virtual user simulation in the log. For example, enter TestUsers1. In the log, virtual users will be identified as TestUsers1 (1) and TestUsers1 (2).
• Make sure the other columns contain the following data:
  • Task: local - This means virtual users will simulate the traffic that is stored in the local task.
  • Workstation: Master - This means virtual users will be simulated on the computer, on which TestComplete is installed.
  • Browser: <Recorded> - This means virtual users will "work" through the same browser that was used for recording the task.
  • Start Delay: 0 - Start Delay is the time between the start of the test and the start of virtual user simulation. If Start Delay is 0, the virtual user simulation starts when the test starts.
  • Connection Speed: Maximum speed - The download traffic will be simulated at maximum allowed connection speed.
  • Correct Host Field: Enabled - Specifies whether TestComplete should correct the Host field of recorded requests.

Let’s create two more virtual users:

• Right-click somewhere within the Virtual Users table and choose New Item from the context menu. This will add a new row to the table.
• Specify 2 in the User Count column.
• In the User Name column, enter TestUsers2.
• In the Browser column, choose Internet Explorer 8.0 from the drop-down list.
• Specify the following values for the other columns:
  • Task: local.
  • Workstation: Master.
  • Start Delay: 0.
  • Connection Speed: Maximum speed.
  • Correct Host Field: Enabled.

All virtual users will be simulated simultaneously as the Run concurrently check box is selected.

In the Load Test editor you can see properties of the modified load test:

Figure 18 – The modified MyTest load test in the Load Test editor

Executing the Load Test

In TestComplete, you can run load tests from the Project Explorer, from keyword tests, from script code, from the Load Test editor or you can run them as a project test item. Let’s use the project test item approach:

• Double-click the project node in the Project Explorer to open the project editor in the Workspace panel.
• Choose the Test Items page in the editor.
• Right-click somewhere within this page and select New Test Item from the context menu or click the New Test Item button on the editor’s toolbar. A new test item will be created.
• In the Test column, click the ellipsis button. This will invoke the Select Test dialog.
• In the dialog, select the LoadTesting | Tests node from the tree on the left (the Test Categories section) and choose MyTest on the right (the Available Tests section):

Figure 19 – The Select Test dialog

• Click OK.

Now our project contains the test item that will run the MyTest load test:

Figure 20 - The Test Items page in the project editor

To run the test, select Test | Run Project from TestComplete’s main menu.

TestComplete will start the test execution and display the indicator that will reflect the status of load tests:

Figure 21 – Playback Indicator

The rest of load test procedure is invisible: TestComplete simulates recorded traffic (sends requests to the tested web server and receives responds to these requests from the server), but it does not simulate user actions over the tested web application’s pages (mouse clicks, keystrokes, and so on).

After the test run is over, you can view the load testing results.

Analyzing Load Testing Results

You can get detailed information about the results of your load testing from TestComplete’s Load Testing Log page:

Figure 22 – Results of the load test run

On this page you can find detailed information about simulated virtual users (user name, task name, workstation on which the user was simulated, time of the task execution, and more), information about each connection that was created by the task during load testing (the number of requests the user sent during the connection, the number of bytes sent by all requests, and others), information about requests sent during each connection and other useful information about the load test run.

If you run the load test several times, for example, with a different number of users, it may be necessary to compare results of these test runs. For this purpose, you can use the summary results from the Load Testing Analysis page:

Figure 23 – Summary results in the Load Testing Analysis page

Conclusion

In this article we have provided an overview of load testing and its basic concepts, and also described TestComplete’s features that are used to record load testing requests sent to the tested web server and to simulate this traffic via a number of virtual users. We hope this information will help you with your load testing. If you are interested in trying TestComplete for your load testing needs, download and try it today.