Cloud Testing With TestComplete

Applies to: TestComplete 10

This article opens a series of articles on using TestComplete in cloud testing.

For related technical articles on cloud testing, please see the Other Articles About Cloud Testing section.

About Cloud Computing

These days, a computer technology involving the provision of computer resources as a service over the Internet is drawing more and more public attention. This technology was named cloud computing.

Today, a lot of companies provide cloud computing services. The most famous among them are Amazon, Yahoo, Sun Microsystems, Microsoft, Google, RackSpace and Zoho. An important thing about the cloud model is that hardware, software and services are not purchased, but rented instead. Besides that, all of the needed hardware and software are located in the providers’ data centers and are accessed via web interfaces, special software (for example, Remote Desktop Connection) or special API’s. Services supplied by cloud computing providers can be split into the following major categories:

  • Infrastructure as a Service (IaaS)
    Here, users are provided with:
    • Hardware: servers, data storage systems and network equipment.
    • Operating systems and system software.
    • Software to control the provided services.
  • Platform as a Service (PaaS)
    • Execution environment.
    • Development environment.
    • Team development services: version control, task management and issue-tracking systems.
    • Testing environment.
  • Software as a Service (SaaS)
    • Access to services (for example, Microsoft Office Live, Google Docs, Gmail) implemented as one or several applications.
    • Updates of applications that provide the services (transparent for service customers).
    • Technical support.

These services are designed for accomplishing a wide range of tasks: from using a mailbox to organizing the entire infrastructure of a company including fully equipped work places — these can be computers residing in a cloud which are accessed via Remote Desktop Connection.

About Cloud Testing

Cloud computing can also be used to test various applications. A testing process involving the use of cloud resources is called cloud testing. It tests any programs and allows you to perform any type of tests (for example, performance, functional and usability tests). Here, we should note that using resources in a cloud is especially helpful when the testing process involves several test machines (for example, using several workstations for distributed testing) or when you are testing applications that operate over LAN or Web (client/server applications, web services and others). So, cloud testing users can benefit from the following key characteristics of this approach:

  • Tests scalability. For example, you can rapidly increase or reduce the number of test machines, the amount of the disk space needed to store data, and so on.
  • Using large amounts of computing resources. Cloud resources are virtually unlimited. For example, you can rent 1,000 workstations and use them to simulate massive load on one web server under test.
  • You have no need to organize your own infrastructure to run tests. That is, you don’t have to purchase new computers, connect them to the LAN, install the needed operating systems, and so on. Note that the cloud model allows renting pre-configured test computers with pre-installed operating systems and other software (for example, Microsoft SQL Server 2008). You can combine your infrastructure with that deployed in the cloud. All of these save you time so that you can organize an infrastructure required for testing.
  • Reliability of the provided infrastructure.

Cloud resources must be paid for. However, the benefit is that they are available on demand and you only pay for those resources that you actually use. Providers offer various payment options, but typically you pay for:

  • The amount of time the given test computer worked. As of January 2014, the average time-based fee for a machine capable of running tests is about $0.10 per hour; though, there are other fee options — for example, you can choose for a monthly subscription basis, an annual subscription basis, etc.
  • The amount of data transferred to and received from the provided computer (on average, $0.09 per 1 Gb of transferred data and $0.19 per 1 Gb of received data). Note that you only pay for data transfer when the data is being transferred outside of the local network of the cloud, and most cloud providers do not charge for incoming data.
  • Data repository capacity (data can change dynamically thus affecting the cost of the repository) and the time period the repository is used (on average, $0.10 for 1 Gb per month).

The number of companies that choose to use cloud resources is constantly growing as this model tends to be less capital-consuming than purchasing new computers, network equipment, licenses for operating systems and software.

Using TestComplete in Cloud Testing

Using cloud computing, you can organize your own infrastructure to test applications in a cloud. Here, you can have all of your test workstations located in one internal local network of the provider’s cloud (there can be several internal local networks in the provider’s cloud). This allows the data transfer rate in such a network to be the same as a regular LAN. Also, the data transfer in this network is free of charge.

As for access from other machines located both inside (including those in one local network) and outside the cloud, you can gain access by using public DNS names, external IP addresses or a VPN. However, in this case, you will have to pay for data transfer charges (except for the situation when test computers located in one internal local network of the cloud are connected via the VPN and local IP addresses are used).

One can install TestComplete as well as TestExecute on computers in such networks.Then, it can be used as if it were operating in a regular local network. Then, distributed tests in TestComplete will be set up in the same way as you would normally do this in a LAN. However, we should make an important note here: it’s more reasonable to specify local IP addresses for the computers involved in the testing process, wherever required (for example, when setting up workstations for Network Suite, when creating DCOM objects, and so on), not to pay for traffic.

Thus, a testing process deployed in a local network that involves using TestComplete can easily be transferred into a cloud fully or partially. This way, all cloud testing strong sides — scalability, infrastructure reliability, availability of large computing resources, and others — will be at your disposal. Here, you also can organize your testing process so that it does not use computing resources every day, rather, once a month, for example. This will allow you to save expenses on costs of the rented resources and still perform intensive stress tests from time to time.

TestComplete and Amazon Elastic Compute Cloud

We used the Amazon Elastic Compute Cloud service (Amazon EC2) — the IaaS provided by the Amazon company — to test TestComplete’s functionality. The test results showed that using TestComplete in a cloud or over a VPN does not differ from using it inside a regular LAN.

A test can be run from a test instance where TestComplete is installed (test instance or just instance is a term for a test computer residing in a cloud). You can run the test connecting to the instance using the Remote Desktop Connection. Alternatively, advanced users can implement a web interface, which will run the test by using TestComplete’s COM interfaces. Also, you can run the test from your local computer where TestComplete is installed if it is connected to the instances in the cloud via the VPN.

The following image illustrates the ways of organizing cloud testing with TestComplete:

Cloud testing with TestComplete

Here, test instances are the computers in the cloud which are set up to run tests. On these computers TestComplete, TestExecute or the HTTP Load Testing Remote Agent and other software needed for the testing process are installed.

We will continue discussing ways of organizing cloud testing with TestComplete in the ensuing articles.

Licensing TestComplete on Cloud Computers

TestComplete uses two license types: the Node-Locked license that is bound to one computer and the Floating User license that allows running several TestComplete instances concurrently at a time.

Since cloud computers are virtual computers, licensing TestComplete in cloud environment is similar to licensing TestComplete in virtual environment.

The Node-Locked license cannot be used on cloud computers. In order to run TestComplete on cloud machines, you need the Floating User license. This license type allows you to install and run TestComplete instances on cloud computers, but the License Manager should be installed and activated on a physical computer. In other words, the License Manager PC should be a physical machine.

Licensing TestComplete on Cloud Computers

The cloud computers on which you install TestComplete must have network access to this physical machine, and the TestComplete license search settings on the cloud computers must point to this machine. For more information on setting the license search options, see the Specifying License Manager for Connection topic in TestComplete Licensing Guide.

If you hold the Node-Locked license, to run tests in cloud environment, you need to upgrade to the Floating User license. An alternative approach is to run tests on cloud machines with TestExecute.

TestExecute is a lightweight resource-friendly utility for running tests on computers where TestComplete is not installed. It is always licensed with the Floating User license. So, you can install it on cloud computers and use to run tests there. In this scenario, TestExecute instances will retrieve a license for the run from the TestComplete License Manager (that is, from the physical computer on which the License Manager utility is activated). You can purchase additional TestExecute licenses to increase the number of testing machines. Cloud computers must have network access to this computer and their license search settings must point to it.

For more information on licensing TestComplete in cloud environment, see Using TestComplete in Virtual Environment or TestComplete Licensing Guide.

Note for users of the TestComplete trial version: don’t activate the TestComplete trial version on cloud computers. Due to specifics of license activation, you will not be able to use the license after your cloud computer stops. To try running tests in cloud environment, activate the license on a physical machine and use TestExecute to run tests on cloud computers. See How to Activate TestComplete Licenses for detailed information.

Conclusion

In this article, we have considered cloud services and described how TestComplete can be used in cloud testing. Tests organized with TestComplete (including distributed tests) in a cloud or over a Virtual Private Network (including test instances in the cloud or not) are set up in the same way as this would be done in a regular local network. We hope this information will help you create a flexible and effective testing infrastructure.

Further your knowlegdge of TestComplete with SmartBear's official test automation training.

Other Articles About Cloud Testing