Preparing Your Cloud Environment for Automated Testing

Applies to: TestComplete 8.20-10

This article explains how to use Amazon Web Services for software testing and provides a step-by-step description of the procedure for preparing your Amazon cloud environment for automated testing with TestComplete. For related technical articles on cloud testing, please see the Other Articles About Cloud Testing section.

In this article:

About Amazon Web Services

One of the most famous cloud computing providers is the Amazon Company. It offers a lot of services that are appropriately called: Amazon Web Services (AWS). For testing purposes, the following three services tend to be the most important:

  • Amazon Elastic Compute Cloud (Amazon EC2) – provides workstations with different technical features and certain operating systems and software installed on them, as well as load balancers, which distribute the workload among these workstations;
  • Amazon Elastic Block Storage (Amazon EBS) – provides hard drives for workstations launched using the Amazon EC2 service;
  • Amazon Simple Storage Service (Amazon S3) – provides the ability to store and receive data.

For detailed information on these services, please visit the Amazon Web Services web page - Here, you can also find the latest news and documentation, communicate on special forums and get access to the AWS Management Console (a web interface to manage services provided by AWS).

Preparing a Cloud Environment

Step 1 – Creating an Amazon Account

To work with the services, you need to have an Amazon account. To create an account, open the web page and click Sign Up Now:

Amazon Web Services

Figure 1. The AWS web page

Use the web page that appears after that to either sign in if you already have an Amazon account or create a new account:

Amazon Web Services sign in

Figure 2. The AWS web page to sign in the system or to create a new Amazon account

To create an account, you will need to specify your e-mail address and a password to sign into the system. You will also need to specify some contact details – your name, the company address, your phone number, and so on. Follow the instructions in the Create an Amazon Web Services Account wizard, and after finishing all of the steps, you will see a web page indicating that a new account has been created and providing links to Amazon Web Services:  

Amazon Web Services account

Figure 3. The web page displayed after an Amazon account is successfully created

Step 2 – Signing up for Amazon EC2

After the Amazon account has been created, you need to sign up for Amazon EC2. To do this, go to the page and click Sign Up for Amazon EC2:

Amazon EC2 service

Figure 4. The Amazon EC2 service web page

If you are not logged in, the system will ask you to follow the steps to create an account and login.

After that, the page providing up-to-date pricing for all available resources appears. On this page, you can also select the payment method you prefer:

Amazon EC2 service pricing

Figure 5. The web page to view pricing for the Amazon EC2 service’s resources and select a payment method

To select a payment method for the services you will use, you first have to log in and click Continue. Then, you will be offered to perform additional actions required to work with Amazon EC2 (selecting the billing address associated with your credit card, identity verification by phone, and so on). Follow the on-screen instructions, and after all of the steps have been successfully completed, you will be able to log into the system.

Step 3 – Getting Acquainted With the AWS Management Console

After you have logged into the Amazon Web Services, click the AWS Management Console link:

Amazon Web Services Management Console

Figure 6. Switching to the AWS Management Console

You can use the console page to manage the Amazon EC2, Amazon Elastic MapReduce and Amazon CloudFront services. At this stage, we are interested only in Amazon EC2 service. This service is selected by default. If you have switched to another service, click Amazon EC2 in the top left corner of the web page to return to the Amazon EC2 settings:

Amazon Web Services Management Console

Figure 7. AWS Management Console

The Navigation panel contains the links to the pages designed for managing various resources of the Amazon EC2 service. The image above demonstrates the Amazon EC2 Console Dashboard page (the link to it is EC2 Dashboard in the Navigation panel). In the My Resources group on this page, you can see what resources are being currently used.

Amazon’s data centers are located in three different geographic regions. You can view them in the Region drop-down list.

Amazon data center's regions

Figure 8. Regions where Amazon’s data centers are located

Each Region contains its own set of used resources. You can view and manage these resources if you select the appropriate item in the Navigation panel.

Also, each Region contains several availability zones – these are the data centers interconnected with a high-speed network and constructed so that malfunctions in services’ functioning in one of the availability zones doesn’t affect services’ functioning in another availability zone.

The use of several Regions and Availability zones may play an important role when organizing distributed tests. We will discuss this in future articles.

Step 4 – Preparing to Launch an Instance

For testing purposes, first, we need our computers working in a cloud. These computers are called instances. If they are involved in cloud testing, they are also called test instances.

Before launching an instance and getting access to it, you need to perform a series of actions:

  1. Create a key pair. A key pair includes public and private keys. The keys are used in cryptography to encrypt and decrypt data respectively. Right after launching an instance, Amazon EC2 generates, sets and encrypts a password for the Administrator user. For encrypting, the public key is used; for decrypting, the private key is used. You can get the password and decrypt it using the AWS Management Console. This will be expanded on later in this article.
  2. Set security groups. A security group is a set of rules that allow connecting to an instance via certain protocols and ports. The set of security groups associated with the instance is defined when it is launched and cannot be changed later. However, the rules in the security groups may be changed later, and they take effect right after they are changed.

To create a key pair, click Key Pairs in the NETWORKING & SECURITY section of the Navigation panel and click Create Key Pair in the ensuing window:

Amazon Web Services EC2 Key Pairs

Figure 9. The Key Pairs web page

In the Create Key Pair dialog, type the name of a new key pair and click Create:

Creating a new Amazon Web Services EC2 key pair

Figure 10. Creating a new key pair

After the key pair is created, the browser will ask you to save the .pem file containing the private key. In our case, the file’s name is SmartBear_US_East.pem.

Note: A key pair can be used only within the region it was created in. To launch instances in another region, you have to use the key pair created in this region. Also, you should save the corresponding .pem file. Therefore, we have added the US_East suffix to distinguish between future .pem files.

Important: Save the .pem file. It is needed to decrypt the Administrator user’s password.

Now, let’s create a new security group and set the rules for it. To do this, go to the Security Groups page (the Navigation panel > NETWORKING & SECURITY > Security Groups) and click Create Security Group:

Amazon Web Services EC2 Security Groups

Figure 11. The Security Groups web page

In the Create Security Group dialog that appears, specify the name and description of the new group and then click Create:

Creating a new Amazon Web Services EC2 security group

Figure 12. Creating a new security group


Note that Amazon EC2 creates one security group by default, but we need another one with a more appropriate name (Amazon EC2 security groups cannot be renamed).

We will add the following two rules to the SmartBear_Common security group: the ability to connect to the instance’s desktop via the Remote Desktop Protocol and the ability to ping the instance (that is, to rapidly check whether the instance can be accessed or not).

To add the first rule, enable the check box to the left of the SmartBear_Common security group and, in the lower panel on the page, select RDP as a connection method. All columns of the Allowed Connections table will automatically display predefined values for this connection method. We will not change the values. Click Save to save the changes:

Amazon Web Services EC2 RDP connection to the AQA_Common security group

Figure 13. Adding the ability to connect via RDP to the SmartBear_Common security group

Unlike the first rule, the second one is not predefined. To add it, select Custom in the Connection Method column, enter the values to the other fields as it is shown in the picture below and click Save:

AQA_Common security group

Figure 14. The Allowed Connections table for the SmartBear_Common security group

Note that ICMP is a network-level protocol, and it doesn’t use ports like transport-level protocols (TCP and UDP, for example) do. That is why, in the From Port and To Port fields, you need to enter the -1 value (see the picture above).

When launching an instance, you can use several security groups. This makes managing security settings more flexible. For example, every instance we launch will be associated with the SmartBear_Common group, which will allow us to change settings for all of the launched instances at once. Besides the SmartBear_Common group, we could create and use the following groups, for example:

  • SmartBear_TestComplete – for those computers in the cloud where TestComplete will be installed;
  • SmartBear_TestExecute – for those computers in the cloud where TestExecute will be installed.
  • SmartBear_MyProductTests – for computers intended for testing the MyProduct application, and so on.

This approach allows you to easily change security settings for individual groups of instances.

Important: the list of security groups that are associated with the instance is specified when launching the instance. You won’t be able to modify it later.

Step 5 – Launching the Instance

Now, after creating the key pair and the security group, we can launch the instance. To do this, in the AWS Management Console, select INSTANCES > Instances in the Navigation panel and click the Launch Instance button:

Amazon Web Services EC2 Instances

Figure15. My Instances web page

After that, the Request Instances Wizard is displayed. On the CHOOSE AN AMI page of the wizard, you need to choose a computer image (Amazon Machine Image, AMI), which will be used to launch your instance. Let us use a computer with the i386 architecture and the Microsoft Windows Server 2003 R2 operating system. To select an AMI which meets our requirements, switch to the Community AMIs tab, select EBS Images from the Viewing drop-down list and type “2003” in the search bar. The wizard will display the list of machine images that meet the search conditions:

Amazon Web Services EC2 AMI

Figure 16. Selecting an AMI

The Viewing drop-down list specifies the type of images the search is performed within. The list contains two items – EBS Images and Instance-Store Images – which correspond to two AMI types. These types differ in the way Amazon stores the instances’ hard disk data between runs:

Selecting the Amazon Web Services EC2 AMI type

Figure17. Selecting the AMI type by the way it stores data on the instances’ hard disks

We have selected EBS Images as it provides some additional abilities as compared to Instance-Store Images. More specifically, we will further discuss the differences with working with different images later in this article: Step 9 – Stopping and Terminating Instances section.

After the page displays the list of AMIs, click the Select button of the target AMI (in our case, it is ami-ba22c0d3). The next page - INSTANCE DETAILS – will be displayed:

Amazon Web Services EC2 instance launch parameters

Figure 18. Setting instance launch parameters

On this page, you can specify the number of instances to be launched, the availability zone they will be launched in, the instance types (computers’ hardware characteristics) and one of the following launch types:

  • Launch Instances – an ordinary launch for an hourly fee.
  • Request Spot Instances – you specify the cost you are ready to pay for using instances per hour. These instances will be provided when Amazon EC2 has computing resources which are not used and if the cost you offered is higher than that offered by other users.

We will keep all of the default settings provided on this page unchanged. Click Continue and a page is displayed where advance instance options can be set:

Amazon Web Services EC2 advance instance options

Figure 19. Setting advance instance options

Here, we will also leave all of the settings unchanged.

The data you specify in the User Data edit box will be passed to the launched instance as a parameter. You can then check it using applications running on the instance and perform the appropriate operations according to the parameter value. For example, you can pass a command-line for your TestComplete test project and use this command line to launch tests.

Click Continue. The CREATE KEY PAIR page will appear. The page allows you to select an existing key pair, to create and use a new one or not to use any key pair at all. Select the key pair we created earlier (in our case, it is SmartBear_US_East) and click Continue to switch to the next page of the wizard:

Amazon Web Services EC2 key pair

Figure 20. Selecting a key pair

On the next CONFIGURE FIREWALL page, you can specify the security group or groups that will be used for the instance to be launched. Select the SmartBear_Common group and click Continue:

Selecting Amazon Web Services EC2 security groups

Figure 21. Selecting security groups

The next page is the Review page and is the last page of the wizard. It displays the settings you have chosen. You can change the settings if you follow the corresponding links. To launch the instance with the specified settings, click Launch on this page:

Launching Amazon Web Services EC2 instances

Figure 22. Launching instances

Step 6 – Connecting to the Instance

To connect to the instance, we will use the Remote Desktop Connection application, which is a component of most Windows operating systems. To be able to log into the instance’s OS, first, you need to find out the password for the Administrator. For that, go to the My Instances page in the AWS Management Console, select the needed instance on this page (by enabling the check box) and then select Instance actions > Instance Management > Get Windows Admin Password:

Getting the Administrator password for Cloud Testing

Figure 23. Getting the Administrator password

The Retrieve Default Windows Administrator Password dialog will appear. Here, you need to specify the private key that will be used to decrypt the password. To do that, in Notepad or any other text editor, open the .pem file which was used when launching the instance (in our case, it is SmartBear_US_East.pem) and copy the file contents to the clipboard. After that, switch to the dialog window, paste this data into the Private Key field and click Decrypt Password:

Decrypting the Administrator password for Cloud Testing

Figure 24. Decrypting the Administrator password

The dialog will display the decrypted password. Now, you have all of the needed information to connect to the instance:

Amazon Web Services EC2 instance required information

Figure 25. Information required to connect to the instance

Now, you can connect to the instance. Select Instance Actions > Instance Management > Connect in the AWS Management Console. The Console Connect – Remote Desktop Connection dialog is displayed and describes how to establish a connection. Follow the instructions that the dialog provides.

Amazon Web Services EC2 instance connection instructions

Figure 26. Instructions on connecting to the instance

When connecting to an instance, the Remote Desktop Connection window looks like this:

The Remote Desktop Connection window for Cloud Testing

Figure 27. The Remote Desktop Connection window when connecting to the instance

After you have connected to the instance, you can change the Administrator password to one that is more familiar to you or create a new user account.

Now, we are ready to work with the instance and to install TestComplete on it.

Step 7 – Installing TestComplete on the Instance

To install TestComplete on the launched instance, you need to copy the TestComplete installation program to this instance first. To do that, connect to the instance using the Remote Desktop Connection and follow the instructions provided in the OPTION 2: Step-by-Step Instructions section of the Console Connect – Remote Desktop Connection dialog.

In addition to these instructions, you will need to make the remote computer able to use your workstation’s disk drives. To do that, in the Remote Desktop Connection dialog click Options, then More and then choose the desired drives:

Remote Desktop - local computer’s disk drives

Figure 28. Enabling the ability to use the local computer’s disk drives on the remote computer

After that, local disk drives can be used on the remote computer. Now, when connecting, you need to copy the TestComplete installation program from one of your local disks to the instance. The procedure of installing TestComplete to the instance does not differ from installing TestComplete to a regular computer in any way:

Installing TestComplete on the Amazon Web Services EC2 instance for Cloud Testing

Figure 29. Installing TestComplete on the instance

After TestComplete has been installed on the instance, you need to activate the TestComplete license.

Step 8 – Configuring the TestComplete License

Since cloud computers are virtual machines, licensing TestComplete on cloud computers is similar to licensing TestComplete on virtual machines.

SmartBear offers two license types for TestComplete: a Node-Locked license and a Floating User license. A Node-Locked license is bound to the computer where it is activated. A Floating User license supports activating the license on one computer and running TestComplete instances on other computers (the total number of concurrent instances cannot exceed the number allowed by the license).

Node-Locked licenses cannot be activated and used on virtual machines, therefore, they cannot be used on cloud computers. To use TestComplete in cloud environment, you have to use a Floating User license.

An important note is that the Floating User license must be activated on a physical machine (we call it License Manager PC). After you have done that, you can run TestComplete instances on cloud computers. These TestComplete instances will connect to the physical machine and request permission for the run:

TestComplete Licensing in the Cloud

Figure 30. TestComplete Licensing in the Cloud

To be able to get permission for the run, “cloud” TestComplete instances must have network access to the License Manager PC, and their license settings must point to this PC.

So, you can assign a static IP address to your License Manager PC and then configure TestComplete’s licensing subsystem settings on the cloud computer:

  • Log on to the cloud computer, launch the web browser and go to the URL http://localhost:1947 to open the Sentinel HASP Admin Center.
  • In the Administration Options pane of the Admin Control Center, click Configuration.
  • Switch to the Access to Remote License Manager tabbed page.
  • Disable the Broadcast Search for Remote Licenses option.
  • In the Specify Search Criteria box enter the IP address of your License Manager PC.

Configuring the licensing subsystem on cloud computers

Figure 31. Configuring the licensing subsystem on cloud computers

  • Click Submit to apply the changes.

If you have a Node-Locked license, you may want to consider upgrading it to the Floating User License or using TestExecute to run tests on cloud computers. See the Using TestComplete in Virtual Environments topic of TestComplete Licensing Guide.

Step 9 – Stopping and Terminating Instances

Amazon offers Windows OS controlled instances for an hourly fee. So, leaving them running all the time tends to be expensive. As a rule, to avoid overpayment, Amazon EC2 users stop or terminate instances when they do not use them. Depending on the used instance type, data on its hard disk can be either removed or saved. You can select the type of the instance when launching it: you select EBS Images or Instance-Store Images in the Viewing field when selecting an AMI (see Step 5 – Launching Instances).

  • For Instance-Store Images, all data is available as long as the instance is running. You cannot stop an instance of this image type. After terminating the instance, the data will be lost. Using instances of this type is reasonable if you are using an AMI which was prepared beforehand or if your cloud test session involves both configuring an instance and launching tests on it after that.
  • When using EBS Images, you can select whether the disk will be removed or not. If you select that the disk should not be deleted, all of the data will still be stored on the hard disk when you stop the instance and the operating system will be shut down. Thus, to start a test later, you will not need to select and prepare an instance. You can use an already prepared “computer”. Storing data is a paid service, but the fee in this case is less than that for a running instance. For example, as of January, 2014, the cost of an EBS Standard volume was $ 0.10 for 1 Gb per month. Note that after terminating the instance, the respective hard disk will be removed. Thus, using EBS images is one of the simplest ways to store data between test runs in a cloud.

Still, there are other methods to do this:

  • To store data, you can also use the Amazon S3 service. For Amazon S3, the size of stored objects is limited to 5 Gb, however, the number of stored objects is unlimited.
  • You can create an AMI from the current state of the instance. The size of this image is limited to 10 Gb. Such images are also stored using Amazon S3 (this case is an exception to the 5 Gb limitation).
  • You can request an Amazon EBS hard disk, connect it to your instance and copy the needed information to it. Such a disk can be connected to any running instance (but only to one at a time), and it can be stored if the instance it is connected to is terminated. As of January, 2014, the cost of data exchange with such hard disks was $0.10 per 1 mln. I/O requests. The Amazon EBS disk size is limited to 1 Tb.
  • You can create a snapshot for the Amazon EBS disk.

You can use these methods if you are working with Instance-Store Images or if you need to store data after terminating your instances. More details on storing data are available on the web site.

Now, let’s stop the instance we launched earlier. We can do this the same way we selected the EBS Image when launching the instance. Go to the My Instances page – to do that, click INSTANCES > Instances in the Navigation panel. Select the launched instance (by enabling the check box) and then select Instance Actions > Instance Actions > Stop:

Stopping the Amazon Web Services EC2 instance

Figure 32. Stopping the instance

The instance is now stopped. To terminate the instance, you would need to select Instance Actions > Instance Actions > Terminate.

Now, you can either start the instance or terminate it – to do that, select the needed instance (by enabling the check box) and then select Instance Actions > Instance Actions > Start or Instance Actions > Instance Actions > Terminate respectively:

Starting or terminating an Amazon Web Services EC2 instance

Figure 33. Starting or terminating an instance

Problems with Changing IP Addresses

When launching an instance, it gets an IP address which you can use to connect to the instance. When terminating the instance, the connection between the IP address and the instance is deleted. The same applies to stopping the instance. That is, if you stop an EBS image instance and then start it again, all addresses associated with it (internal and external IP addresses, private and public DNS names, etc.) will change. So, all of the tested projects using these addresses will stop working. That is why, before performing the tests, you need to set up these addresses in your test project. To solve the changing addresses problem, you can use a special elastic IP resource provided by Amazon EC2. We will discuss how to do this in later articles of this series.


In this article, we have discussed the most important cloud computing services that the Amazon Company provides for automated testing and explained how to use them to prepare a cloud environment for testing with TestComplete. The following articles in the series will explain how to organize distributed and distributed testing in a cloud and how to resolve problems that may occur during the automated testing process.

Other Articles About Cloud Testing