Author: SmartBear Software
Applies to: TestComplete 9.0
In addition to TestComplete’s support for automation of unit, regression, distributed and other automated testing types, it offers support for traditional record-and-playback-style GUI and functional testing. While certainly not a comprehensive method of automated testing on its own, recordable automated tests represent an excellent starting place for creating more robust and complete test projects. One of the complaints leveled against traditional playback-type automated testing is that it is inflexible and that the automated test cases are not easily extended. To answer this argument, the Data-Driven Testing (DDT) methodology was born. Although this technique does not originate with TestComplete, it is a fairly straightforward concept and is easily implemented within TestComplete’s automated tests.
This article contains the following information on how data-driven testing is implemented in TestComplete:
The simplest explanation of data-driven testing is this: data that is external to your functional tests is loaded and used to extend your automated test cases. One of the best examples is that of a customer order form. To use data-driven testing in this scenario, you might record a single automated test, entering values into the various fields. After you have recorded the automated test, it only contains values that you entered during the recording. Most likely, data you have specified does not cause errors in the application, but other data may cause them. To be sure that the application works as expected you can alter the test to accept variables, entering those variables into the data fields. Thus, data-driven testing allows you to run this test each time you want to add an order record, passing in a new set of data each time. Our recent webinar, Become a Data-Driven Testing Ninja, covers this topic in great detail.
It is convenient to keep data for automated tests in special storages that support sequential access to a set of data, for example, Excel sheets, database tables, arrays, and so on. Often data is stored either in a text file and are separated by commas or in Excel files and are presented as a table. This lets you modify them easily. If you need to add more data, you simply modify the file either in any text editor or in Microsoft Excel (in case of hard-coded values, you should modify both data and code).
Each row in the file holds values to be assigned or checked. One row may contain values for one automated test. However, you are free to use several rows for one automated test. In this case, you may need to add a field that will specify which values are input, and which are verifying values.
Note that you can create and populate such storages manually, or you can command TestComplete to do this automatically. For this purpose, TestComplete includes the Data Generator that lets you generate test data of various types and store it either to a TestComplete Table variable or to an Excel file. After you specify all the desired parameters for the generated data, TestComplete will automatically generate it according to the defined parameter values and save these values to the variable or file. This approach decreases the time spent on preparing data for data-driven tests. For more information, see the Using Data Generators section in TestComplete’s help.
During the data-driven testing, you can also execute expressions specified in cells of the processed storage (for example, your storage can contain the value of 5+5). To execute this expression, you can either use the Eval function (in VBScript and JScript) or the Evaluate function (in DelphiScript). To do that, you need to obtain the expression first and then pass it as a parameter of the Eval or Evaluate function. After you run your test, it will automatically retrieve the needed expression from the data storage, execute it and return its result (in the example described above, the test will return 10).
Data-driven testing let you create automated test projects that can be infinitely extended by simply adding new lines of text to a text file or a spreadsheet. For example, data is read from an external source and fed line-by-line into the functional test until there is no more external data. In this way, new automated test cases that are added to the external data simply extend the loop for each new line of data.
When the storage contains both input and verifying data, a data-driven test includes the following operations performed in a loop:
Later, we will illustrate how to perform these steps with an example.
TestComplete offers you two possible ways that can be used to create data-driven tests:
Each DB Table store links to database tables and queries and provide serial access to their records.
The DB Table variables are usually used by the Data-Driven Loop keyword-driven test operation. It allows you to iterate through the table rows. When you are adding this operation to the test, you specify the DB Table variable that will access the specified database table and specify the records that will be processed by the operation. This approach is very accessible and lets you create data-driven tests very quickly. See below for more information.
Table variables. These variables let you store test data in two-dimensional arrays (tables) and then use this data in your data-driven tests. The array elements have the Variant type and can store any value: a number, a string, a date and so on. You can also specify the names of array columns and then address the values using these names.
One more approach to creating data-driven tests, is using the methods of the DDT program object. Via this object you can create scripts for data-driven tests. It can be very useful, if you want to create a custom test, go through the database records in a defined direction (from the last record to the first, for instance) and it is also used for backward compatibility.
You can use these approaches to iterate through records of the desired table, retrieve them and use in your automated tests. In this article we will exemplify creating data-driven tests via the DB Table variables.
Let’s see how to perform data-driven testing with TestComplete. We will create a simple data-driven test illustrating how to add new records from the external source to the sample application using DB Table variables. For this purpose, we will use the Orders sample application that is shipped with TestComplete. The application displays a list of orders and lets users create, modify and delete orders. You can find the application in the following folder: <TestComplete Samples>\Open Applications\C#\bin\Release\Orders.exe.
We will use the Excel sheet as data storage. You can find it in the following folder: <TestComplete Samples>\Data-Driven Testing\TestBook.xlsx. Each row of this table contains the values for one order, whose input will be simulated. Note that the data contains not only input values, but also the verifying values. You can see them in the last three columns of the table. One of these values will be used as a baseline copy for comparison with a calculated value.
The purpose of our data-drive test is to read data from the data storage, simulate data input into the form’s edit boxes and check whether the application calculates values for the Total edit box correctly.
At first, we will create a new TestComplete project. To do this:
In the following steps, we will record a keyword test (keyword-driven test). The test should perform actions like opening the tested application, opening the Order form, populating the form, creating checkpoints, closing the form and closing the application.
To record the test:
The recorded test looks like this:
Now we will modify the recorded data-driven test to make it use data stored in an external Excel file. The test will iterate through the rows of the specified file, each time storing one row to the DB Table variable and then using stored data to populate the Order form’s fields.
You can create the DB Table variable on the Variables edit page of the KeywordTest editor manually or add this variable to the test automatically while creating a data loop.
We will create a data-driven loop including the operations between the RunTestedApp operation and the operation that closes the Orders application. To do this:
Creating a data-driven loop
TestComplete then adds the Data-Driven Loop operation to the keyword-driven test, and all the selected operations will be moved as its child operations.
Also, the selected operations now have the new parameter values that are specified through the DBTable1 variable. You can see this in the picture below:
The modified test
Note that in the test described above the Master Card option button is always selected. However, you may want your test to select the card type against the Credit Card column of the TestBook.xlsx file. In this case you will have to specify the conditions for this manually. For example, you can do it using the If … Then operation. To demonstrate this approach, we’ll change the keyword-driven test in the following way: delete the MasterCard.ClickButton operation and insert the segment below:
After modifying, your test becomes more reliable and flexible. So you can use this approach to improve your test’s reliability.
Now we will run the data-driven test. As you can see, TestComplete opens the tested application and adds new records until there is no more data in the TestBook sheet. After the data-driven test is over, you can find the test results in the Test Log.
As you can see, the data-driven test run was successful.
If the output value had not matched the appropriate verifying value, the "The property value does not equal the template value" error messages would have been posted into the log. This indicates that the application works incorrectly. In this case, the values should be revised. You can include any information you need in error messages. For instance, you can specify the values of all input parameters that were used when the error occurred.
This article serves as an introduction to data-driven testing and TestComplete’s support for it. We hope this information will help you create your data-driven tests and make them more flexible and powerful. If you are interested in trying TestComplete for free, download it here.
© 2012 SmartBear Software. All rights reserved.