Integrating TestComplete and AQtime

Files

Author: SmartBear Software
Applied to: TestComplete 10

Introduction

TestComplete and AQtime are SmartBear’s automated tools for testing and profiling applications. TestComplete is an automated testing tool that allows you to create automated tests for Win32, .NET and WPF applications, web pages and web services. AQtime is used for profiling 32-bit and 64-bit Windows and .NET-connected applications. The difference is that TestComplete logs user actions on the tested application and how the application responds to these actions, while AQtime traces how the application does what it does, that is, which methods and functions are called, which objects and classes are used.

TestComplete is tightly integrated with AQtime. This feature provides you with the ability to integrate AQtime into your automated tests. The process of application profiling in AQtime is similar to application testing in TestComplete: you start an application, perform some operations with it (click buttons, open windows, enter text via keyboard, and others) and then close the application. By using AQtime with TestComplete, you can profile and test your application at the same time. It lets you significantly reduce the amount of time spent for testing and managing the application delivery process.

Using the different AQtime profilers you can trace application performance, check test coverage, search for memory and resource leaks, obtain exception call stacks and perform other tasks that check and improve application quality. Running AQtime’s profiler from TestComplete tracks one or several measures automatically during the test run, generates results upon closing the application and reports them to the TestComplete log. Note that to profile the application, it must be compiled with debug information. To do this, you should set certain compiler options. For more information about this, see AQtime’s documentation.

TestComplete provides two methods for integrating AQtime: using the Profile run mode for tested applications and using special program objects. The first method is only applied for AQtime versions 4.7—8.x, while using special program objects you can work with any version of AQtime from your automated tests.

In this article, we will demonstrate how to integrate AQtime into your automated tests using these methods. For this purpose, we will use AQtime 8 and the Orders sample application that is shipped with TestComplete. The application displays a list of orders and lets users create, modify and delete orders. You can find the application in the following folder:<TestComplete 10 Samples>\Desktop\Orders\C#\bin\Release\Orders.exe. Note that this example works only if you have TestComplete’s Desktop module installed. If you have the Web module installed, you can use a similar Orders.exe application that was created with Silverlight and that is located in the <TestComplete 10 Samples>\Web\Orders\Silverlight\Orders\bin\Release folder.

Creating a New TestComplete Project to Use With AQtime

In the following steps, we will create a new TestComplete project that launches the sample application and performs user actions on it. Later, we will use this automated test for profiling the tested application with AQtime.

To create a new TestComplete project:

  • If TestComplete is not running yet, launch it now. Once it runs, select New | New Project from TestComplete’s File menu. This will call the Create New Project wizard.
  • On the first page of the wizard, specify the name and path of the new project and click Next.
  • On the next page, you can specify the type of the application to be tested by the project. Select Generic Windows Application and click Next.
  • On the next page, click the Add button and use the ensuing dialog to browse for the Orders application executable file. You can find the executable file in the following folder: <TestComplete 10 Samples>\Desktop\Orders\C#\bin\Release\Orders.exe or <TestComplete 10 Samples>\Web\Orders\Silverlight\Orders\bin\Release\Orders.exe.
  • After you choose the desired executable, the wizard will display the application’s name and path in the list of tested applications. Make sure you select the Autorun check box that is displayed in the list next to your application. This will allow TestComplete to automatically launch the application when you start recording.
  • Click Next in the wizard to continue.
  • On the last page, select the VBScript scripting language and click Finish.

TestComplete will create a test project and add the tested application to it.

Now we can create an automated test. Let’s record an automated test script where we will later add the code that works with AQtime. We will use the automated test script to demonstrate how to use special scripting objects provided by TestComplete for integrating with AQtime. You can also create a similar keyword test (keyword-driven test) and set properties of these program objects and call their methods using the Call Object Method or Run Code Snippet operation.

Let’s start recording:

  1. Select Record Script from TestComplete’s Test Engine toolbar, or choose Test | Record | Record Script from TestComplete’s main menu.
  2. After you start the recording, TestComplete will automatically launch the Orders application you have added to the project’s list of tested applications. This happens because we selected the Autorun check box for the application while adding the application to the project.
  3. Choose Orders | New order from the main menu of the Orders application.
  4. Enter the following values into the edit boxes of the Order form:
    Field Value
    Product FamilyAlbum (select it from the drop-down list)
    Quantity 20
    Date 05/06/2005
    Customer Name John Smith Jr
    Street 12, Orange Blvd
    City Grovetown, CA
    State US
    Zip 111155
    Card Select MasterCard.
    Card No 555777555888
    Expiration Date 05/06/2005
  5. Click OK on the Order window to close the Order form and add the new record to the Orders application.
  6. Close the Orders application.
  7. Click No on the Confirmation window to close the application without changes.
  8. Click Stop on the Recording toolbar.
  9. Select File | Save All from TestComplete’s main menu to save the changes.

The recorded automated test looks like this:

Sub Test

  Dim orders

  Dim mainForm

  Dim orderForm

  Dim groupBox

  Dim numericUpDown

  Dim textBox
 

  ' Launches the tested application

  TestedApps.Orders.Run()
 
    
  ' Performs the testing operations
  
  Set orders = Aliases.Orders
  
  Set mainForm = orders.MainForm
  
  Call mainForm.MainMenu.Click("Orders|New order...")
  
  Set orderForm = orders.OrderForm
  
  Set groupBox = orderForm.Group
  
  Call groupBox.ProductNames.ClickItem("FamilyAlbum")
  
  Set numericUpDown = groupBox.Quantity

  Call numericUpDown.UpDownEdit.Drag(27, 2, -36, 3)

  numericUpDown.wValue = 20

  groupBox.Date.wDate = "05/06/2005"

  Set textBox = groupBox.Customer

  Call textBox.Click(55, 9)

  textBox.wText = "John Smith Jr"

  Set textBox = groupBox.Street

  Call textBox.Click(47, 5)

  textBox.wText = "12, Orange Blvd"

  Set textBox = groupBox.City

  Call textBox.Click(79, 3)

  textBox.wText = "Grovetown, CA"

  Set textBox = groupBox.State

  Call textBox.Click(104, 9)

  textBox.wText = "US"

  Call groupBox.Click(404, 220)

  Set textBox = groupBox.Zip

  Call textBox.Click(66, 9)

  textBox.wText = "111155"

  groupBox.MasterCard.ClickButton

  Set textBox = groupBox.CardNo

  Call textBox.Click(33, 14)

  textBox.wText = "555777555888"

  groupBox.ExpDate.wDate = "05/06/2005"

  orderForm.ButtonOK.ClickButton

  mainForm.Close

  orders.dlgConfirmation.btnNo.ClickButton

  End Sub

Using the Profile Run Mode in Automated Tests

Now we will describe how to profile the tested application with AQtime’s Performance profiler using the Profile run mode. We will set this mode and specific settings for the tested application so that when TestComplete launches the tested application, it also launches AQtime to profile this application using the desired profiler during the automated test run. Thus, the TestComplete project will simulate user actions and AQtime will simultaneously profile the application.

You can set up the tested application parameters either in the TestedApps Editor, or in the automated test.

We will set up the tested application parameters in the editor:

  • Right-click the TestedApps node in the Project Explorer panel and select Edit from the context menu. TestComplete will display the Tested Application editor in the Workspace panel.
  • In the editor, select the Orders application we are going to profile during testing.
  • Specify Profile in the Run Mode drop-down list.
  • Click the ellipsis button in the Parameters column to specify the parameters for profiling. TestComplete will obtain the profilers list from AQtime and then display the Parameters dialog, where you can select the profiler, AQtime’s run mode and project:

     Parameters_Dialog

    • In the Profiler box, select Performance Profiler.

      Note that TestComplete obtains the profiler list when you open the dialog for the first time. Then, it works with the stored list. If the set of AQtime profilers was changed (for instance, if you installed new plug-ins), you can update the profiler’s list stored in TestComplete by pressing Get profilers.
    • In the Run mode box, specify the Silent run mode. TestComplete will apply the specified run mode to AQtime after profiling is over. In this mode, AQtime will not display any dialog boxes and messages during the run. If a dialog should be displayed, the default processing will be applied.
    • If you need to change the mode right after profiling has started, enable the Set Run mode after profiling started option.

      TestComplete runs and controls AQtime via COM. By launching it this way, AQtime always functions in the Silent mode, so that we need not select the Set Run mode after profiling started check box.
    • Make sure the Use AQtime project check box is not selected. AQtime will create a temporary project for the tested application and will start profiling using the default settings. This project will be deleted after profiling is over.

      If you have previously created and configured an AQtime project to profile your application, you can use it to test your application in the Profile mode. In this case, AQtime will use settings for profiling from this project file. The results obtained during profiling will also be added to the AQtime project results.

      If you want to use the existence project, enable the Use AQtime project check box and specify the AQtime project in the Project file name box. If the Override project profiler check box is selected, AQtime will ignore the profiler specified in the project file and will use the profiler specified in the Parameters dialog. To modify the project, press Edit project.
  • Press OK to save the settings and close the dialog.

You can also set up the run-mode parameters from the automated test. Use the TestedApp.Params.ProfileParams property to obtain the TestedAppProfileParams object that provides a scripting interface to parameters of the Profile mode. The following code demonstrates how you can set up parameters from an automated test script:

 Set MyApp = TestedApps.Orders
 

' Specifies the run mode parameters

MyApp.Params.ProfileParams.AQtimeVersion = 7

MyApp.Params.ProfileParams.ProfilerName = "Performance Profiler"

MyApp.Params.ProfileParams.RunMode = "Silent"

MyApp.Params.ProfileParams.UseProject = False
 

' Activates the Profile run mode

MyApp.Params.ProfileParams.Activate

After we set the Profile run mode for the application and specify the run parameters, TestComplete will launch the application under AQtime when we call the Run method of the TestedApp object.

Note that when we call the TestedApp.Run method to launch the application in the Profile mode, the method returns the AQtime object whose methods and properties can be used to control the profiling. To obtain this object in the automated test, replace the line of script code that launches the tested application with the following code:

   Set AQtimeObj = TestedApps.Orders.Run()

Then we will use this object to finish profiling, to export profiling results and to close AQtime.

AQtime profiles the tested application until the application is closed, or until you terminate the profiling by using AQtime’s menu items or methods of TestComplete’s AQtime object. The profiling results are generated when the tested application is closed or when you command AQtime to do this. You can command AQtime through AQtime’s UI, or by using actions defined in your AQtime project, or by calling the special methods of the AQtime object from your automated tests.

By default, the profiling results are not imported into TestComplete. That is, after the automated test has finished, you get two result sets: one in TestComplete and another in AQtime. You can export AQtime results to a file and post the file link to the test log: this way the profiling results will be saved along with the automated test results.

To export profiling results and close the tested application, we will add two lines of script code into the end of the routine. The first line calls the WaitAndExportResults method of the AQtime object. This method pauses the script execution until the profiling run is over and then saves both brief and full results to two XML files and posts the file links to the test log. The second line closes AQtime.

Here is the code of the modified automated test:

Sub Test

  Dim orders

  Dim mainForm

  Dim orderForm

  Dim groupBox

  Dim numericUpDown

  Dim textBox
 

  ' Launches the tested application and obtains the AQtime object

  Set AQtimeObj = TestedApps.Orders.Run()
 

  ' Performs the testing operations

  Set orders = Aliases.Orders

  Set mainForm = orders.MainForm

  Call mainForm.MainMenu.Click("Orders|New order...")

  Set orderForm = orders.OrderForm

  Set groupBox = orderForm.Group

  Call groupBox.ProductNames.ClickItem("FamilyAlbum")

  Set numericUpDown = groupBox.Quantity

  Call numericUpDown.UpDownEdit.Drag(27, 2, -36, 3)

  numericUpDown.wValue = 20

  groupBox.Date.wDate = "05/06/2005"

  Set textBox = groupBox.Customer

  Call textBox.Click(55, 9)

  textBox.wText = "John Smith Jr"

  Set textBox = groupBox.Street

  Call textBox.Click(47, 5)

  textBox.wText = "12, Orange Blvd"

  Set textBox = groupBox.City

  Call textBox.Click(79, 3)

  textBox.wText = "Grovetown, CA"

  Set textBox = groupBox.State

  Call textBox.Click(104, 9)

  textBox.wText = "US"

  Call groupBox.Click(404, 220)

  Set textBox = groupBox.Zip

  Call textBox.Click(66, 9)

  textBox.wText = "111155"

  groupBox.MasterCard.ClickButton

  Set textBox = groupBox.CardNo

  Call textBox.Click(33, 14)

  textBox.wText = "555777555888"

  groupBox.ExpDate.wDate = "05/06/2005"

  orderForm.ButtonOK.ClickButton

  mainForm.Close

  orders.dlgConfirmation.btnNo.ClickButton
 

  ' Waits until AQtime finishes profiling and exports results

  Call AQtimeObj.WaitAndExportResults("C:\SummaryResultsFile.xml", "C:\ResultsFile.xml")
 

  ' Closes AQtime

  Call  AQtimeObj.Close
 End Sub 

Now we will run the automated test. TestComplete launches the Orders application under AQtime and performs automated testing. AQtime profiles the application until the profiling is terminated by the WaitAndExportResults method of the AQtime object. Then the profiling results are generated and exported to XML files, AQtime is closed. After the automated test is over, we can see the results in the Test Log:

Test log

As you can see, the automated test run was successful. TestComplete posted messages about the chosen profiler, the start of profiling, simulating user actions and posted links of the profiling results to the test log.

You can open and view profiling results in your browser. The file SummaryResultsFile contains a quick summary of the run. The file ResultsFile contains a full overview of the profiling results and includes all the information about the tested application AQtime obtained.

Using Special Program Objects

Now we will demonstrate how you can automate AQtime using special program objects.

TestComplete includes several objects to work with AQtime:

  • The AQtimeIntegration object provides a scripting interface to the functionality of AQtime 4.7 or later. This object contains methods that let you start AQtime, select the desired profiler, and so forth. Once again, this object can work with AQtime version 4.7 and later.
  • The AQtime4, AQtime5, AQtime6, AQtime7 and AQtime8 objects are used to automate AQtime 4, AQtime 5, AQtime 6, AQtime7 and AQtime8 respectively from your automated tests.
  • The AQtime object lets you automate profiling in AQtime 4, 5, 6, 7 or 8 in a unified manner.

If you have AQtime 4.7 or later, it is recommended that you automate AQtime with the AQtimeIntegration, AQtime, AQtime4, AQtime5, AQtime6 , AQtime7 or AQtime8 objects. They provide the most flexible control over AQtime. We will use these objects in our automated test.

To profile the application using program objects, we don’t need to use the Profile run mode. We will use the Simple run mode instead. To specify this mode to the Orders application, switch to the TestedApps editor and specify Simple in the Run Mode column.

Then we will add the additional routine to the automated test. This routine will launch the tested application in AQtime, select the Performance profiler, start the profiler run, call the Test routine, generate and export profiling results to several files, and close AQtime.

Here is the code of the additional routine:

Sub Integration

  AppPath = TestedApps.Orders.FullFileName
 

  ' Checks AQtime version 

  If AQtimeIntegration.IsSupportedVersionAvailable (8) = true Then
 

    ' Creates a new project

    AQtimeIntegration.AQtime.CreateProjectFromModule(AppPath)
 

    ' Selects the desired profiler

    AQtimeIntegration.AQtime.SelectProfiler("Performance Profiler")
 

    ' Starts profiling

    AQtimeIntegration.AQtime.StartProfiling
 

    ' Runs the Test routine

    Test
 

    ' Gets results and exports it to XML files 

    AQtimeIntegration.AQtime.GetResults(20000)

    Call AQtimeIntegration.AQtime.ExportCurrentResults("C:\SummaryResultsFile.xml", "C:\ResultsFile.xml")
 

    ' Pauses the script execution until the profiling run is over

    AQtimeIntegration.AQtime.WaitForEndOfProfiling
 

    ' Closes AQtime

    AQtimeIntegration.AQtime.Close

  Else

    Call Log.Error("You should install AQtime version 8")

  End If 
 End Sub 

This routine calls the properties and methods of the AQtimeIntegration and AQtime objects. In the beginning of the automated test, we check the AQtime version by the IsSupportedVersionAvailable method of the AQtimeIntegration object. In order for the automated test to work, you should specify your AQtime version number as the MajorVersion parameter of this method.

Note that when we start profiling with this routine, the Orders application to be profiled is launched by AQtime. There is no need to launch it from the Test routine using the TestedApps.Run method. To avoid confusions between several instances of the same application, we will comment out the TestedApps.Orders.Run() line included in the Test routine. Note that we use the initial Test routine we wrote when creating a new TestComplete project.

Below is the complete automated test script:

Sub Integration

  AppPath = TestedApps.Orders.FullFileName
 

  ' Checks AQtime version 

  If AQtimeIntegration.IsSupportedVersionAvailable (8) = true Then
 

    ' Creates a new project

    AQtimeIntegration.AQtime.CreateProjectFromModule(AppPath)
 

    ' Selects the desired profiler

    AQtimeIntegration.AQtime.SelectProfiler("Performance Profiler")
 

    ' Starts profiling

    AQtimeIntegration.AQtime.StartProfiling
 

    ' Runs the Test routine

    Test
 

    ' Gets results and exports it to XML files 

    AQtimeIntegration.AQtime.GetResults(20000)

    Call AQtimeIntegration.AQtime.ExportCurrentResults("C:\SummaryResultsFile.xml", "C:\ResultsFile.xml")
 

    ' Pauses the script execution until the profiling run is over

    AQtimeIntegration.AQtime.WaitForEndOfProfiling
 

    ' Closes AQtime

    AQtimeIntegration.AQtime.Close

  Else

    Call Log.Error("You should install AQtime version 8")

  End If 
 End Sub 
 
 Sub Test 

  Dim orders

  Dim mainForm

  Dim orderForm

  Dim groupBox

  Dim numericUpDown

  Dim textBox
 

  ' Launches the tested application 

  ' TestedApps.Orders.Run()
 

  ' Performs the testing operations 

  Set orders = Aliases.Orders

  Set mainForm = orders.MainForm

  Call mainForm.MainMenu.Click("Orders|New order...")

  Set orderForm = orders.OrderForm

  Set groupBox = orderForm.Group

  Call groupBox.ProductNames.ClickItem("FamilyAlbum")

  Set numericUpDown = groupBox.Quantity

  Call numericUpDown.UpDownEdit.Drag(27, 2, -36, 3)

  numericUpDown.wValue = 20

  groupBox.Date.wDate = "05/06/2005"

  Set textBox = groupBox.Customer

  Call textBox.Click(55, 9)

  textBox.wText = "John Smith Jr"

  Set textBox = groupBox.Street

  Call textBox.Click(47, 5)

  textBox.wText = "12, Orange Blvd"

  Set textBox = groupBox.City

  Call textBox.Click(79, 3)

  textBox.wText = "Grovetown, CA"

  Set textBox = groupBox.State

  Call textBox.Click(104, 9)

  textBox.wText = "US"

  Call groupBox.Click(404, 220)

  Set textBox = groupBox.Zip

  Call textBox.Click(66, 9)

  textBox.wText = "111155"

  groupBox.MasterCard.ClickButton

  Set textBox = groupBox.CardNo

  Call textBox.Click(33, 14)

  textBox.wText = "555777555888"

  groupBox.ExpDate.wDate = "05/06/2005"

  orderForm.ButtonOK.ClickButton

  mainForm.Close

  orders.dlgConfirmation.btnNo.ClickButton
 End Sub 

After running this automated test, you can see that the results are the same as the results of the automated test that used the Profile run mode.

Conclusion

In this article, we have demonstrated TestComplete’s integration with AQtime. This functionality allows you to trace a tested application execution during the automated test run. In other words, while your TestComplete project is simulating user actions or performing unit tests, AQtime will check for memory leaks, performance bottlenecks, or perform coverage analysis. By using these tools together, you can automate the testing and profiling process and save time in your daily development activities. If you are new to TestComplete or AQtime, download a free trial of try them today.

More Information