10. Analyze Test Results

Applies to TestComplete 15.31, last modified on March 17, 2022

TestComplete keeps a complete log of all operations performed during testing. The Project Explorer keeps the test results under the Project Suite Logs > SmartStore Log node. This is where you are viewing the test history of the project and project suite. Each node corresponds to a test run. An icon to the left of the node shows whether the corresponding test run passed successfully:

TestComplete adds nodes for the last results automatically after the test execution is over. It does not display the results when the test is running (you can view intermediate results if you pause the test execution).

Because we have run only one test so far, we have only one log node in the Project Explorer. By default, TestComplete automatically opens that log. To open a test log manually, double-click the needed log node in the Project Explorer panel.

Note: By default, TestComplete stores all test results in log files. The number of the log files will grow with every test run, and this will result in the memory consumption increase. To reduce memory usage, you can delete files from the log manually or limit the number of log files to be kept.

In our example, the log is as follows:

Getting Started with TestComplete (Web): The sample test log

The log window shows the results of one test run at a time. On the left side of the window, there is a tree-like structure of the executed tests; you can select each of them to view their results. We have run only one test, so in our case this tree only contains one node. The node icon indicates whether the test passed successfully or failed.

The test log contains error, warning, informative, and other types of messages. The icon on the left indicates the message type. With the check boxes at the top of the message list, you can hide or view messages by type.

For each message, the log also shows the time that each action was performed. You can see it in the Time column.

TestComplete may post additional text and images along with the message. To view them, select the desired message in the log and look in the Details and Picture panes that are below the message list.

The Picture panel shows the images of the expected and the actual application state when executing the selected test command: the “Expected” image was captured for the command during the test recording, the “Actual” image was captured during the test run. You can click View Comparison Results to compare the images and easily see the difference. This simplifies the search for errors that may occur in your test. For more information, see the topics of the Test Visualizer section.

To view a test operation that posted a message to the log, double-click the desired message in the log. TestComplete will open the keyword test in the editor and highlight the appropriate operation. For instance, if you double-click the “The button was clicked with the left mouse button.” message in the log, TestComplete will highlight the keyword test operation that performed this action:

Getting Started with TestComplete (Web): Switching to the command

For detailed information on the test log panels, on posting messages to the log and on working with the results, see the Test Results section.

Note: The log that we described is typical for TestComplete keyword tests and scripts. Tests of other types may form a log of a different structure. For detailed information about these logs, see the description of the appropriate project item or simply click within the log page and press F1.
Resolving Errors

Your test may fail. There can be several possible reasons for this. For example:

  • Developers have changed the application behavior.

  • Recognition criteria of pages and web elements have changed.

  • A third-party application may have overlapped your web browser and so on.

One of the most typical reasons that novice users face is the difference in the application state during the test creation and playback. To avoid this problem, make sure that the initial conditions of the test run correspond to those you had when creating the test. For instance:

  • If your web browser had been running before you recorded a test, it must also be running before you run the test.

  • If your tested web page was opened in your web browser when you recorded your test, it should also be opened when you run the test.

  • If you edited any data in the application, and then saved it, you need to revert the changes.

If the tested application has changed and the recognition criteria that your test uses to find tested objects are no longer valid, TestComplete will try to detect the changes to find the missing objects. If your test log reports the “Object <Object_Name> was replaced with a similar object” warning, this means that TestComplete was not able to find the needed tested object during the test run and used a similar object instead.

Resolve the warning by applying suggested fixes

Click the image to enlarge it.

Do not ignore this warning. Examine it to learn about the missing and the replacement objects and update the missing object’s recognition criteria to resolve the issue.

For information on searching for the cause of errors and resolving typical problems, see Handling Playback Errors.

Prev     Next

See Also

About Tested Web Page
Test Results
Handling Playback Errors

Highlight search results