Author: SmartBear Software
Applies to: TestComplete 7.x, 8.x and 9
This article describes how to implement negative testing in TestComplete and provides an overview of negative testing with some common negative test scenarios.
The article contains the following sections:
There are two main testing strategies in software testing: positive testing and negative testing.
Positive testing determines that your application works as expected. If an error is encountered during positive testing, the test fails.
Negative testing ensures that your application can gracefully handle invalid input or unexpected user behavior. For example, if a user tries to type a letter in a numeric field, the correct behavior in this case would be to display the “Incorrect data type, please enter a number” message. The purpose of negative testing is to detect such situations and prevent applications from crashing. Also, negative testing helps you improve the quality of your application and find its weak points.
The core difference between positive testing and negative testing is that throwing an exception is not an unexpected event in the latter. When you perform negative testing, exceptions are expected – they indicate that the application handles improper user behavior correctly.
It is generally considered a good practice to combine both the positive and the negative testing approaches. This strategy provides higher tested application coverage as compared to using only one of the specified testing methodologies.
Negative testing is aimed at detecting possible application crashes in different situations. Below are several possible examples of such situations:
TestComplete allows you to create negative tests using any possible variants of data input (both valid and invalid). This approach to tests creation will make your application more reliable and stable. Here, you can also specify certain values that will lead to an exception and assign error messages to them - this will warn users of possible exceptions.
Note that the response to a thrown exception depends on the tested application. For example, the application can show a message at the bottom of the window, highlight the field with invalid data, show a pop-up containing the name of the incorrectly populated field or, which is more often used, show an error message window containing a description of the exception’s cause.
To catch these windows, you need to add additional operations to your test.
To track error messages from a keyword test, use the If Object operation. For example, using this operation, you can create a test that will check whether the specified window exists. Insert the operation right after the operation with invalid data input and specify an error message window as the needed object. If the application handles input data correctly, the error message window appears. If the window does not appear, this means that an unexpected exception occurred, and you need to find its cause (see below).
The image below shows an example of using the If Object operation:
To track error messages from a script, add the WaitWindow method to your code. This method delays the script execution until the specified window appears on the screen. Also, you can specify a time period that TestComplete will wait for the window. If the timeout expires and the window does not appear, the WaitWindow method returns an empty stub object. For instance, the code below waits for the “Path not found” message box which the Save File dialog shows when you attempt to save a file to a non-existent drive:
Aliases.notepad.WaitWindow('#32770', 'Save As', 1, 10000);
Below is the list of our recommendations that will simplify your negative testing with TestComplete:
As you can see, negative testing improves the testing coverage of your application. Using the negative and positive testing approaches together allows you to test your applications with any possible input data (both valid and invalid) and can help you make your application more stable and reliable.
We hope, this article contains fresh and useful information that will help you create thorough tests with TestComplete. If you haven’t used TestComplete yet, download and try it for free.
© 2012 SmartBear Software. All rights reserved.