The difference between testing and simply exploring is that in the case of testing, output is compared against an expected standard. In other words, the basic test sequence is (see Planning Tests) --
- Define expected output.
- Feed corresponding input.
- Gather output.
- Compare to expected output.
- Call for attention if the comparison fails.
With TestComplete you can create tests of different types: scripts, low-level procedures and so on. However, the two primary test sources are script routines and keyword tests. Both test types provide support for creating the sequence of test steps described above. Both can be recorded or created from scratch with special editors. Keyword tests are simpler and easier to create, because the user does not need to know any particular scripting statements. Scripts, however, provide more powerful functionality. For more information on keyword tests and scripts, see Keyword Tests and Script Tests sections.
In a professional development context, almost all tests should succeed. The point of the test is to check if perhaps, unfortunately, something does not behave as expected. The first task of an automated testing tool is to allow you to run a great many tests while paying attention only to the few that fail.
Thus, the heart of TestComplete is the test log. This records messages from tests. All messages are graded. The log lets you filter them by grade, so that in the end you only pay attention to those that deserve attention.
Software testing is a discipline with forty years of lessons behind it. TestComplete is a leading-edge tool, but it is meant to build on that discipline, not to make it obsolete. Traditionally, testing is divided by the object of the test. Unit testing (or module testing) tests the interface between units (modules, functions, libraries, classes, etc.) and the rest of the code, as well as the user or the system (where this applies). Functional (or user interface) testing tests the interface between the application on one side, and the rest of the system and users on the other side. See the references for explanations of how these different testing stages are run, and how TestComplete can help.
Unit testing or Functional testing is often done poorly, incompletely or too late. There are reasons for this, and the point of TestComplete is to make these reasons void by ensuring that Unit testing and Functional testing are productive practices for you. Now, if you actually carry out the testing thoroughly and in good time, the natural way to bring all this together is regression testing.
Regression testing is based on the idea that tests repeat. To “regression test” something means to run the same test you ran before the last modification, and check that you get the same result you got and accepted then. Thus, each test builds support for the next run of tests. The Stores function in TestComplete is meant to provide for this. And the test log is meant to accumulate and track results throughout the life of a development project, and to allow you to filter out those that are “as expected”.
It was only natural that TestComplete should ensure superior support for regression testing. But it also stands out in another way: its unmatched support for white-box testing.
When a user approaches an application, what they see is defined as the application “black box”. A test tool that merely records input and gathers output does no better. It too stands outside the application as if outside a black box. For TestComplete, under some conditions, the application becomes much more transparent. The “external” testing can reach much further into the application. In fact, in many cases, it can reach further than any other tool.
Nonetheless, white-box testing is not a testing method, it is a testing capacity. The White-Box Testing topic shows some things that can be done with TestComplete and not with other tools -- and especially not with black-box testing. But these things do not constitute a method that will substitute for the recognized means of testing at a professional level: unit, functional (or UI) and regression testing.
To the contrary, the TestComplete superior white-box capability is provided to “make the tough parts easier” in traditional testing. It is equally provided to give testers an easy way to orient themselves and with a quick means of finding where they are going. The TestComplete white-box capability is there not to replace conventional testing but to make it more widely adopted, less confining and worrisome, and more productive earlier.