Running a Test Case

When starting a test case, the following actions are taken internally:

1. Context initialization for a test case execution. It is passed to all scripts, Property Expansions, and other objects that will appear during the test case execution allowing you to share resources and various data between them.

   For example, it is used internally to store the HttpState object in the context when the Maintain HTTP Session option has been selected in the Test Case Options dialog.

2. Execution of a Setup Script for a test case.

3. Setting the maximum execution time for a test case. If you have specified the maximum execution time in the Test Case Options dialog and when the time is up, the test case will be terminated and fail if it has not finished in the specified time.

4. Applying the beforeRun event to TestRunListeners which are defined either externally or as Event Handlers at the project level.

5. Performing any WS-RM or AMF-related session or transaction initializations that were configured under the corresponding tabs in the Test Case Options dialog.

Actions 2 and 4 can terminate the test case if any external conditions are not met by calling testRunner.cancel or testRunner.fail.

When running from the UI, an entry marking the start of the test case will be written to the execution log.

Since actual testing takes place inside test steps, running a test case is mostly a case of running the contained test steps.

Execution starts with the first enabled test step (or with the currently selected test step if started with the Run from here test step context menu).

For each test step, the following happens:

1. The beforeTestStep event is triggered to all applicable TestRunListeners.

2. The test step itself is run and its TestStepResult is added to the testRunner.results collection.

3. The test step itself executes other test steps (for example a Run Test Case test step or Script test step calling testRunner.runTestStep), they are recursively executed as described here.

4. The afterTestStep event is triggered to all applicable TestRunListeners.

5. If running in the UI, the corresponding log is updated with TestStepResult saved in (2):

   

6. Depending on the outcome and internal logic of the test step, the following happens:

   • If the test step fails (due to assertion failures, internal errors, and so on) the test case is aborted if the Abort on error option is selected in the Test Case options (default setting).
   • If the test step does not fail and if the Discard Successful Results option is selected in the Test Case Options dialog, the data contained in TestStepResult is discarded from the memory. Selecting the Discard Successful Results option can save a lot of memory when executing long-running tests.

   • If the current test step transfers execution to another test step via the testRunner.gotoTestStepByXXX method, that test step is executed next instead of the one "next in line". The following test steps do this internally:

     1. Conditional GoTo if a condition is met.

     2. DataSource Loop if more data is available.

Once the execution passes the last test step, the test case is finished, and the following events are triggered:

1. All test steps are internally "finished", which allows them to close any resources, write results to files, and so on.

2. The TearDown script of the test case is called, for example, for cleaning up any resources or doing final validations.

3. The TestRunListener.afterRun event is triggered for applicable listeners.

The Status of the test case will be set to FAIL if one of the following conditions is met:

• The test case was aborted due to an internal error.

• The test case was aborted because of a failed test step, or if there were failed test steps and the Fail Test Case on Error option in the Test Case Options dialog is selected.

• A script in any of the triggered events, scripts, or handlers failed the test case with testRunner.fail explicitly.

You can find the final state of the test case in the Run Log.

When a test case is run as a load test, a complete copy of the test case is created for each executing thread. Each thread then runs the test case according to the test case options.

• Any test step result that does not lead to a load test assertion failure is always discarded from the memory if the Discard Successful Results option is set.

• Any test step result that leads to the failure of a load test assertion is never discarded from the memory.

Scripting