The Execution Plan editor shows the project’s test items in the order they will be run. A test item can be:
- A keyword test
- A script routine
- A low-level procedure
- A network suite, or its job or task
- A BDD scenario or several scenarios
- A unit test or a Selenium test
- A ReadyAPI functional or security test
- A specifying one or several tests (keyword tests, script routines, and BDD scenarios) to run
You can organize test items into a plain list, grouped list or a tree-like structure (where child items are run after the parent item has finished running). You can exclude test items from or include them in the test run using the check box next to the test item name.
The following table contains descriptions of test item properties.
Test item properties are not applied to tests that are not launched as test items (for example, they are launched from the Project Explorer or are called from another test).
Name |
The test item name. |
Test case |
Marks the test item as a test case. This item will be included in the Summary report.
Notes:
-
This column is not applicable to BDD feature files. You can mark individual BDD scenarios as test cases though.
Scenarios and features are always added to the Summary report regardless of the column value.
-
This column value is ignored if you run tests by tags. Tests that match a tag or a tag expression are always added to the Summary report.
-
Though you can mark network suites, their jobs, and tasks as test cases, the results of the actions executed on remote computers will not affect the test case results.
-
Regardless of this column value, TestComplete will generate a detailed test log for the test item anyway.
|
Execution entity |
The project item to be run. It can be:
- a keyword test,
- a script routine,
- a BDD scenario or several scenarios,
- a low-level procedure,
- a network suite, or its job or task
- a unit test or Selenium test,
- a ReadyAPI functional or security test,
- a tag,
- a tag expression.
To choose a test, click the ellipsis button and use the resulting Select Test dialog.
|
Count |
The number of times the test will be repeated. Default is 1.
|
Timeout |
The maximum run time (in minutes) for the test item. If the run time exceeds the timeout, TestComplete posts an error to the test log and generates the OnTimeout event. You can create a handler for this event to stop the current test item and proceed to the next one. See Stopping Tests on Timeout.
Default is 0, which means no timeout.
|
Parameters |
If a test item is a script routine or a keyword test that takes parameters, this column specifies the parameter values. If the test item is a tag expression, it specifies the expression.
Click the ellipsis button and specify the needed values or expression in the resulting Test Parameters dialog.
You can use either hard-coded parameter values, or project variables.
|
On error |
Specifies what TestComplete will do if an error occurs during the test item run:
-
Use project 'On error' setting - Default. The behavior will depend on the value of the Error handling > On error property of your project. See Project Properties - Playback Options.
-
Continue running - TestComplete will ignore the error and continue the test run.
-
Stop project - TestComplete will stop the entire project run.
-
Stop current item - TestComplete will stop the current test run and move to the next test.
If the test item has child items, they will not be run.
If the test item corresponds to a tag or a tag expression, it will affect the current matching test only. That is, TestComplete will stop the current matching test and will move to the next matching test.
If the test item corresponds to a scenario outline, it will affect the current example only. TestComplete will stop the current example and proceed with the next example.
-
Stop and rerun current item - TestComplete will stop the current item and rerun it several times. The number of retries is controlled via the project Rerun failed test items property.
For more information on using the On error property, see About Controlling Test Execution Flow.
Note: This property does not apply to errors that occur on remote computers during distributed testing.
|
On exception |
Specifies what TestComplete will do if an unhandled script exception occurs during the test item run. Some examples of these exceptions are: an invalid function name, an invalid parameter value, and so on. This property is effective only if the Show a notification when an unhandled script exception occurs option is disabled in your project.
-
Continue running - TestComplete will ignore the exception and continue the test run.
-
Stop project - TestComplete will stop the entire project run.
-
Stop current item - TestComplete will stop the current test item and its child items and move to the next test item.
For more information on using this option, see About Controlling Test Execution Flow.
Note: |
This option does not apply to exceptions that occur on remote computers during distributed testing. |
|
Link to external test case |
Specifies an external test case to which your test item is linked. It can be a test case stored in:
-
Zephyr for Jira
-
Azure DevOps
You can click the ellipsis button and select the needed test case in the resulting wizard. If your TestComplete project is not bound to any external project yet, select either Zephyr for Jira or Azure DevOps, configure the connection to the needed project in the selected system, and then select the needed test case.
Linking TestComplete test items to Azure DevOps test cases makes sense only if your current TestComplete project is stored in your team project’s repository. Otherwise, those Azure DevOps test cases will not be able to access your TestComplete tests.
If you use Zephyr for Jira, you can also type the identifier of an external test manually. To bind your test item to a Zephyr test case, use the following format:
zfj://“your_zephyr_test_case_key” Notes:
-
This column is not applicable to test items that run an entire BDD feature file. It is only applicable to items that run individual BDD scenarios.
-
This column is not applicable to test items that run tests specified by their tags.
-
This column is not applicable to test items that are not marked as test cases. Only test cases can be bound to external tests.
|
Description |
An arbitrary description of the test item.
|
If a column is hidden, to make it visible, right-click anywhere in the editor, click Field Chooser, and then drag the needed column from the resulting list to the editor.
You can customize the column layout. For example, you can reorder or hide some columns. For more information, see Arranging Columns, Lines and Panels.
Add a new test item
Add a new test itemAdd a new test itemTo add a new top-level test item, do any of the following:
-
Click New Test Item on the toolbar.
-
Right-click somewhere within the Execution Plan editor and choose New Item from the context menu.
-
Drag the desired test (for instance, a keyword test, a unit test and so on) from the Project Explorer panel to the Execution Plan editor.
Notes:
-
If you drag and drop a script unit, TestComplete will add test items corresponding to all of the script routines contained in that unit.
-
TestComplete will name the created test items based on the source script routine and keyword test names.
-
Drag a script routine from the Code Explorer panel to the Execution Plan editor.
Note: |
By default, all added test items are marked as test cases. |
To add a child test item to an existing test item, do any of the following:
-
Select the parent test item or group and click New Child Test Item on the toolbar.
-
Right-click the parent test or group and choose New Child Test Item from the context menu.
-
Drag the desired test from the Project Explorer or Code Explorer and drop it in the parent test item.
Notes:
- By default, the Code Explorer panel is hidden from TestComplete. To make the panel visible, select View > Select Panel from the TestComplete main menu, and then select Code Explorer in the resulting Select Panel dialog.
- By default, TestComplete considers child test items as test steps rather than test cases. The added child test items are not marked as test cases. Test items that are not marked as test cases will not be included in the Summary report.
Once you have added a new test item, you can modify its properties and relocate it in the list.
Add a new group
Add a new groupAdd a new groupTo add a top-level group:
– or –
To add a subgroup:
– or –
Modify test item properties
Modify test item propertiesModify test item propertiesSelect the cell that contains the data you want to change, and press F2 or click twice (not double-click).
To change the test to be executed:
-
Click the ellipsis button of the Execution entity column.
-
In the resulting Select Test dialog, select the needed test or tag. To run tests that match several tags, select [Tag Expression].
To modify test item parameters, click the ellipsis button within the Parameters cell and use the resulting Test Parameters dialog.
To modify a tag expression:
-
Click the ellipsis button in the Parameters column of the [Tag Expression] test item.
-
In the resulting Test Parameters dialog, specify the needed tag expression.
Tag expression syntax
Tag expression syntaxTag expression syntax
-
Use @
before tag names, that is, use @tag1
rather than tag1
.
-
To combine tags, use the and
, or
and not
operations, for example:
@tag1 or @tag2
@tag1 and not @tag3
not @tag5
These operations don’t depend on the scripting language you use in your TestComplete project. That is, for example, in JavaScript projects you should use and
, or
and not
rather than &&
, ||
and !
—
@tag1 and not @tag2 ← Correct!
@tag1 && !@tag1 ← Incorrect!
-
If needed, you can use braces to group tags, for example:
@tag1 and (@tag2 or @tag3)
(@tag1 or not @tag2) and (not @tag3 or @tag4)
Rename a group
Rename a groupRename a group
-
Select the group and press F2, or click the group twice (not double click).
-
Enter the new name for the group and press Enter to apply it.
Copy an existing test item or group
Copy an existing test item or groupCopy an existing test item or group
-
Select the source test item or group and click Copy on the Standard toolbar or press Ctrl+C.
-
Click Paste on the Standard toolbar or press Ctrl+V.
– or –
-
Right-click the source test item or group and select Copy from the context menu.
-
Right-click somewhere within the Execution Plan editor and choose Paste from the context menu.
Note: |
Currently, you can copy test items and groups within the same project. You cannot copy an item from one project to another. |
Include or exclude test items from a test run
Include or exclude test items from a test runInclude or exclude test items from a test runTestComplete allows you to easily specify which test items should participate in testing and which should be temporarily excluded.
To enable or disable a single test item or group:
To enable or disable several test items at once:
-
Select the desired test items in the panel using Ctrl- and Shift-clicks.
-
Click the Enable Selected or Disable Selected button on the toolbar.
– or –
Right-click the selection and choose Enable\Disable > Enable Selected or Enable\Disable > Disable Selected from the context menu.
To enable or disable all of the test items:
– or –
To toggle the enabled property of all test items:
– or –
Mark a test item as a test case
Mark a test item as a test caseMark a test item as a test caseIn TestComplete, you can mark test items in your project as test cases to collect information on their results to a Summary report.
To mark a test item as a test case, select or clear the Test Case check box of the test item.
To mark (or unmark) several test items at once:
-
Select the needed test items. You can use Ctrl- and Shift-clicks for multiselection.
-
Right-click the selection, click Mark/Unmark Test Case, and then click Mark Selected or Unmark Selected.
To mark (or unmark) all test items as test cases:
-
Right-click anywhere within the Execution Plan editor.
-
Click Mark/Unmark Test case and then click Mark All or Unmark All.
To toggle all test cases and test items:
-
Right-click anywhere within the Execution Plan editor.
-
Click Mark/Unmark Test case and then click Toggle All.
Link a test item to a test case in an external test management system
Link a test item to a test case in an external test management systemLink a test item to a test case in an external test management system
-
Mark the needed test item as a test case.
-
Click in the Link to external test case column.
-
If your TestComplete project is not bound to an external test management system yet, select either Zephyr for Jira or Azure DevOps and connect to it in the resulting wizard.
Before linking your TestComplete project to an Azure DevOps team project, make sure to add the TestComplete project to your team project’s repository.
-
Select an external test case to which you want to bind your TestComplete test case.
If you are connecting to Zephyr for Jira:
In the Link to external test case wizard, select the external test case you want to bind to your TestComplete test case.
– or –
Type the identifier of the external test case manually. Use the following format:
zfj://<your_zephyr_test_case_key>
If you are connecting to Azure DevOps:
On the Edit test case associations page of the wizard, click the ellipsis button and select the needed test case in the resulting Choose Test Case dialog. Then click Add association.
If needed, repeat the step to link the TestComplete test case with several test cases of the test plan.
Remove a link between a test item and an external test case
Remove a link between a test item and an external test caseRemove a link between a test item and an external test case
-
Select a test item linked to an external test case.
-
If the item is bound to a Zephyr test case, clear the Link to external test case column.
If the item is bound to an Azure DevOps test case:
To remove a single link:
-
Click the ellipsis button in the Link to external test case column.
-
In the resulting Associate with Azure DevOps / TFS Test Cases wizard, in the Existing associations table, select the needed test case.
-
Click Remove and then click Finish to apply the changes.
To remove all the links at once and unbind your TestComplete project from the Azure DevOps / TFS team project:
-
Switch to the Properties page of your project.
-
Select Azure DevOps in the tree on the left.
-
Click Unbind from Azure DevOps / TFS Project.
Run a test item or group
Run a test item or groupRun a test item or groupWhen you execute your test project, TestComplete automatically runs all the enabled test items that can be managed in the Execution Plan editor. Besides, you can run the desired test items manually in one of the following ways:
– or –
TestComplete will run the selected test item or group along with all child items.
Note: |
Only enabled test items can be run. |
You can run test items not only from TestComplete UI. For information about all possible ways of running test items (from the command line, from Jenkins and so one), see information in the Running Test Items section.
Change the run order of test items
Change the run order of test itemsChange the run order of test items
– or –
Navigate to the test source
Navigate to the test sourceNavigate to the test source
– or –
TestComplete will open the corresponding editor in the Workspace panel. If the test item relates to a certain scripting routine, TestComplete will navigate to the beginning of the routine in the Code Editor.
Expand or collapse test items
Delete a test item or group
Save or cancel changes
Save or cancel changesSave or cancel changesTo save changes made to the project’s test items, select File > Save from the TestComplete main menu.
To cancel any changes, close the editor and answer No when TestComplete asks you whether to save the changes.
This section describes how you can run test items other than manually from the Execution Plan editor (see above).
-
From the command line.
By default, when you run a project from the command line, the project executes all enabled test items specified in its Execution Plan editor. In addition, you can use the /test (/t) command-line argument to explicitly specify test items you want to run.
Note: |
TestComplete will run only the enabled test items, it will not run the disabled items. |
For example, the following command runs TestComplete, loads the specified project suite (MySuite.pjs), and commands TestComplete to run the ProjectTestItem1 test item of the MyProj project:
"C:\My Projects\MySuite.pjs" /r /p:MyProj /t:ProjectTestItem1
-
As part of an automated build and test process managed by Jenkins.
You can add the TestComplete Test build step to your build process in Jenkins and configure it to run the needed test item. See Integration With Jenkins.
-
From an external application via COM.
Use the Integration.RunProjectTestItem
method to run a test item of a project. To learn more about running tests via COM, see Working With TestComplete via COM.