Run BDD Tests

Applies to TestComplete 14.30, last modified on November 21, 2019

In TestComplete, you can run BDD tests in several ways.

Run from the Project Explorer

In the Project Explorer panel, right-click a Feature item and select one of the Run items from the context menu:

Run BDD tests from Project Explorer

Click the image to enlarge it.

You can run either all the scenarios in the feature file, or only one scenario.

If the Project Explorer is hidden, select View > Project Explorer to show it.

Run from the Gherkin editor

You can run features, scenarios and individual test steps directly from the Gherkin editor:

  • In the Gherkin editor, right-click the line of Feature, Scenario or a test step (Given, When, Then, And, But) and select Run This Feature File, Run This Scenario, or Run This Test Step from the context menu:

    Run BDD tests from Gherkin editor

    TestComplete will run the feature, scenario or test step.

Test steps typically depend on earlier test steps and on the current state of the tested application, so if you run test steps, they might fail. The same can happen to scenarios as well if they depend on earlier scenarios. So, we recommend running BDD tests from the Gherkin editor for debugging purposes only. To run these tests on a regular basis, run them in other ways.

Run from test items

Test items help you organize the tests of your project into a test set and run them in the desired order. You can include BDD tests in test items to make them part of your project test run and execute them on a regular basis.

  1. Open the Test Items page of your project editor. You can do this in different ways. For example, right-click a project in the Project Explorer and select Edit > Test Items from the context menu:

    Open the Test Items page of the project editor

    Click the image to enlarge it.

  2. Add a new test item (this can be a top-level item or a child test item).

  3. Click the ellipsis button in the Test cell. In the subsequent dialog, expand the Scenarios node on the left and select your feature file. On the right, select a scenario for the run or select [All scenarios] to run the entire feature file. Click OK to complete:

    Running a BDD test as a test item

    Click the image to enlarge it.

You can quickly append a scenario to the test item list directly from the Gherkin editor: simply right-click the scenario and select Add Scenario to Test Items from the context menu:

'Add Scenarios to Test Items' menu item

Run from scripts

TestComplete provides a scripting interface to features and scenarios defined in your test project. You can access them through the Features object. The script objects that match the Feature and Scenario elements have the Run() method that lets you run them from scripts:

JavaScript, JScript
Python
VBScript
DelphiScript
C++Script, C#Script

JavaScript, JScript

// Run the first feature defined in the project
Features.Items(0).Run();

// Run the first scenario of the first feature
Features.Items(0).Scenarios.Item(0).Run();

Python

# Run the first feature defined in the project
Features.Items[0].Run()

# Run the first scenario of the first feature
Features.Items[0].Scenarios.Item[0].Run()

VBScript

' Run the first feature defined in the project
Features.Items(0).Run

' Run the first scenario of the first feature
Features.Items(0).Scenarios.Item(0).Run

DelphiScript

  // DelphiScript does not support BDD tests.

C++Script, C#Script

// Run the first feature defined in the project
Features["Items"](0)["Run"]();

// Run the first scenario of the first feature
Features["Items"](0)["Scenarios"]["Item"](0)["Run"]();

You may want to run BDD tests from scripts when you need to do some additional checks before running a scenario, or when you need to implement a specific execution flow.

For more examples of running features and scenarios from script code, see Scripting.

Run from keyword tests

Run from command line

A good approach to automating test runs is execution of BDD tests from the command line. For this purpose, you use a command line like this one:

TestComplete.exe project-suite-file /project:project-name /test:"Scenarios|Feature1|My scenario description" /run

  • project-suite-file is the fully-qualified name of your project suite file.

  • /project:project-name (or /p:project name) is the name of your test project as it is displayed in the Project Explorer.

  • /test:"Scenarios|..." (or /t:"Scenarios|...") specifies the name of the feature or scenario to run. Scenarios is a required part of the feature or scenario reference.

    To run feature files and scenarios by tag, use /t:"@tag-name" or /tags:"tag-expression".

  • /run (or /r) is used to command TestComplete to run tests after it loads the specified project suite.

Note: TestExecute uses the same command-line argument for running BDD tests. If you use TestExecute to run automated tests, simply replace the TestComplete path with the TestExecute one.

Examples

  • Run a feature

    Specify the Scenarios|feature-name in the /test parameter. Type the feature-name as it is specified in the Project Explorer panel:

    TestComplete.exe TestComplete.exe "C:\Work\My Projects\MySuite.pjs" "/p:My Project" /test:"Scenarios|Feature1" /r

  • Run a scenario

    Specify the feature file name and the first line of the scenario description in the /test parameter:

    TestComplete.exe TestComplete.exe "C:\Work\My Projects\MySuite.pjs" "/p:My Project" /test:"Scenarios|Feature1|The 1st line of scenario description" /r

  • Run scenarios by tag

    Specify @Tag1 in the /test parameter:

    TestComplete.exe TestComplete.exe "C:\Work\My Projects\MySuite.pjs" "/p:My Project" /test:"@Tag1" /r

  • Run scenarios by tag expression

    Specify the @Tag1 and @Tag2 expression in the /tags parameter:

    TestComplete.exe TestComplete.exe "C:\Work\My Projects\MySuite.pjs" "/p:My Project" /tags:"@Tag1 and @Tag2" /r

For complete information on the supported command-line arguments, see TestComplete Command Line.

Run by tags

In TestComplete, you can assign tags to features and scenarios, and then use just one command to run all the features and scenarios that have a tag assigned to them. You can do this from the TestComplete UI, from the command line, from test items, or from scripts. For complete information on this, see Run by Tag.

Note: When you run a feature by a tag, TestComplete runs all the scenarios defined in the feature file, even if these scenarios don’t have the tag assigned to them.

Run via COM

If you automate TestComplete via COM, then you can run BDD scenarios with the Integration.RunTestByName(...) method. It is used to run any TestComplete test – a script function, a keyword test, and so on – by its name. Below are several examples of how you can specify BDD tests for the run:

  • Run a scenario from a feature file:

    tc.Integration.RunTestByName("TestProjectName", "Scenarios|Feature1|First line of the scenario description")

  • Run all the scenarios in a feature file:

    tc.Integration.RunTestByName("TestProjectName", "Scenarios|Feature1")

  • Run scenarios and features by some tag:

    tc.Integration.RunTestByName("TestProjectName", "Scenarios|@tagName")

Full code example
C#

C#

private void TestFunc()
{
  const string TCProgID = "TestComplete.TestCompleteApplication.14";
  object TestCompleteObject = null;

  // Obtain access to TestComplete
  try
  {
    TestCompleteObject = Marshal.GetActiveObject(TCProgID);
  }
  catch
  {
      TestCompleteObject = Activator.CreateInstance(Type.GetTypeFromProgID(TCProgID));
  }

  if (TestCompleteObject == null) return;

  // Obtain the Integration object
  TestComplete.ITestCompleteCOMManager TestCompleteManager = (TestComplete.ITestCompleteCOMManager)TestCompleteObject;
  TestComplete.ItcIntegration IntegrationObject = TestCompleteManager.Integration;

  try
  {
    // Open the project suite
    IntegrationObject.OpenProjectSuiteEx(@"C:\My Project Suite\MyProjectSuite.pjs");

    // Run tests
    IntegrationObject.RunTestByName("TestProjectName", "Scenarios|Feature Name|First line of the scenario description");
    IntegrationObject.RunTestByName("TestProjectName", "Scenarios|Feature Name");
    IntegrationObject.RunTestByName("TestProjectName", "Scenarios|@tagName");
  }
  catch (System.Runtime.InteropServices.COMException ex)
  {
    System.Windows.Forms.MessageBox.Show("An exception occurred: " + ex.Message);
  }
  finally
  {
    // Close TestComplete and release COM objects
    TestCompleteManager.Quit();
    Marshal.ReleaseComObject(IntegrationObject);
    Marshal.ReleaseComObject(TestCompleteManager);
    Marshal.ReleaseComObject(TestCompleteObject);
  }
}

To run this code, you need to configure your application’s manifest. For complete more information on this and on running tests via COM, see Working With TestComplete via COM - Overview.

What happens during the run

  • Running a BDD test means that TestComplete runs script functions associated with test steps. Script code has special statements that link test steps to script functions. For instance, in JavaScript, they look in the following way:

    Test step link statements in JavaScript

  • When you command TestComplete to run a scenario, it runs all test steps of this scenario one by one.

  • When you run a feature file, TestComplete runs all the scenarios defined in that feature file in the order of their appearance in the file.

  • TestComplete considers a test step failed if an error message is posted to the test log during the test step run. The value the test step’s function returns (true or false) doesn’t matter.

    If an unhandled exception occurs, the test engine will post an error message to the log. However, if an exception is handled, the test engine will not log an error, and TestComplete will consider the test step successful.

  • You can define special event handlers (hooks) to be executed before or after a feature or scenario run. See Before and After Hooks.

After the run

Regardless of how you run your BDD tests, TestComplete considers each executed BDD scenario and each BDD feature file as a test case and collects information on their run in a Summary report.

In addition, you can view detailed information on all the test operations executed during each test step run.

See Also

Behavior-Driven Development (BDD) With TestComplete

Highlight search results