Run Test Automations for Zephyr for Jira

Applies to TestComplete 14.10, last modified on June 5, 2019

If you use Zephyr for Jira to plan and monitor your testing efforts, you can use TestComplete to create automations for your test cases in your Jira project. The results of those automation runs will be sent to Jira and included in your test metrics.

This topic describes how to run test automations created with TestComplete and how to send their results to the desired releases and test cycles in Jira.

Requirements

  • Jira Server 7.2.x–7.13.x

    – or –

    Jira Data Center 7.2.x–7.13.x

    – or –

    Jira Cloud

  • TestComplete version 14.10 (or later)

  • Zephyr for Jira Integration Plugin must be enabled in TestComplete. By default, the plugin is installed and enabled automatically when you install TestComplete. If you experience issues with Zephyr integration, make sure the plugin is enabled in the File > Install Extensions dialog (you can find the plugin in the Common group).

  • Your TestComplete workstation must have access to your Jira project.

Prerequisites

  1. In TestComplete, create tests you want to use as automations for your test cases in Jira. To learn how to create tests, see Creating Tests.

  2. Click Connect to Zephyr for Jira on the TestComplete toolbar.

  3. On the resulting Properties > Zephyr for Jira page of your TestComplete project, bind your TestComplete project to your Jira project that contains the Zephyr test cases you want to automate. To learn how to connect projects, see 1. Connect your TestComplete project to your Jira project.

    Connecting TestComplete project to Zephyr for Jira

    Click the image to enlarge it.

  4. (Optional.) Specify the release version and test cycle of your Jira project to which you want to send automation test results.

    Specifying Jira project’s release and test cycle to which test results will be published

    Click the image to enlarge it.

    Note: If you are planning to run tests from the command line or from a CI system, you can specify the release version and test cycle later.

  5. Open the Test Items page of your TestComplete project and add the test items that will run the created tests. To learn how to create test items, see the description of the project’s Test Items page.

    Make sure that the added test items are marked as test cases. Only test cases can be used for Zephyr test case automation:

    Adding test items to your TestComplete project

    Click the image to enlarge it.

    For each test case in TestComplete, specify the Zephyr test case it will automate.

    Assigning test items to Zephyr tests

    Click the image to enlarge it.

    – or –

    Use the aqTestCase.Begin and aqTestCase.End methods to mark the code that you want to use as automations for Zephyr test cases. To learn more, see To Assign TestComplete Test Items to Zephyr Test Cases.

Now, you can run your test automations.

Run test automations

Running test automations from Jira is not currently supported.

Manually from the TestComplete UI

If you assign test items to Zephyr test cases

To run a single test case, select it on the Test Items page and then click Run Focused Item.

To run all the enabled test cases, run the entire project by selecting Test > Run Project from the TestComplete main menu or by clicking Run Project on the Project Explorer toolbar.

If you assign test cases to Zephyr test cases directly in your script units

Run a script routine that contains the aqTestCase.Begin method call. To learn about various ways you can run script routines, see To Run a Script Routine.

Automatically

From the command line

To run all the enabled test cases of your TestComplete project, use the project command line argument:

TestComplete.exe "C:\AutomatedTests\WebStore_Sample_Project_Suite.pjs" /r /p:WebStore_Sample_Project

To run an individual test case of your TestComplete project, use the project and test command line arguments:

TestComplete.exe "C:\AutomatedTests\WebStore_Sample_Project_Suite.pjs" /r /p:WebStore_Sample_Project /t:"Simple test"

To run a script routine that contains the aqTestCase.Begin method call, use the project, unit and routine command line arguments:

TestComplete.exe "C:\AutomatedTests\WebStore_Sample_Project_Suite.pjs" /r /p:WebStore_Sample_Project /unit:Unit1 /routine:Main

To specify the release version and test cycle to which you want to send test results, use the ZephyrVersion and ZephyrCycle command line arguments:

TestComplete.exe "C:\AutomatedTests\WebStore_Sample_Project_Suite.pjs" /r /p:WebStore_Sample_Project /t:"Simple test" /ZephyrVersion:2.0 /ZephyrCycle:Regression

From your continuous integration (CI) system

If you use Jenkins, use the TestComplete Support plugin to add a TestComplete Test build step to your build and configure the step to run the needed test items (or script routines). To learn more, see Integration With Jenkins.

If you use Microsoft Azure DevOps Services or Team Foundation Server, add a TestComplete Test item to your Visual Studio test project, configure it to run the needed test items, and then configure your build definition or pipeline to run the created TestComplete test. To learn more, see Integration With Azure DevOps and Team Foundation Server.

If you use a CI system (for example, Bamboo), use the TestComplete command line arguments described above to run the needed test cases (or script routines) and send the test results to the needed release and test cycle of your Jira project.

Send Test Results to Jira

After TestComplete finishes running the specified test case (or test cases), it will send the test results to your Jira project automatically.

It will send the test results to the Jira project’s release and test cycle the ZephyrVersion and ZephyrCycle command line arguments specify. If the arguments are not specified, TestComplete will send the results to the release and test cycle your TestComplete project’s properties specify. If the properties are not specified either, TestComplete will send the test results to the default release version (Unscheduled) and to the default test cycle (Ad hoc).

If the test cycle that the ZephyrCycle command line argument or the Cycle project property specifies does not exist, it will be created and added to the specified release version.

View Test Results

TestComplete sends the test status (Pass or Fail) to Jira and assigns it to the specified test case execution. You can view it in the Test Execution section of the test case in Jira:

View test results in the Test Executions section of a test case

Click the image to enlarge it.

You can also view it on the Test Executions page of your Jira project:

View the test execution

Click the image to enlarge it.

If a TestComplete test fails, the first error message of the test log and the screenshot of your computer desktop taken when the error occurred will be added to the test execution:

View failed test execution details

Click the image to enlarge it.

If several test cases are assigned in TestComplete to a single Zephyr test case, their results will be added to the test case’s execution history one after another.

In the same way, if your test has several iterations (for example, for test items, you can set the number of iterations in the Count column of your project’s Test Items page), the iteration results will be added to the test case’s execution history one after another.

The test results sent from TestComplete will also be included in your test metrics and charts:

Test metrics in Jira

Click the image to enlarge it.

See Also

Integration With Zephyr for Jira
Running Tests
Integration With Continuous Integration (CI) Systems
Integration With Jenkins

Highlight search results