Sending TestComplete Test Results from Jenkins to Zephyr

Applies to TestExecute 14.72, last modified on January 12, 2021

If you use Zephyr to plan and monitor your testing efforts and use Jenkins to run your automated TestComplete tests, you may want to include your TestComplete test results in your test reports and metrics in Zephyr.

There are several way to do this:

Approach 1: Use TestExecute integration with Zephyr for Jira

Applicable only if you use TestExecute version 14.10 and Zephyr for Jira.

Requirements and Prerequisites

  • Jira Server 7.2.x–7.13.x

    – or –

    Jira Data Center 7.2.x–7.13.x

    – or –

    Jira Cloud

  • Your Jira must have the following plugins installed:

  • TestExecute version 14.10 (or later)

  • The Zephyr Integration Plugin must be enabled in TestExecute. By default, the plugin is installed and enabled automatically when you install TestExecute. If you experience issues with the Zephyr integration, make sure the plugin is enabled in the File > Install Extensions dialog (you can find the plugin in the Common group).

  • Your TestExecute workstation must have access to your Jira project.

  • Your Jenkins server must have the TestComplete Support plugin installed.

How to send results

  1. Open your project in TestComplete. If your project does not have test items, create them. TestExecute can send the results of only test items to Zephyr. To learn how to create test items and assign tests to them, see the TestComplete documentation.

    Test items of the TestComplete project

    Click the image to enlarge it.

  2. In TestComplete, open the Properties > Zephyr page of your project and bind the project to your Jira project:

    Connecting TestComplete project to Zephyr for Jira

    Click the image to enlarge it.

  3. To send test results to your Jira project, you need to specify the release versions and the test cycle to which you want to add them.

    You can do this directly in the TestComplete project:

    Setting the target Jira project release and test cycle

    Click the image to enlarge it.

    As an alternative, you can do this later when configuring your Jenkins job or pipeline (see below).

  4. In the Execution Plan editor of your project, in the Report To column, bind your project’s test items to Zephyr tests in your Jira project:

    Assigning test items to Zephyr tests

    Click the image to enlarge it.

  5. In Jenkins, configure your job or pipeline to run your TestComplete project’s test items. To learn how to do this, see the following topics:

  6. If you didn't specify your Jira project’s release version and test cycle when configuring your TestComplete project, you can do this now. Use the ZephyrVersion and ZephyrCycle command-line arguments of TestExecute.

    If you use a freestyle job, to specify the release and test cycle, use the Advanced > Additional command line arguments property of the TestComplete Test build step:

    Setting the target Jira project release and test cycle in Jenkins (freestyle job)

    Click the image to enlarge it.

    If you use a pipeline, use the commandLineArguments property of the TestComplete Test step:

    Setting the target Jira project release and test cycle in Jenkins (pipeline)

    Click the image to enlarge it.

    Note: If you specified a test cycle neither in your TestComplete project, nor in your Jenkins job or pipeline, TestComplete will send the test results to the default Ad hoc test cycle of the specified release. If you specified a release version neither in your TestComplete project, nor in Jenkins, TestComplete will send the test results to the default Unscheduled release.

  7. In Jenkins, run your job or pipeline.

  8. TestExecute will run the tests, send their results to Jira and assign them to the specified Zephyr tests:

    TestComplete test results in Jira

    Click the image to enlarge it.

    If the test run fails, information on the first error posted to the test log will be added to the test execution’s comment section:

    Error message posted from TestExecute

    Click the image to enlarge it.

Approach 2: Use Zephyr integration with Jenkins

Applicable if you use Zephyr for Jira or Zephyr Enterprise.

Not applicable for Jenkins Pipeline.

Requirements and Prerequisites

How to send results

  1. In Jenkins, configure your job to run your TestComplete tests. To learn how to do this, see Configuring Jenkins Freestyle Jobs to Run TestComplete Tests:

    Running TestComplete tests from Jenkins

    Click the image to enlarge it.

  2. Important: Configure your job to export test results to JUnit-style reports. To do this, enable the Advanced > Generate JUnit-style report property of your TestComplete Test build step:

    Generating JUnit-style reports in Jenkins

    Click the image to enlarge it.

    Otherwise, Jenkins will not be able to publish test results to Zephyr.

  3. Configure your Jenkins build to post test results to Zephyr after the build is over:

    Sending test results from Jenkins to Zephyr

    Click the image to enlarge it.

    For detailed instructions on how to do this, see the Configuring plugin as a post-build action section in the documentation for your plugin version on the Jenkins web site:

  4. Run your build.

  5. TestExecute will run the tests and export the test results.

  6. The Zephyr Test Management plugin will capture the exported test results and send them to Zephyr.

    For each executed test, the plugin will create a test case and assign test results to it:

    Test case created from a TestComplete test

    Click the image to enlarge it.

    The name of the created test case includes the name of the TestComplete project suite, project and executed test.

    If the name of the executed test matches the name of an existing test case, the plugin will not create a new test case. It will add a new test execution to the existing test case.

See Also

Integration With Jenkins
Requirements
Configuring Jenkins Freestyle Jobs to Run TestComplete Tests
Configuring Jenkins Pipelines to Run TestComplete Tests

Highlight search results