Sending TestComplete Test Results From Jenkins to Zephyr

Applies to TestComplete 15.44, last modified on November 10, 2022

If you use Zephyr to plan and monitor your testing efforts and use Jenkins to run your automated TestComplete tests, you may want to include your TestComplete test results in your test reports and metrics in Zephyr.

There are several ways to do this:

Approach 1: Use TestComplete integration with Zephyr Squad

Applicable only if you use TestComplete version 14.10 and Zephyr Squad.

Requirements and Prerequisites

Jira Cloud

– or –

Jira Server 7.2.x–7.13.x

– or –

Jira Data Center 7.2.x–7.13.x

  • TestComplete version 14.10 (or later)

  • Zephyr Squad Integration Plugin must be enabled in TestComplete. By default, the plugin is installed and enabled automatically when you install TestComplete. If you experience issues with Zephyr integration, make sure the plugin is enabled in the File > Install Extensions dialog (you can find the plugin in the Common group).

  • Your TestComplete workstation must have access to your Jira project.

How to send results

  1. If your TestComplete test project does not have test items, create them.

    Make sure that the items are marked as test cases. TestComplete can only send test case results to Zephyr.

    To learn how to create test items, mark them as test cases, and assign tests to them, see the description of the Execution Plan editor.

    Test items of a TestComplete project

    Click the image to enlarge it.

  2. Click Connect to Zephyr Squad on the TestComplete toolbar to open the Properties > Zephyr Squad page of your project and bind the project to your Jira project. To learn how to do this, see Integration With Zephyr Squad:

    Connecting TestComplete project to Zephyr Squad

    Click the image to enlarge it.

  3. To send TestComplete test results to your Jira project, you need to specify the release versions and the test cycle to which you want to add them.

    You can do this directly in the TestComplete project:

    Setting the target Jira project release and test cycle in TestComplete

    Click the image to enlarge it.

    As an alternative, you can do this later when configuring your Jenkins job or pipeline (see below).

  4. Open the Execution Plan editor and in the Link to external test case column, bind your project’s test items to Zephyr tests in your Jira project:

    Assigning TestComplete test items to Zephyr tests

    Click the image to enlarge it.

    For detailed instructions on how to do this, see Integration With Zephyr Squad.

  5. In Jenkins, configure your job or pipeline to run your TestComplete project’s test items. To learn how to do this, see the following topics:

  6. If you did not specify your Jira project’s release version and test cycle to which you want to send test results earlier when configuring your TestComplete project, you can do this now. Use the ZephyrVersion and ZephyrCycle command-line arguments of TestComplete.

    If you use a freestyle job, to do this, use the Advanced > Additional command line arguments property of the TestComplete Test build step:

    Setting the target Jira project release and test cycle in Jenkins (freestyle job)

    Click the image to enlarge it.

    If you use a pipeline, use the commandLineArguments property of the TestComplete Test step:

    Setting the target Jira project release and test cycle in Jenkins (pipeline)

    Click the image to enlarge it.

    Note: If you specified the test cycle neither in your TestComplete project, nor in your Jenkins job or pipeline, TestComplete will send the test results to the default Ad hoc test cycle of the specified release. If you specified the release version neither in your TestComplete project, nor in Jenkins, TestComplete will send the test results to the default Unscheduled release.

  7. In Jenkins, run your job or pipeline.

  8. TestComplete will execute the tests, send their results to Jira and assign them to the specified Zephyr tests as test executions:

    TestComplete test results in Jira

    Click the image to enlarge it.

Approach 2: Use Zephyr integration with Jenkins

Applicable if you use Zephyr Squad or Zephyr Enterprise.

Not applicable for Jenkins Pipeline.

Requirements and Prerequisites

How to send results

  1. In Jenkins, configure your job to run your TestComplete tests. To learn how to do this, see Configuring Jenkins Freestyle Jobs to Run TestComplete Tests:

    Running TestComplete tests from Jenkins

    Click the image to enlarge it.

  2. Important: Configure your job to export TestComplete test results to JUnit-style reports. To do this, enable the Advanced > Generate JUnit-style report property of your TestComplete Test build step:

    Generating JUnit-style reports for TestComplete tests in Jenkins

    Click the image to enlarge it.

    Otherwise, Jenkins will not be able to publish TestComplete test results to Zephyr.

  3. Configure your Jenkins build to post test results to Zephyr after the build is over:

    Sending test results from Jenkins to Zephyr

    Click the image to enlarge it.

    For detailed instructions on how to do this, see the Configuring plugin as a post-build action section of the documentation for your plugin version on the Jenkins web site:

  4. Run your build.

  5. TestComplete will run tests and export the test results.

  6. The Zephyr Test Management plugin will capture the exported test results and send them to Zephyr.

    For each executed TestComplete test, the plugin will create a test case and assign the test results to it:

    Test case created from TestComplete test

    Click the image to enlarge it.

    The name of the created test cases include the name of the TestComplete project suite, project and executed test.

    If the name of the executed TestComplete test matches an existing test case, the plugin will not create a new test case. It will add a new test execution to the existing test case.

See Also

Integration With Jenkins
Preparing Jenkins Nodes for Running TestComplete Tests
Integration With Zephyr Squad

Highlight search results