Sending TestComplete Test Results From Jenkins to Zephyr

Jira Cloud

– or –

Jira Server 7.2.x–7.13.x

– or –

Jira Data Center 7.2.x–7.13.x

• TestComplete version 14.10 (or later)

• Zephyr Squad Integration Plugin must be enabled in TestComplete. By default, the plugin is installed and enabled automatically when you install TestComplete. If you experience issues with Zephyr integration, make sure the plugin is enabled in the File > Install Extensions dialog (you can find the plugin in the Common group).

• Your TestComplete workstation must have access to your Jira project.

• Your Jenkins server must have the TestComplete Support plugin installed.

1. If your TestComplete test project does not have test items, create them.

   Make sure that the items are marked as test cases. TestComplete can only send test case results to Zephyr.

   To learn how to create test items, mark them as test cases, and assign tests to them, see the description of the Execution Plan editor.

   

   Click the image to enlarge it.

2. Click Connect to Zephyr Squad on the TestComplete toolbar to open the Properties > Zephyr Squad page of your project and bind the project to your Jira project. To learn how to do this, see Integration With Zephyr Squad:

   

   Click the image to enlarge it.

3. To send TestComplete test results to your Jira project, you need to specify the release versions and the test cycle to which you want to add them.

   You can do this directly in the TestComplete project:

   

   Click the image to enlarge it.

   As an alternative, you can do this later when configuring your Jenkins job or pipeline (see below).

4. Open the Execution Plan editor and in the Link to external test case column, bind your project’s test items to Zephyr tests in your Jira project:

   

   Click the image to enlarge it.

   For detailed instructions on how to do this, see Integration With Zephyr Squad.

5. In Jenkins, configure your job or pipeline to run your TestComplete project’s test items. To learn how to do this, see the following topics:

   Configuring Jenkins Freestyle Jobs to Run TestComplete Tests

   Configuring Jenkins Pipelines to Run TestComplete Tests

6. If you did not specify your Jira project’s release version and test cycle to which you want to send test results earlier when configuring your TestComplete project, you can do this now. Use the ZephyrVersion and ZephyrCycle command-line arguments of TestComplete.

   If you use a freestyle job, to do this, use the Advanced > Additional command line arguments property of the TestComplete Test build step:

   

   Click the image to enlarge it.

   If you use a pipeline, use the commandLineArguments property of the TestComplete Test step:

   

   Click the image to enlarge it.

   Note: If you specified the test cycle neither in your TestComplete project, nor in your Jenkins job or pipeline, TestComplete will send the test results to the default Ad hoc test cycle of the specified release. If you specified the release version neither in your TestComplete project, nor in Jenkins, TestComplete will send the test results to the default Unscheduled release.

7. In Jenkins, run your job or pipeline.

8. TestComplete will execute the tests, send their results to Jira and assign them to the specified Zephyr tests as test executions:

   

   Click the image to enlarge it.

• Zephyr Cloud or Zephyr Enterprise versions 4.7.x–6.2.x.

• Your Jenkins server must have the following plugins installed:

  • Zephyr Squad - Test Management for Jira

    – or –

    Zephyr Enterprise Test Management

  • TestComplete Support

• Your Jenkins server must be configured to access your Zephyr server. To learn how to do this, see the documentation for your version of the Zephyr Test Management plugin.

1. In Jenkins, configure your job to run your TestComplete tests. To learn how to do this, see Configuring Jenkins Freestyle Jobs to Run TestComplete Tests:

   

   Click the image to enlarge it.

2. Important: Configure your job to export TestComplete test results to JUnit-style reports. To do this, enable the Advanced > Generate JUnit-style report property of your TestComplete Test build step:

   

   Click the image to enlarge it.

   Otherwise, Jenkins will not be able to publish TestComplete test results to Zephyr.

3. Configure your Jenkins build to post test results to Zephyr after the build is over:

   

   Click the image to enlarge it.

   For detailed instructions on how to do this, see the Configuring plugin as a post-build action section of the documentation for your plugin version on the Jenkins web site:

   plugins.jenkins.io/zephyr-enterprise-test-management (for Zephyr Enterprise plugin)

   plugins.jenkins.io/zephyr-for-jira-test-management (for Zephyr Squad plugin)

4. Run your build.

5. TestComplete will run tests and export the test results.

6. The Zephyr Test Management plugin will capture the exported test results and send them to Zephyr.

   For each executed TestComplete test, the plugin will create a test case and assign the test results to it:

   

   Click the image to enlarge it.

   The name of the created test cases include the name of the TestComplete project suite, project and executed test.

   If the name of the executed TestComplete test matches an existing test case, the plugin will not create a new test case. It will add a new test execution to the existing test case.