Configuring Jenkins Pipelines to Run TestComplete Tests

Applies to TestComplete 14.91, last modified on July 20, 2021

To run your TestComplete tests as part of a Jenkins Pipeline:

1. Specify nodes on which tests will run

The node where you will run tests must have TestComplete or TestExecute installed. To make sure you will run your tests on the appropriate nodes only, add code to your Pipeline script (Jenkinsfile) that will specify the needed nodes. For example, you can use labels, tags or node names to specify where the tests should run:

Declarative Pipeline

pipeline {
  agent {
    label 'testagent'
  }
  …
}

Scripted Pipeline

node('testagent') {
  …
}

For details, see the Jenkins Pipeline documentation at jenkins.io/doc/book/pipeline/syntax/.

2. Add steps that will prepare testing environments

Prepare your nodes to run TestComplete tests:

  1. Copy TestComplete project files, your tested application, and other test-related files to the nodes where the test will run.

    Note: TestComplete test project files must reside in the node’s working folder or in its subfolders.
  2. Make sure TestComplete tests can access all the files they needed during the test run.

Otherwise, Jenkins will not be able to run your test.

You can copy all the needed files to the nodes in advance, or you can add steps to your Jenkins Pipeline to perform all the preparation tasks. To learn more about the steps you can use, see jenkins.io/doc/pipeline/steps/.

3. Add script statements to run TestComplete tests

  1. If you manage your Pipeline from the Jenkins UI, open the Pipeline section of your project and locate the Script box.

    If you use SCM, open your Jenkinsfile.

  2. Add the code that will run TestComplete tests to your Pipeline script. You can write the code manually (see the Pipeline Syntax section below).

    – or –

    You can use the built-in Jenkins Snippet Generator utility to generate the code automatically:

    1. Open the Pipeline Syntax page of your Pipeline.

    2. On the resulting page, select testcomplete: TestComplete test from the Steps drop-down list:

      Jenkins Snippet Generator: Select the TestComplete Test step

      Click the image to enlarge it.

    3. Configure the step to run your TestComplete tests:

      View instructions

    4. Click Generate Pipeline Script:

      Jenkins Snippet Generator: Configure the step and generate the code

      Click the image to enlarge it.

    5. Copy the generated code, and then paste it to your Pipeline script (Jenkinsfile). For example:

      Declarative Pipeline

      pipeline{
        agent none
        stages{
          stage('QA'){
            agent{
              label 'TestExecute'
            }
            steps{
              testcompletetest credentialsId: 'tester',
                executorType: 'TE',
                launchType: 'lcRoutine',
                project: 'TestProject1',
                routine: 'Test1',
                suite: 'ProjectSuite1\\ProjectSuite1.pjs',
                unit: 'Unit1',
                useTCService: true
             }
          }
        }
      }

      Scripted Pipeline

      node('testexecute'){
        stage('QA'){
          testcompletetest credentialsId: 'tester',
            executorType: 'TE',
            launchType: 'lcRoutine',
            project: 'TestProject1',
            routine: 'Test1',
            suite: 'ProjectSuite1\\ProjectSuite1.pjs',
            unit: 'Unit1',
            useTCService: true
          }
      }

Examples

  • The following sample script runs a TestComplete project suite (all the enabled test items in all the enabled projects that belong to that project suite):

    testcompletetest suite: 'Projects\\JenkinsTests.pjs'
  • The following sample script runs the MyProj project of the project suite:

    testcompletetest launchType: 'lcProject', project: 'MyProj', suite: 'Projects\\JenkinsTests.pjs'
  • The following sample script runs the ProjectTestItem3 test item that is a child item of the ProjectTestItem1 test item of the MyProj project:

    testcompletetest launchType: 'lcItem', project: 'MyProj', suite: 'Projects\\JenkinsTests.pjs', test: 'ProjectTestItem1|ProjectTestItem3'
  • The following sample script runs the MyKDT keyword test of the MyProj project:

    testcompletetest launchType: 'lcKdt', project: 'MyProj', suite: Projects\\JenkinsTests.pjs', test: 'MyKDT'
  • The following sample script runs the Main script routine located in the Unit1 unit of the MyProj project:

    testcompletetest launchType: 'lcRoutine', project: 'MyProj', routine: 'Main', suite: 'Projects\\JenkinsTests.pjs', unit: 'Unit1'
  • The following sample script runs the MyProj project’s tests that match the @tag1 or (@tag2 and @tag3) tag expression:

    testcompletetest launchType: 'lcTags', project: 'MyProj', suite: 'Projects\\JenkinsTests.pjs', tags: '@tag1 or (@tag2 and @tag3)'
  • The following sample script opens an interactive user session on the node and runs the MyProj project:

    testcompletetest credentialsId: 'tester1', launchType: 'lcProject', project: 'MyProj', suite: 'Projects\\JenkinsTests.pjs',
    useTCService: true
  • The following sample script runs the MyProj project of the project suite and exports the test results to a .mht file:

    testcompletetest generateMHT: true, launchType: 'lcProject', project: 'MyProj', suite: 'Projects\\JenkinsTests.pjs'

TestComplete Test step pipeline syntax

suite

Required. Specifies the path to the TestComplete project suite that contains the tests you want to run. The path should be relative to the node’s working folder.

Type: string

accessKeyId

If you use SmartBear License Management to control your TestComplete (TestExecute) licenses, use this parameter to specify the identifier of the access key assigned to your SmartBear account and that will be used to license the TestComplete (TextExecute) instances that will run tests.

Learn how to get the access key

Note: To store the access key in the Credentials storage in Jenkins, use the secret text credentials type.

Type: string

actionOnErrors

Optional. Specifies whether Jenkins should mark the entire build as failed or unstable if the test log contains errors, or whether it should not mark the build in any special way. If the parameter is not specified, Jenkins will mark the build as unstable.

Type: string

Possible values: NONE, MAKE_UNSTABLE, MAKE_FAILED

Default value: MAKE_UNSTABLE

actionOnWarnings

Specifies whether Jenkins should mark the entire build as failed or unstable if the test log contains warnings, or whether it should not mark the build in any special way. If the parameter is not specified, Jenkins will not mark the build in any special way.

Type: string

Possible values: NONE, MAKE_UNSTABLE, MAKE_FAILED

Default value: NONE

commandLineArguments

Optional. Specifies arbitrary command-line arguments to be passed to TestComplete (TestExecute). To learn more about the TestComplete command line, see TestComplete Command Line.

Type: string

executorType

Optional. Specifies the product that the step will use for the test run. If several products are installed and the parameter is not specified, TestExecute will be used.

Type: string

Possible values: TC, TE, TElite

Default value: TE

Note: The specified product must be installed on the node. If the 64-bit version of the product is available on the node, it will be used to run tests. Otherwise, the 32-bit version will be used.

executorVersion

Optional. Specifies the product version to be used if several versions of the products are installed on the node. If the parameter is not specified, the latest installed version will be used.

Type: string

generateMHT

Optional. Specifies whether the test results will be exported to a .mht file. You can download this file from the Jenkins results page.

Type: boolean

Default value: false

launchType

Optional. Specifies which part of the TestComplete project suite will be run.

Type: string

Possible values:

  • lcSuite - The default value. Runs the entire project suite (that is, all the enabled test items in all the enabled projects of that project suite). The required suite parameter specifies the needed project suite.

  • lcProject - Runs a project of the project suite. To specify the project name, use the project parameter:

    project

    Required. Specifies the name of the project that contains the tests to run. TestComplete (TestExecute) will run all enabled test items of the specified project. You can view the list of test items in the Execution Plan editor of the project.

    Type: string

  • lcRoutine - Runs a script routine. To specify the needed routine, use the following parameters:

    project

    Required. Specifies the name of the project that contains the tests to run.

    Type: string

    routine

    Required. Specifies the name of the script routine to run (without the unit name).

    Type: string

    unit

    Required. Specifies the name of the script unit that contains the script routine to run.

    Type: string

  • lcKdt - Runs a keyword test. To specify the needed test, use the following parameters:

    project

    Required. Specifies the name of the project that contains the tests to run.

    Type: string

    test

    Required. Specifies the name of the keyword test to run. For example, MyKDTest1.

    Type: string

  • lcCBT - (Obsolete) Runs all the tests defined in CrossBrowserTesting manager.

    This approach to run tests in CrossBrowserTesting environments is deprecated starting with TestComplete 14.40. Starting with TestComplete 14.72, it is removed from the product. If you have TestComplete 14.72 installed on your Jenkins agents, your test run will fail.
    To run tests in CrossBrowserTesting environments, we recommend that you use the cross-platform web tests.

    Type: string

  • lcItem - Runs any test your project contains: a project test item, a low-level procedure, a unit test, a network suite test, and so on. To specify the items to run, use the following parameters:

    project

    Required. Specifies the name of the project that contains the tests to run.

    Type: string

    test

    Required. Specifies the full path to the project item or test item to run. The vertical pipe character ( | ) is used as a separator.

    Example

    The path to your test must contain collection names only. Do not include any project folders in the path.

    Type: string

  • lcTags - Runs the project’s tests that match a tag or a tag expression. To specify a project and a tag expression, use the following parameters:

    project

    Required. Specifies the name of the project that contains the tests to run.

    Type: string

    tags

    Required. A tag or a tag expression that specifies the tests to run.

    Tag expression syntax

    Type: string

publishJUnitReports

Optional. Specifies whether Jenkins will generate JUnit-style reports.

Note: This JUnit-style report slightly differs from standard JUnit reports. To generate JUnit reports, use the /ExportSummary command line argument. You can specify it using the commandLineArguments parameter.

Type: boolean

Default value: true

useTCService

Optional. Specifies whether your TestComplete test will run in an interactive session.

Note: If this option is disabled and Jenkins is running as a service, your TestComplete tests will fail.

Type: boolean

To configure the session, use the following parameters:

credentialsId

If you use the Credentials plugin to store your credentials, this parameter specifies the identifier of the user name/password pair of the user account that will be used to open an interactive user session on the target node and run the test.

Type: string

userName

If you use the TestComplete Support plugin to store your credentials (not recommended), this parameter specifies the user account that will be used to run the test.

Type: string

userPassword

If you use the TestComplete Support plugin to store your credentials (not recommended), this parameter specifies the password used to log in to the user account.

Type: string

useActiveSession

Optional. Specifies whether an existing user session will be used to run tests. If the parameter is not specified, TestComplete (TestExecute) will run in the current session. Otherwise, Jenkins will close all the applications, terminate the currently active session, start a new session under the specified user account, and then start the test.

Note: Jenkins will not stop the user session if another user is logged on.

Type: boolean

Default value: true

sessionScreenResolution

Optional. Specifies the screen resolution of the user session in which your tests will run.

If the useActiveSession parameter is set to true, the sessionScreenResolution parameter will be ignored, and the actual resolution of the active user session will be used.

Type: string

Default value: 1280×1024

useTimeout

Optional. Specifies whether Jenkins should wait for the test run to finish until the specified timeout is reached. If the test run is not over before the timeout is reached, Jenkins will mark the entire build as failed.

If the parameter is set to false, the waiting time is infinite.

Type: boolean

Default value: false

To specify a timeout value, use the following parameter:

timeout

Optional. Specifies the maximum test run time, in seconds.

The timeout includes both the startup time of TestComplete (TestExecute) and the test run time.

If the parameter is not specified, the waiting time is infinite.

Type: string

See Also

Integration With Jenkins
Preparing Jenkins for Running TestComplete Tests
Preparing Jenkins Nodes for Running TestComplete Tests

Highlight search results