Configuring Jenkins Freestyle Jobs to Run TestComplete Tests

1. Choose a Node to Run Tests on

2. Add Steps That Will Prepare and Copy Files

3. Add TestComplete Test Step

Example

The node, on which you will run tests, must have TestComplete or TestExecute installed. To make sure the job will run only on this node, select the Restrict where this project can be run check box and enter the name of the node, where you will run TestComplete tests.

Before adding a TestComplete Test build step, you need to prepare and copy TestComplete project files, the tested application and other test-related files and folders to the working folder on the node. If these files are absent from the node, Jenkins will be unable to run the test.

You can place all the files to the node beforehand, or you can add one or multiple build steps to your Jenkins job to perform all the preparation tasks. For example, you can use the Folder Copy operation provided by the File Operations Plugin.

1. In the Build section, click Add build step and choose TestComplete Test from the drop-down list:

   

2. The TestComplete Test build step will appear:

   

3. Configure the step to run the needed TestComplete tests:

   1. Specify the test project suite in the Project suite file field.

      Note:

      The path should be relative to the job’s working folder on the node.

   2. Select the test of the project suite to be run:

      • Entire suite - runs the entire project suite.

      • Project test - runs a specified project in the project suite.

      • Tags (TestComplete 14.20 or later is required) - runs the specified project’s tests that match a tag expression.

        Tag expression syntax

        Tag expression syntaxTag expression syntax

        • Use @ before tag names, that is, use @tag1 rather than tag1.

        • To combine tags, use the and, or and not operations, for example:

          @tag1 or @tag2
          @tag1 and not @tag3
          not @tag5

          These operations don’t depend on the scripting language you use in your TestComplete project. That is, for example, in JavaScript projects you should use and, or and not rather than &&, || and !  —

          @tag1 and not @tag2  ← Correct!
          @tag1 && !@tag1      ← Incorrect!

        • If needed, you can use braces to group tags, for example:

          @tag1 and (@tag2 or @tag3)
          (@tag1 or not @tag2) and (not @tag3 or @tag4)

      • Script test - runs a script routine.

      • Keyword test - runs a keyword test.

      • Other - Runs any test your project contains: a project test item, a low-level procedure, a unit test, a network suite test, and so on. In the Project text box, enter the name of your project. In the Test name text box, enter the full path to the project item or test item to run. Use vertical bars ( | ) as separators.

        Example

        ExampleExample

        Test	Description
        ProjectTestItem1 ProjectTestItem1|ProjectChildTestItem1	The specified test item and all its child test items (if any). Only those test items that are enabled on the Test Items page of the project will be run.
        KeywordTests|Test1	Keyword test Test1.
        Script|Unit1|Main	Script function Main that is located in the Unit1 script unit.
        LLCollection1|LLP1	Low-level procedure LLP1 from the LLCollection1 project item.
        ManualTests|ManualTest1	Manual test ManualTest1.
        UnitTesting1|NUnit1	Unit test NUnit1 from the UnitTesting1 collection.
        ReadyAPI1|Test1	API test Test1 from the ReadyAPI1 collection.
        NetworkSuite|Jobs|Job1|Task1	Task1 from the Job1 job of the network suite.
        NetworkSuite|Jobs|Job1	Job1 of the network suite.

        The test path should contain collection names only. Do not include any project folders into the path.

      • CrossBrowserTesting tests – (Obsolete) Runs tests assigned to Windows-based CrossBrowserTesting environments in the CrossBrowserTesting manager of the specified project suite.

        Running tests by using the CrossBrowserTesting manager with TestExecute is not supported. Starting with version 14.72, running such tests with TestComplete is not supported either.
        We recommend that to run web tests in CrossBrowserTesting environments, you use the cross-platform web tests. This way you can run such tests from Jenkins nodes by using both TestComplete and TestExecute. See the About Cross-Platform Web Tests section in the TestComplete documentation.

   3. Click Advanced to configure additional settings of the TestComplete Test step:

      • Test runner - Specifies the product that the step will use for the test run. If several products are installed and (Any) is selected, TestExecute is run.

        Note:

        The selected product must be installed on the node.

      • Version - Specifies the product version to be used in case you have several versions of these products installed on the node. Jenkins uses the latest installed version by default, and we recommend using the default setting to avoid problems with running your tests in the latest product version.

      • Access key - If you use SmartBear License Management to control your TestExecute licenses, you can specify the access key assigned to your SmartBear account. This key will be used to license the TestExecute instances that will run tests.

        To add a new key, click Add and specify the key in the resulting dialog. To store the key, use the secret text credentials type.

        Learn how to get the access key

        Learn how to get the access keyLearn how to get the access key

        1. Log in to the SmartBear License Management system at manage.smartbear.com with your SmartBear account.

        2. Click  > Access Key at the top right of the page.

        3. From the resulting dialog, copy the access key to the clipboard.

        

        Click the image to enlarge it.

      • Action on warnings - Specifies whether Jenkins should mark the entire build as failed or unstable when the test log contains warnings.

      • Action on errors - Specifies whether Jenkins should mark the entire build as failed or unstable when the test log contains errors.

      • Additional command-line arguments - Specifies arbitrary command-line arguments to pass to TestExecute. To learn more about TestExecute's command line, see TestExecute Command Line.

      • Use test timeout - Specifies the maximum test execution time in seconds. If the test is not finished until the specified period elapses, Jenkins will mark the entire build as failed.

      • Run interactive user session - Specifies whether the test will run in an interactive session.

        This option is effective only if you use TestComplete or TestExecute 10.60 or later.

        If this option is disabled and Jenkins runs as a service, your tests will fail.

        For details on when you need to use this option, see Preparing Jenkins Nodes for Running Tests.

        • Credentials - Specifies the user name/password pair of the account that will be used to run the test. To add a new pair, click Add and specify the credentials data in the resulting dialog.

        • Use active session - Specifies whether an existing user session will be used to run a test. If selected, TestExecute will run in the currently running session. Otherwise, Jenkins will close all applications, terminate the currently active session, start a new session under the specified user account and then start the test.

          Note:

          Jenkins will not stop the user session if another user is logged on.

        • Session screen resolution - Specifies whether an existing user session will be used to run a TestComplete test. The default value is 1280×1024.

          If you use a TestExecute version prior to version 12.30, the specified screen resolution will be ignored, and then the 1280×1024 default screen resolution will be used.

          If the Use active session option is enabled, the specified screen resolution will be ignored, and the actual screen resolution of the appropriate user session will be used.

          Note:

          The maximum resolution that actually can be set for a user session depends on the operating system version and on the Remote Desktop Protocol version installed on the target computer. To learn more, see Remote Desktop Protocol Maximum Supported Resolutions.

      • Generate MHT log file - If this option is enabled, TestExecute exports test results to a .mht file after the build is over. You can download this file from the Jenkins page displaying test results.

        If this option is disabled, TestExecute does not export test results in the MHT format, but you can still examine results in Jenkins. See Viewing TestComplete Test Results in Jenkins.

      • Generate JUnit-style report - Specifies whether Jenkins will generate JUnit-style reports. For more information on how JUnit reports are represented in Jenkins, see Viewing TestComplete Test Results in Jenkins.

        Note: This JUnit-style report slightly differs from standard JUnit reports. To generate JUnit reports, use the /ExportSummary command-line argument. You can specify it in the Additional command-line arguments text box of the step. To learn more about the argument, see TestExecute Command Line.

Here is a sample setup of a TestComplete Test build step:

Example

ExampleExample