Configuring Jenkins Pipelines to Run Cross-Platform Web Tests in Parallel

Applies to TestExecute 14.72, last modified on January 12, 2021

If you use Jenkins Pipeline to control your development and testing workflow, you can include your cross-platform web tests into your workflow and configure your Pipeline to run the tests in parallel. This way, you can distribute the resources of your device cloud more effectively.

If you manage your Pipeline from the Jenkins UI, open the Pipeline section of your project and locate the Script box. If you use SCM, open your Jenkinsfile.

1. Specify nodes on which tests will run

The node where you will run tests must meet the requirements described in Running Cross-Platform Web Tests in Parallel by Using Jenkins. To make sure you will run your tests on the appropriate nodes only, add code to your Pipeline script (Jenkinsfile) that will specify the needed nodes. For example, you can use labels, tags, or node names to specify where the tests should run:

Declarative Pipeline

pipeline {
  agent {
    label TELite'
  }
  …
}

Scripted Pipeline

node('TELite') {
  …
}

2. Add steps that will prepare nodes to run your cross-platform web tests

  1. Copy TestComplete project files and other test-related files to the nodes where the test will run.

    Note: TestComplete test project files must reside in the node’s working folder or in its subfolders.

  2. Make sure that tests can access all the files they need during the run.

Otherwise, Jenkins will not be able to run your test.

You can copy all the needed files to the nodes in advance, or you can add steps to your Jenkins Pipeline to perform all the preparation tasks. To learn more about the steps you can use, see jenkins.io/doc/pipeline/steps/.

3. Add script statements to run tests

To run a test, you use the TestComplete Test step that the TestComplete Support plugin provides. Each step can run a single test.

To run several tests in parallel:

  • Configure your TestComplete Test steps to use TestExecute Lite (Device Cloud Parallel) as the test runner. You can write the step code manually, or you can use the Jenkins Snippet Generator utility to generate the code automatically. For the full test step syntax reference, see:

    TestComplete Test Step Pipeline Syntax for Parallel Test Runs

  • Use the parallel section to run several tests at once. The parallel section allows creating stages and steps that run in parallel. To learn more about the parallel section, see the appropriate section in the Jenkins documentation:

    Pipeline Syntax - Parallel

  • The TestComplete Test step will add test results to your build results automatically (see Viewing Cross-Platform Web Test Results in Jenkins). To export test results to an external file, configure the step to do it manually. See Exporting Test Results.

The code snippet below shows how to run two TestComplete Test steps in parallel:

Declarative Pipeline

pipeline{
  …
  stages{
    stage('QA'){
      steps{
        parallel(
          Test1:{
            testcompletetest executorType:'TElite',
            credentialsId: '',
            launchType: 'lcItem',
            test:'Test 1',
            project: 'TestProject',
            suite: 'TestProject\\TestProject.mds'
          },
          Test2:{
            testcompletetest executorType:'TElite',
            credentialsId: '',
            launchType: 'lcItem',
            test:'Test 2',
            project: 'TestProject',
            suite: 'TestProject\\TestProject.mds'
            }
            )
      }
    }
  }
}

Scripted Pipeline

node {
  stage('QA') {
    parallel(
      Test1:{
        testcompletetest executorType:'TElite',
        credentialsId: '',
        launchType: 'lcItem',
        test:'Test 1',
        project: 'TestProject',
        suite: 'TestProject\\TestProject.mds'
      },
      Test2:{
        testcompletetest executorType:'TElite',
        credentialsId: '',
        launchType: 'lcItem',
        test:'Test 2',
        project: 'TestProject',
        suite: 'TestProject\\TestProject.mds'
      }
      )
  }
}

You can find more examples in the section below.

Examples

The examples below show how to configure both declarative and scripted Pipelines to run tests in parallel. To iterate the tests to run, they use loops.

Note: Because the declarative Pipeline does not support loops directly, the examples of the declarative Pipeline use the script step. This step allows running an arbitrary script code, including a loop.

Example 1: Run several tests in parallel

The following example shows how to run several tests in parallel. The name of the tests to run is defined in a list:

Declarative Pipeline

def tests = [:]
def testName = [
  Test1: 'Test 1',
  Test2: 'Test 2',
  Test3: 'Test 3',
  ]
def projectPath = 'TestProject\\TestProject.mds'

def makeTestStep(aTestName, aProjectPath) {
  return {
    testcompletetest credentialsId: '', executorType: 'TElite', launchType: 'lcItem', test:"${aTestName}", project: 'TestProject', suite: "${aProjectPath}"
  }
}

pipeline{
  agent any
  stages{
    stage('QA')
    {
      steps{
        script{
          testName.each{ entry ->
            tests["${entry.value}"] = makeTestStep("${entry.value}", "${projectPath}")
          }

          parallel tests
        }
      }
    }
  }
}

Scripted Pipeline

def tests = [:]
def testName = [
  Test1: 'Test 1',
  Test2: 'Test 2',
  Test3: 'Test 3',
  ]
def projectPath = 'C:\\Users\\tester\\Documents\\TestComplete 14 Projects\\TestProject\\TestProject\\TestProject.mds'

def makeTestStep(aTestName, aProjectPath) {
  return {
    testcompletetest credentialsId: '', executorType: 'TElite', launchType: 'lcItem', test:"${aTestName}", project: 'TestProject', suite: "${aProjectPath}"
    }
}

testName.each{ entry ->
  tests["${entry.value}"] = makeTestStep("${entry.value}", "${projectPath}")
} node()
{
  stage('QA'){
    parallel tests
  }

}

Example 2: Run a test in several environments in parallel

The following example shows how to run a test in 5 various environments in parallel. The environments where the test will run are specified in a browsers list.

To send the environment to the test to run, the example uses the -pv command-line argument that it sends to the test runner utility by using the commandLineArguments parameter.

View test code example and description

Declarative Pipeline

def tests = [:]
def makeTestStep(aBrowser) {
  return {
        testcompletetest executorType: 'TElite', launchType: 'lcProject', project: 'TestProject', suite: 'TestProject\\TestProject.mds', commandLineArguments: "-pv:browser=${aBrowser}", credentialsId: ''
    }
  }

def browsers = [
  Safari : 'Safari',
  Chrome: 'Chrome',
  Firefox: 'Firefox',
  Edge: 'Edge',
  IE: 'IE']

pipeline{
  agent any
  stages{
    stage('QA'){
      steps{
        script{
          browsers.each{ entry ->
            tests["${entry.value}"] = makeTestStep("${entry.value}")
          }
          parallel tests
        }
      }
    }
  }
}

Scripted Pipeline

def tests = [:]
def makeTestStep(aBrowser) {
  return {
      testcompletetest executorType: 'TElite', launchType: 'lcProject', project: 'TestProject', suite: 'TestProject\\TestProject.mds', commandLineArguments: "-pv:browser=${aBrowser}", credentialsId: ''
      }
  }

def browsers = [
  Safari : 'Safari',
  Chrome: 'Chrome',
  Firefox: 'Firefox',
  Edge: 'Edge',
  IE: 'IE']

configs.each{ entry ->
  tests["${entry.value}"] = makeTestStep("${entry.value}")
  }

node(){
  stage('QA'){
    parallel tests
  }
}

See Also

Running Cross-Platform Web Tests in Parallel by Using Jenkins

Highlight search results