Run Tests in TestEngine Using Jenkins

Applies to TestEngine 1.31, last modified on June 07, 2024

About

You might want to launch ReadyAPI functional and security tests in TestEngine as a part of your CI/CD flow in Jenkins. The ReadyAPI TestEngine Jenkins Plugin allows you to easy do that.

Requirements

  • The 1.7.0 plugin version requires Jenkins 2.222.4 or later.

  • You require a license to run tests using the plugin. See here for licensing.

ReadyAPI TestEngine Jenkins Plugin

Install or update the plugin

The plugin version 1.2.0 no longer supports specifying project password and user credentials in the corresponding fields. It uses the Credentials plugin instead.
Modify test jobs that use the plugin after you update it to version 1.2.0.

To install the plugin:

  1. Download the Jenkins plugin from our website:

    https://support.smartbear.com/downloads/readyapi/
  2. Open Jenkins and select Manage Jenkins > Manage Plugins:

    Install the TestEngine Jenkins plugin: Manage Jenkins

    Click the image to enlarge it.

  3. In the Advanced tab, go to the Upload Plugin section, click Choose File and select the .hpi file with the plugin.

  4. Click Upload:

    Install the TestEngine Jenkins plugin: Upload the plugin

    Click the image to enlarge it.

Global settings

To configure global settings of the plugin, select Manage Jenkins > Configure System:

Manage Jenkins

Click the image to enlarge it.

You will find the settings in the ReadyAPI TestEngine Plugin section:

Plugin's global settings

Click the image to enlarge it.

Option Description
Server URL

The URL of the TestEngine server.

Credentials
Note: Only in plugin version 1.2.0 and later.

Select credentials of a TestEngine user.

Username
Only in plugin version 1.1.2 and earlier.
If you use it, consider updating the plugin to use the Credentials plugin instead.

The username that will be used to log in to the TestEngine instance.

Server password
Only in plugin version 1.1.2 and earlier.
If you use it, consider updating the plugin to use the Credentials plugin instead.

The user's password

JUnit Report directory

The directory in your workspace to which JUnit reports will be saved. If it is not specified, no reports will be saved.

Click Test Connection to check whether Jenkins can connect to TestEngine using the specified URL, username and password.

Run tests

By using a freestyle project

To run ReadyAPI test in TestEngine by using Jenkins:

  1. Make sure the ReadyAPI TestEngine Jenkins Plugin is installed.

  2. In Jenkins, select a job and click Configure.

  3. In the Build section on the General tab, click Add build step > Run ReadyAPI Projects in TestEngine:

    Run the test using Jenkins: Add the build step
  4. Jenkins will add the step to the build. Configure the build step options and click Save:

    Run the test using Jenkins: Build step options

    Click the image to enlarge it.

The ReadyAPI TestEngine Jenkins Plugin supports launching tests from Jenkins Pipeline. To do this, use the testEngine step:

  1. Make sure the ReadyAPI TestEngine Jenkins Plugin is installed.

  2. In Jenkins, select a job and click Pipeline.

  3. In the Pipeline section, add the testEngine step to your Pipeline script. For example:

    Jenkins pipeline

    pipeline {
       agent any
       stages {
          stage('Tests') {
             steps {
                testEngine (
                    pathToProjectFile: '/home/user/work/my-ready-api-project.xml',                 projectFileCredentialsId: 'i5n70i1i-b392-69l3-c047-5l183b89icj7'             )
             }
          }
       }
    }

Build test step options

You can configure the following build step options:

Option Description
Path to project file or directory

The fully-qualified path to the ReadyAPI project you want to run.

Pipeline option: pathToProjectFile.
Usage: pathToProjectFile: '/home/user/work/my-ready-api-project.xml'.

Project options
Test suite name

The test suite to run. To run all the test suites of your project, leave the field blank.

Pipeline option: testSuiteName.
Usage: testSuiteName: 'My test suite'.

Test case name

The test case to run. If you leave the field blank, the runner will execute all the test cases of the specified test suite, or, if you have not specified a test suite, all the test cases of your project.

Pipeline option: testCaseName.
Usage: testCaseName: 'My test case'.

Security test name

The security test to run.

Pipeline option: securityTestName.
Usage: securityTestName: 'My security test'.

If you specify a security test to be run, the Test suite name and Test case name fields must be empty because TestEngine cannot run both functional tests and security tests within the same test job. Use separate build test steps for functional and security tests.
Active environment

The environment configuration for the test run.

Pipeline option: environment.
Usage: environment: 'staging'.

Callback URL

The URL to which Jenkins will post the results of the test run.

Note: The URL must allow POST requests.

Pipeline option: callbackUrl.
Usage: callbackUrl: 'http://my-service:8081'.

Priority Job

Enable this option to skip all non-priority jobs in the queue.

Pipeline option: priorityJob.
Usage: priorityJob: true.

To be able to use this option, the TestEngine user specified either in the global plugin settings or in the TestEngine connection options must have administrative rights.

Click Advanced to see more options.

Advanced project options
Project file password

The encryption password. It is needed if you have encrypted the entire project or some of its custom properties. See Protecting Sensitive Data.

In plugin version 1.2.0 or later, select a credentials instance containing the desired password.

Pipeline option: projectFileCredentialsId. To specify the password in a pipeline, use the id of the corresponding credentials. You can get it from the Credentials section or by using the Snippet Generator utility.
Usage: projectFileCredentialsId: 'i5n70i1i-b392-69l3-c047-5l183b89icj7'.

Endpoint

The endpoint in the <host>:<port> format. For tests that are run by using the plugin, all the HTTP requests will use the endpoint specified here.

Pipeline option: hostAndPort.
Usage: hostAndPort: 'http://my-service:8082'.

Tags

A comma-separated list of tags to filter test cases and test suites by. Only the test cases and test suites matching all the tags will be run.

Pipeline option: tags.
Usage: tags: 'Login operations, Samples'.

JUnit Report directory

The directory in your workspace where JUnit reports will be saved.

Note: Overrides the JUnit Report Directory global setting.

Pipeline option: reportDirectory.
Usage: reportDirectory: 'testengine/reports'.


Custom Properties

In the Custom Properties section, you specify values of the custom properties to be used in the test run.

  • Select the test suite or test case that contains the custom property to override. Click Add to create the needed target if needed.

  • Click Add to add a custom property. Specify the name of the property in the Property Key column and the needed value in the Property Value column:

    Specify custom properties in the ReadyAPI TestEngine Jenkins Plugin

    Click the image to enlarge it.

TestEngine Connection Options
Server URL

The URL of the TestEngine instance.

Note: Overrides the Server URL global setting.

Pipeline option: serverUrl.
Usage: serverUrl: 'http://testengine:8080.

Credentials
Note: Available only in plugin version 1.1.2 and later.

Select credentials of a TestEngine user.

Note: Overrides the Credentials specified in the global settings.

Pipeline option: serverCredentialsId. To specify the credentials in a pipeline, use its id. You can get it from the Credentials section or by using the Snippet Generator utility.
Usage: serverCredentialsId: 'b2385f93-9d91-4b42-ae38-5580527bee48'.

Username
Only in plugin version 1.1.2 and earlier.
If you use it, consider updating the plugin to use the Credentials plugin instead.

The username that will be used to log in to the TestEngine instance.

Note: Overrides the Username global setting.
Password
Only in plugin version 1.1.2 and earlier.
If you use it, consider updating the plugin to use the Credentials plugin instead.

The user’s password.

Note: Overrides the Password global setting.

Click Test Connection to check whether Jenkins can connect to TestEngine using the specified URL, username and password.

Specify credentials

For specifying sensitive data - the encryption project password and credentials of a TestEngine user - the ReadyAPI TestEngine plugin uses the Credentials Jenkins plugin.

To specify new credentials:

  1. Click Add next to the credentials drop-down list and select the needed provider (Jenkins by default):

    ReadyAPI TestEngine plugin: Add credentials on the spot

    Click the image to enlarge it.

    Alternatively, you can go to the Credentials section and click Add credentials in the needed domain:

    ReadyAPI TestEngine plugin: Add credentials in the Credentials manager

    Click the image to enlarge it.

  2. Specify the needed values:

    ReadyAPI TestEngine plugin: Specifying new credentials

    Click the image to enlarge it.

    Option Description
    Domain

    Select the domain where you want to store the credentials.

    Kind

    Select the kind of the credentials:

    • Username and password to store the username and password of a TestEngine user.

    • Secret text to store the encryption project password.

    Scope

    Select the scope for the new credentials.

    Username
    Password

    Specify the username and password of a TestEngine user.

    Secret text

    Specify the encryption project password.

    ID

    The credentials id.

    You need the id to specify credentials when specifying a TestEngine test step in the pipeline project.

    Leave the field blank to assign a randomly generated id.

    Description

    Specifies the description to help distinguish one credentials instance from another in the selection drop-down list.

  3. Click Add to create credentials.

After that, Jenkins shows the created credentials in the corresponding drop-down list:

ReadyAPI TestEngine plugin: Select credentials

Click the image to enlarge it.

See Also

Administrative Tasks

Highlight search results