Integration With Test Management Systems (TMS)

Applies to TestComplete 14.71, last modified on April 22, 2021

Test Management Systems (TMS) are software tools that help you manage and monitor your testing process. You can create and store test cases there, organize them into test runs, and analyze the results of their executions.

Supported TMS

TestComplete can integrate with almost any test management system. TestComplete has built-in integrations with the following test management systems:

  • Zephyr for Jira – You can automate your test cases by assigning TestComplete tests to them and send the test results to Jira.

  • Azure Test Plans – You can run TestComplete tests as part of your build and release pipeline or use TestComplete tests to automate manual test cases you manage with test plans.

  • QAComplete – You can export your TestComplete tests to the QAComplete Test Library and submit defects to QAComplete directly from TestComplete.

Integration with Zephyr Enterprise is realized on its side. For complete information, see the Zephyr Enterprise documentation.

Other TMS

To integrate with other test management systems, use the OnStopTestCase event handler. You can configure the event handler to send information about test cases that TestComplete runs to your test management system.

Requirements

Your TMS system must have a way to submit results to a specific test case, for example, using an API call.

1. Mark desired test items as test cases

The OnStopTestCase event handler fires when a test case is over, that's why mark test items as test cases using the following approaches:

Note: BDD scenarios and features, and tests specified via tags are always treated as test cases. You do not need to mark them manually.

For more information about test cases, see Tests, Test Items, and Test Cases.

2. Add the OnStopTestCase event handler

  1. Right-click your project node in the Project Explorer panel, select Add > New Item in the subsequent dialogs. In the Create Project Item dialog, select Events and then click OK:

    Adding the Events project item

    Click the image to enlarge it.

    TestComplete will add the Events project item to your project.

  2. Right-click the added Events item and add the Event control in a similar way:

    Adding the Event control

    Click the image to enlarge it.

    After you click OK, TestComplete will open the Event Control Editor.

  3. Expand the Test Engine Events section of the Available Events list and drag the OnStopTestCase item to the Events to Handle list on the right. Select the added item and click New next to it. TestComplete will display the New Event Handler dialog:

    Adding the Event control

    Click the image to enlarge it.

  4. In the dialog, specify either the name of the event-handling routine and the unit that will hold this routine or the name of the event handling a keyword test. In our example, we will add the event handler to the script unit. To learn how you can use event handlers in keyword tests, see Creating Event Handlers for TestComplete Events.

    After you click OK, TestComplete will add the OnStopTestCase event handler to your script test:

    Adding the OnStopTestCase event handler

    Click the image to enlarge it.

3. Write the event handler

The StopTestCaseParams parameter of the OnStopTestCase event handler provides you with information about the executed test case (for example, the name of the test case, its status). For more information, see Properties of the StopTestCaseParams object.

Here is an example of how to send test case results to TestRail:

JavaScript, JScript
Python
VBScript
DelphiScript
C++Script, C#Script

JavaScript, JScript

// Before using this example, enable TestRails API
// in TestRails > Administration > Site Settings > API

function EventControl1_OnStopTestCase(Sender, StopTestCaseParams)
{
  var testId = 1; // TestRail ID of the test to add the result to
  var address = "https://mysite.testrail.io/index.php?/api/v2/add_result/" + testId;
  var username = "[email protected]";
  var password = "p@55w0rd";
  // Convert the user credentials to base64 for preemptive authentication
  var credentials = aqConvert.VarToStr(dotNET.System.Convert.ToBase64String
    (dotNET.System_Text.Encoding.UTF8.GetBytes_2(username + ":" + password)));

  var request = aqHttp.CreatePostRequest(address);
  request.SetHeader("Authorization", "Basic " + credentials);
  request.SetHeader("Content-Type", "application/json");

  var statusId;
  var comment = "";
  switch (StopTestCaseParams.Status)
  {
    case 0: // lsOk
      statusId = 1; // Passed
      break;
    case 1: // lsWarning
      statusId = 1; // Passed with a warning
      comment = StopTestCaseParams.FirstWarningMessage;
      break;
    case 2: // lsError
      statusId = 5; // Failed
      comment = StopTestCaseParams.FirstErrorMessage;
      break;
  }

  var requestBody =
  {
    "status_id": statusId,
    "comment": comment
  };
  var response = request.Send(JSON.stringify(requestBody));

  if (response.StatusCode != aqHttpStatusCode.OK)
  {
    Log.Warning("Failed to send results to TestRail. See the Details in the previous message.");
  }
}

Python

# Before using this example, enable TestRails API
# in TestRails > Administration > Site Settings > API

import json
import base64

def EventControl1_OnStopTestCase(Sender, StopTestCaseParams):
    testId = 1 # TestRail ID of the test to add the result to
    address = "https://mysite.testrail.io/index.php?/api/v2/add_result/" + str(testId)
    username = "[email protected]"
    password = "p@55w0rd"
    # Convert the user credentials to base64 for preemptive authentication
    credentials = base64.b64encode((username + ":" + password).encode("ascii")).decode("ascii")

    request = aqHttp.CreatePostRequest(address)
    request.SetHeader("Authorization", "Basic " + credentials)
    request.SetHeader("Content-Type", "application/json")

    comment = ""
    if StopTestCaseParams.Status == 0: # lsOk
        statusId = 1 # Passed
    elif StopTestCaseParams.Status == 1: # lsWarning
        statusId = 1 # Passed with a warning
        comment = StopTestCaseParams.FirstWarningMessage
    elif StopTestCaseParams.Status == 2: # lsError
        statusId = 5 # Failed
        comment = StopTestCaseParams.FirstErrorMessage

    requestBody = {
        "status_id": statusId,
        "comment": comment
    }
    response = request.Send(json.dumps(requestBody))

    if response.StatusCode != aqHttpStatusCode.OK:
      Log.Warning("Failed to send results to TestRail. See the Details in the previous message.")

VBScript

' Before using this example, enable TestRails API
' in TestRails > Administration > Site Settings > API

Sub EventControl1_OnStopTestCase(Sender, StopTestCaseParams)
  Dim testId, address, username, password, credentials, request, statusId, comment, requestBody, response
  testId = 1 ' TestRail ID of the test to add the result to
  address = "https://mysite.testrail.io/index.php?/api/v2/add_result/" & testId
  username = "[email protected]"
  password = "p@55w0rd"
  comment = ""
  ' Convert the user credentials to base64 for preemptive authentication
  credentials = aqConvert.VarToStr(dotNET.System.Convert.ToBase64String _
    (dotNET.System_Text.Encoding.UTF8.GetBytes_2(username + ":" + password)))

  Set request = aqHttp.CreatePostRequest(address)
  Call request.SetHeader("Authorization", "Basic " + credentials)
  Call request.SetHeader("Content-Type", "application/json")

  Select Case StopTestCaseParams.Status
    Case 0 ' lsOk
      statusId = 1 ' Passed
    Case 1 ' lsWarning
      statusId = 1 ' Passed with a warning
      comment = StopTestCaseParams.FirstWarningMessage
    Case 2 ' lsError
      statusId = 5 ' Failed
      comment = StopTestCaseParams.FirstErrorMessage
 End Select

  requestBody = "{""status_id"": """ & statusId & """, ""comment"": """ & comment & """}"
  Set response = request.Send(requestBody)

  If Not response.StatusCode = aqHttpStatusCode.OK Then
    Log.Warning("Failed to send results to TestRail. See the Details in the previous message.")
  End If
End Sub

DelphiScript

// Before using this example, enable TestRails API
// in TestRails > Administration > Site Settings > API

procedure EventControl1_OnStopTestCase(Sender, StopTestCaseParams);
var testId, address, username, password, credentials, request, statusId, comment, requestBody, response;
begin
  testId := 1; // TestRail ID of the test to add the result to
  address := 'https://mysite.testrail.io/index.php?/api/v2/add_result/' + VarToStr(testId);
  username := '[email protected]';
  password := 'p@55w0rd';
  // Convert the user credentials to base64 for preemptive authentication
  credentials := aqConvert.VarToStr(dotNET.System.Convert.ToBase64String
    (dotNET.System_Text.Encoding.UTF8.GetBytes_2(username + ':' + password)));

  request := aqHttp.CreatePostRequest(address);
  request.SetHeader('Authorization', 'Basic ' + credentials);
  request.SetHeader('Content-Type', 'application/json');

  comment := '';
  case StopTestCaseParams.Status of
    0: // lsOk
      statusId := '1'; // Passed
    1: // lsWarning
    begin
      statusId := '1'; // Passed with a warning
      comment := StopTestCaseParams.FirstWarningMessage;
    end;
    2: // lsError
    begin
      statusId := '5'; // Failed
      comment := StopTestCaseParams.FirstErrorMessage;
    end;
  end;
  
  requestBody := '{"status_id": "' + statusId + '","comment": "' + comment + '"}';
  response := request.Send(requestBody);

  if response.StatusCode <> aqHttpStatusCode.OK then
  begin
    Log.Warning('Failed to send results to TestRail. See the Details in the previous message.');
  end;
end;

C++Script, C#Script

// Before using this example, enable TestRails API
// in TestRails > Administration > Site Settings > API

function EventControl1_OnStopTestCase(Sender, StopTestCaseParams)
{
  var testId = 1; // TestRail ID of the test to add the result to
  var address = "https://mysite.testrail.io/index.php?/api/v2/add_result/" + testId;
  var username = "[email protected]";
  var password = "p@55w0rd";
  // Convert the user credentials to base64 for preemptive authentication
  var credentials = aqConvert["VarToStr"](dotNET["System"]["Convert"]["ToBase64String"]
    (dotNET["System_Text"]["Encoding"]["UTF8"]["GetBytes_2"](username + ":" + password)));

  var request = aqHttp["CreatePostRequest"](address);
  request["SetHeader"]("Authorization", "Basic " + credentials);
  request["SetHeader"]("Content-Type", "application/json");

  var statusId;
  var comment = "";
  switch (StopTestCaseParams["Status"])
  {
    case 0: // lsOk
      statusId = 1; // Passed
      break;
    case 1: // lsWarning
      statusId = 1; // Passed with a warning
      comment = StopTestCaseParams["FirstWarningMessage"];
      break;
    case 2: // lsError
      statusId = 5; // Failed
      comment = StopTestCaseParams["FirstErrorMessage"];
      break;
  }

  var requestBody =
  {
    "status_id": statusId,
    "comment": comment
  };
  var response = request["Send"](JSON["stringify"](requestBody));

  if (response["StatusCode"] != aqHttpStatusCode["OK"])
  {
    Log["Warning"]("Failed to send results to TestRail. See the Details in the previous message.");
  }
}

4. Run test cases and view results

Run your tests. For complete information about running tests in TestComplete, see Running, Rerunning, Pausing, and Stopping Automated Tests.

When the test case execution is over, its results are sent to your system. To view the results, switch to your test management system.

See Also

Integration Into QA Process
OnStopTestCase Event
Integration With Zephyr for Jira
Integration With QAComplete

Highlight search results