Robot Framework Integration

Robot Framework is an open-source automation framework that can be used for test automation. The integration with Zephyr Scale works through uploading a JUnit XML results file generated by the framework to Zephyr Scale. Once the file is uploaded to Zephyr Scale, it is possible to generate reports and leverage all the other capabilities that Zephyr Scale offers.

For more info on Robot framework, see here.

Writing Robot Framework tests

See below for an example of how to write test cases with Robot Framework and Zephyr Scale.

*** Test Cases ***
# will match Zephyr Scale test case named Calculator.User can clear the display
User can clear the display
    Input number    10
    Press operator    +
    Input number    1
    Press clear 
    Display should be empty

*** Test Cases ***
# will match Zephyr Scale test case with key NET-T1743
NET-T1744 User can calculate with wrong result 
    Input number    1
    Press operator    +
    Input number    1
    Press enter 
    Result should be     3

*** Test Cases ***
# will match Zephyr Scale test case with key NET-T1744
User can calculate two numbers - NET-T1744
    [Template]    Calculate two numbers should pass
    10    +    5    15
    10    -    5    5
    10    /    5    2
    10    *    5    50

Note the comments above each method describing how they map to existing test cases within Zephyr Scale. This varies depending on whether a test case key or name is provided, as shown above.

If any test case, prefixed with the test case key or not, is not found, an error will be returned when the results are uploaded to Zephyr Scale. In this case, if you intend to automatically create test cases that don’t exist previously, a flag autoCreateTestCases=true can be set when uploading test results to Zephyr Scale.

To summarize, the rules for matching a test method to a test case are:

Try to match by parsing a test case key from the test method name.
If no test case is matched, then try to match a test case by using the test name following the pattern <robot filename with no extension>.<robot test case name>.
If no test case is matched, then:
- If the parameter autoCreateTestCases is true, create the test case and create a new test execution for it.
- If the parameter autoCreateTestCases is false or not present, return an error - no test cases have been matched.

Configuring Robot Framework to output JUnit XML results file

No configuration is required beforehand. In order to instruct Robot Framework to generate the JUnit XML results file, all that is required is to execute the tests with -x parameter followed by the xml file name. Here is an example:

robot -x junitresult.xml mytest.robot

Uploading results to Zephyr Scale

Once the Robot Framework tests have executed, the results can be uploaded to Zephyr Scale. There are two options for uploading test results:

Using a single JUnit XML results file generated by Robot Framework.
Using a ZIP file containing multiple JUnit XML results file generated by Robot Framework.

Both options work the same way and can be uploaded using the same API endpoint: https://support.smartbear.com/zephyr-scale-cloud/api-docs/#operation/createJUnitExecutions.

Below, an example using curl of how to use the API endpoint for uploading one single file:

curl -H "Authorization: Bearer ${TOKEN}" -F "[email protected];type=application/xml" https://api.zephyrscale.smartbear.com/v2/automations/executions/junit?projectKey="JQA"&autoCreateTestCases=true

…and for uploading a zip file containing multiple XML files:

curl -H "Authorization: Bearer ${TOKEN}" -F "[email protected];type=application/x-zip-compressed" https://api.zephyrscale.smartbear.com/v2/automations/executions/junit?projectKey="JQA"&autoCreateTestCases=true

Note the query parameters on the URL. The projectKey specifies what project will be used to create the test cycle. The autoCreateTestCase will create a test case using the Robot Framework test method name when no matching test case is found.

Code samples

https://github.com/SmartBear/zephyr-scale-robot-framework-example

Suggested Improvements

Is something missing? If you’d like to suggest an improvement to this integration, take a look at our Ideas Portal.

You can search for existing ideas or create your own. We review ideas regularly and will take them into account for future development.

In this section: