Bamboo is a continuous-integration, deployment, and delivery tool from Atlassian that integrates incredibly well with Jira.
Zephyr Scale’s Bamboo integration enables you to automate tests outside Jira, run them in Bamboo during the CI/CD pipeline and have results sent back to Zephyr Scale for reporting and tracking.
|
Windows users must download and install cURL and 7-Zip and add both system paths to their machines. |
Here’s what’s covered in this topic:
Before you testBefore you test
Set global variables
To initiate the Zephyr Scale integration with Bamboo, an admin must navigate to Bamboo’s Global Variables and set variable names and values for both jira_user and jira_password.
Click the image to enlarge it.
Download feature files
Prior to executing tests, Cucumber users with gherkin definitions in Zephyr Scale should create a script task and add the following code to enable feature files to be downloaded from Zephyr Scale. When you add the code, replace the PROJECT_KEY and JIRA_URL variables with your actual information. Optionally, you can replace the TARGET_PATH, ZIP_FILE, and AUTO_CREATE_TEST_CASES variables with your information, as well.
Windows
USER=%bamboo_jira_user%
PASSWORD=%bamboo_jira_password%
BUILD_DIR=${bamboo.build.working.directory}
TARGET_PATH="src/test/resources/features"
ZIP_FILE="bddfile.zip"
PROJECT_KEY="WEB"
JIRA_URL="http://localhost:2990"
echo "Clean target path"
rm -rf %BUILD_DIR%/%TARGET_PATH%
mkdir -p %BUILD_DIR%/%TARGET_PATH%
echo "Downloading feature files"
curl -u %USER%:%PASSWORD% %JIRA_URL%/jira/rest/atm/1.0/automation/testcases?tql="testCase.projectKey='%PROJECT_KEY%'" --output %BUILD_DIR%/%TARGET_PATH%/%ZIP_FILE%
echo "Unzipping feature files"
7z e -aoa %BUILD_DIR%/%TARGET_PATH%/%ZIP_FILE% -d %BUILD_DIR%/%TARGET_PATH%
echo "Finished"
Linux
USER=$bamboo_jira_user
PASSWORD=$bamboo_jira_password
BUILD_DIR=${bamboo.build.working.directory}
TARGET_PATH="src/test/resources/features"
ZIP_FILE="bddfile.zip"
JIRA_URL="http://localhost:2990"
PROJECT_KEY="WEB"
echo "Clean target path"
rm -rf $BUILD_DIR/$TARGET_PATH
mkdir -p $BUILD_DIR/$TARGET_PATH
echo "Downloading feature files"
curl -u $USER:$PASSWORD $JIRA_URL/jira/rest/atm/1.0/automation/testcases?tql="testCase.projectKey='$PROJECT_KEY'" --output $BUILD_DIR/$TARGET_PATH/$ZIP_FILE
echo "Unzipping feature files"
unzip $BUILD_DIR/$TARGET_PATH/$ZIP_FILE -d $BUILD_DIR/$TARGET_PATH
echo "Finished"
Send test results from Bamboo to Zephyr Scale
To transfer your test-execution results from Bamboo back to Zephyr Scale, you need to create a final script task in the job where you want to report the test results.
Depending on if you use Cucumber or JUnit, add one of the following code sets, and remember to replace the PROJECT_KEY and JIRA_URL variables with your actual information. Optionally, you can replace the PATH_FILE, ZIP_FILE, and AUTO_CREATE_TEST_CASES variables with your information, as well.
When executed properly, the script runs after the testing concludes. Regardless of whether the tests pass or fail, the build completes, and the test results are published to Zephyr Scale.
If you need help applying the scripts, check out Applying Scripts in Bamboo.
Windows/Cucumber
PASSWORD=%BAMBOO_jira_password%
BUILD_DIR=${bamboo.build.working.directory}
FILE_PATH="target/cucumber/*.json"
ZIP_FILE="test_result.zip"
AUTO_CREATE_TEST_CASES="true"
PROJECT_KEY="WEB"
JIRA_URL="http://localhost:2990"
echo "Removing older zips"
rm -f %BUILD_DIR%/%ZIP_FILE%
echo "Creating new zip file"
7z a %BUILD_DIR%/%ZIP_FILE% %BUILD_DIR%/%FILE_PATH%
echo "Sending zip file to Zephyr Scale"
curl --basic --user %USER%:%PASSWORD% -F "file=@%BUILD_DIR%/%ZIP_FILE%;type=application/x-zip-compressed" %JIRA_URL%/jira/rest/atm/1.0/automation/execution/cucumber/%PROJECT_KEY%?=autoCreateTestCases=%AUTO_CREATE_TEST_CASES% -v
echo "Finished"
Windows/JUnit
PASSWORD=%BAMBOO_jira_password%
BUILD_DIR=${bamboo.build.working.directory}
ZIP_FILE="test_result.zip"
AUTO_CREATE_TEST_CASES="true"
PROJECT_KEY="WEB"
JIRA_URL="http://localhost:2990"
echo "Removing older zips"
rm -f %BUILD_DIR%/%ZIP_FILE%
echo "Creating new zip file"
7z a %BUILD_DIR%/%ZIP_FILE% %BUILD_DIR%/test_result.json
echo "Sending zip file to Zephyr Scale"
curl --basic --user %USER%:%PASSWORD% -F "file=@%BUILD_DIR%/%ZIP_FILE%;type=application/x-zip-compressed" %JIRA_URL%/jira/rest/atm/1.0/automation/execution/%PROJECT_KEY%?autoCreateTestCases=%AUTO_CREATE_TEST_CASES% -v
echo "Finished"
Linux/Cucumber
PASSWORD=$bamboo_jira_password
BUILD_DIR=${bamboo.build.working.directory}
PATH_FILE="target/cucumber/*.json"
ZIP_FILE="test_result.zip"
AUTO_CREATE_TEST_CASES="true"
JIRA_URL="http://localhost:2990"
PROJECT_KEY="WEB"
echo "Removing older zips"
rm -f $BUILD_DIR/$ZIP_FILE
echo "Creating new zip file"
zip $BUILD_DIR/$ZIP_FILE $BUILD_DIR/$PATH_FILE -j
echo "Sending zip file to Zephyr Scale"
curl --basic --user $USER:$PASSWORD -F "file=@$BUILD_DIR/$ZIP_FILE;type=application/x-zip-compressed" $JIRA_URL/jira/rest/atm/1.0/automation/execution/cucumber/$PROJECT_KEY?autoCreateTestCases=$AUTO_CREATE_TEST_CASES -v
echo "Finished"
Linux/JUnit
PASSWORD=$bamboo_jira_password
BUILD_DIR=${bamboo.build.working.directory}
ZIP_FILE="test_result.zip"
AUTO_CREATE_TEST_CASES="true"
PROJECT_KEY="WEB"
JIRA_URL="http://localhost:2990"
echo "Removing older zips"
rm -f $BUILD_DIR/$ZIP_FILE
echo "Sending zip file to Zephyr Scale"
zip $BUILD_DIR/$ZIP_FILE $BUILD_DIR/test_result.json -j
echo "Sending a zip to Zephyr Scale"
curl --basic --user $USER:$PASSWORD -F "file=@$BUILD_DIR/$ZIP_FILE;type=application/x-zip-compressed" $JIRA_URL/jira/rest/atm/1.0/automation/execution/$PROJECT_KEY?autoCreateTestCases=$AUTO_CREATE_TEST_CASES -v
echo "Finished"
Add Zephyr Scale annotations
If you are using Cucumber, setup could not be more simple. Add Zephyr Scale annotations to your Cucumber tests, and use our pre-made script to send the results to Jira. The integration is based on the JSON output files generated by Cucumber, so this output option needs to be configured properly, as follows.
-
Configure the JSON output format for your automated tests on the TestRunner class.
Here we’ve added JUnit and Cucumber output formats:
Java
@RunWith(Cucumber.class)
@CucumberOptions(
features = "src/test/resources/calculatorFeatures"
,glue = {"com/adaptavist/tm4j/cucumberExample/stepDefinition"}
,plugin = {"junit:target/cucumber/result.xml", "json:target/cucumber/calculator.json"}
) -
Annotate your automated tests with the @TestCaseKey tag.
The @TestCase(key = "JQA-T1") annotation maps the test method to an existing test case in Zephyr Scale by matching its key. The @TestCase(name = "Sum Two Numbers") annotation adds a name to the test case. This maps this test method to an existing test case in Zephyr Scale by matching its name. If the test case doesn’t exist in Zephyr Scale, then a new one can be automatically created using this name when the results are uploaded.
Scenarios can be mapped to test cases in Zephyr Scale by using the test-case key or name as a scenario tag. For example:
Cucumber
@TestCaseKey = ABC-T123.
Cucumber test annotations
Cucumber
Feature: Calculator
@TestCaseKey = ABC-T123
Scenario: Add a number
Given a calculator i just turned on
And I add 4 and 4
Then the result is 8
JUnit test annotations
Java
public class CalculatorSumTest {
@Test
@TestCase(key = "JQA-T1")
public void sumTwoNumbersAndPass() {
Calculator calculator = new Calculator();
assertEquals(5, calculator.sum(3, 2));
}
@Test
@TestCase(key = "JQA-T2")
public void sumTwoNumbersAndFail() {
Calculator calculator = new Calculator();
assertEquals(2, calculator.sum(1, 2));
}
@Test
public void notMappedToTestCaseAndPass() {
Calculator calculator = new Calculator();
assertEquals(3, calculator.sum(1, 2));
}
@Test
@TestCase(name = "Mapped to a Test Case Name and Pass")
public void mappedToATestCaseNameAndPass() {
Calculator calculator = new Calculator();
assertEquals(4, calculator.sum(2, 2));
}
}
Now you can run the tests with Maven test, and the Zephyr Scale test execution results file will generate in the same execution folder.