Product Comparison
This section covers the key features comparison between Zephyr Squad and Zephyr Scale.
Test Management
Test Management | Zephyr Squad | Zephyr Scale |
|---|---|---|
Requirement management | Yes (via Jira) | Yes (via Jira) |
Requirement and test case coverage traceability | Yes | Yes |
Test case folder structure | No | Yes |
Test case data | No | Yes |
Test case parameters | No | Yes |
Test steps | Yes | Yes |
Test step wiki formatting | Yes | Yes |
Import test cases | Yes | Yes |
Test case repository | No | Yes |
Test case versioning | No | Yes |
Test case archiving | No | Yes |
Test case history | Yes | Yes |
Test case search | Yes | Yes |
Test case saved filter | Yes | Yes |
Test case bulk creation | Yes | Yes |
Test case bulk edit | Yes | Yes |
Test case clone | Yes | Yes |
Reuse and execute test cases across projects | No | Yes |
Cross-project reporting and dashboard | No | Yes |
Call to test to reuse test steps | No | Yes |
Test Planning and Execution
Test Planning and Execution | Zephyr Squad | Zephyr Scale |
|---|---|---|
Test cycles | Yes | Yes |
Test cycle history | No | Yes |
Test cycle folder structure | Yes | Yes |
N-level folder structure | No | Yes |
Flexible move and copy across cycle folders | Yes | Yes |
Test plans | No | Yes |
Test plan history | No | Yes |
Test Execution | Yes | Yes |
Test Step Results | Yes | Yes |
Test Execution history | Yes | No |
Defect link on Test Execution | Yes | Yes |
Defect link on Test Step Results | Yes | Yes |
Time tracking (during the execution) | Yes | Yes |
Search Test Executions | Yes | No |
Export Test Cycles | CSV, HTML, XML | Excel |
User Management
User Management | Zephyr Squad | Zephyr Scale |
|---|---|---|
Roles and groups | Yes (via Jira) | Yes (via Jira) |
Permission control | Yes (via Jira) | Yes |
Reports and Advanced Analytics
Reports and Advanced Analytics | Zephyr Squad | Zephyr Scale |
|---|---|---|
Built-in reports including traceability | Yes (Up to 5) | Yes (Up to 31) |
Jira dashboard gadgets | Yes (Up to 9) | Yes (Up to 19) |
Test execution details gadget | Yes | No |
Top Defects Impacting Testing gadget | Yes | No |
Executions By Date gadget | Yes | No |
Behavior-Driven Development (BDD)
Behavior-Driven Development (BDD) | Zephyr Squad | Zephyr Scale |
|---|---|---|
Integration with Cucumber/Gherkin | Yes | Yes |
Advanced features (Gherkin validations, tag management) | Yes | No |
Automation
Automation | Zephyr Squad | Zephyr Scale |
|---|---|---|
Supported automation frameworks |
|
|
CI/CD script integration | Bitbucket Pipelines, Team City, CircleCI, GitLab, Azure DevOps, and others | Bitbucket Pipelines, Team City, CircleCI, GitLab, Azure DevOps, and others |
CI/CD native integrations | Jenkins and Bamboo | Jenkins and Bamboo |
SmartBear tools integration | ReadyAPI, TestComplete, CrossBrowserTesting, and LoadNinja. | No |
Rich Set of API’s For Custom Integrations | Yes | Yes |
Import test cases from other tools | Yes | Yes |
Other Features
Other Features | Zephyr Squad | Zephyr Scale |
|---|---|---|
Team-managed (NextGen) project support | Yes | Yes |
Internationalisation support (i18n) | Yes | Yes |
Server to Cloud migration support | Yes | Yes |
GDPR complaint | Yes | Yes |
Cloud Security participant | Yes | Yes |
Cloud Fortified support | No | Yes |
Test Cycle Folder Structure
Zephyr Squad | Zephyr Scale |
|---|---|
In Squad, cycles are organized by releases. Folders can be created within the cycle to allow another depth of an organization. Squad UI allow users to group test cycles in a hierarchical structure grouped by releases (or unreleased cycles - cycles not tied to any release). Test cases can be added to the Cycle in a flat list or users can create a folder structure within the cycle to create a new level of hierarchy to organize the test cases. | Scale UI allow customers to group cycles into hierarchical folders. There is not tie to releases, the folders are just folders. Test cases can be added to the Cycle in a flat list. There is no folder structure to organize test cases within the Cycle. |
Test Execution History
Zephyr Squad | Zephyr Scale |
|---|---|
The Test Execution History is logged for the following operations on test executions:
| Nothing in Scale for the test execution. However, all other changes are tracked in the Test Cycle. We could use the test cycle history engine to log any changes during the execution. In the execution screen, we could add a new section to display only the entries related to that execution. |
Time Tracking (During the Execution)
Zephyr Squad | Zephyr Scale |
|---|---|
It offers to disable the option. | • It provides a timer • There is no workflow in Scale, but in progress is a default status for test execution • Scale doesn’t allow to enter time in a natural way (ex: 2h) • Scale doesn’t log the time tracked in the test execution history |
Search Test Executions
Zephyr Squad | Zephyr Scale |
|---|---|
The Search Test Executions page allows searching for the test executions present in the application. It has a few different options to allow the selection of search criteria. • Basic Search Basic search has different fields added as dropdowns where the users can select the field values. The fields that are present in the Basic Search view are as follows: ▪ Project ▪ Fix Version ▪ Cycle Name ▪ Folder Name ▪ Execution Workflow Status ▪ Execution Status ▪ Executed By ▪ Executed On ▪ Priority ▪ Assigned To ▪ Creation Date ▪ Component • Advanced Search Advanced search uses the ZQL(Zephyr Query Language) which is similar to JQL(Jira Query Language). • Search filters Users can save the selected search criteria as a saved search filter. • Pre-defined filters In addition to using the saved search filters, users can use the already-defined filters from the left panel to perform a quick search. | Nothing in Scale for search test execution. |
Export Test Cycles
Zephyr Squad | Zephyr Scale |
|---|---|
UI allows to export test cycles: 1. from two different pages: • Cycle Summary • Search Test Executions 2. The supported export options are: CSV, HTML, XML 3. Cycle Summary Export Body On export, the executions are exported with the respective cycle/folder information which includes: • Test Execution information ▪ ExecutionId ▪ Cycle ▪ Folder Name ▪ Executed By ▪ Executed On ▪ Execution Status ▪ Execution Defect(s) ▪ Creation Date ▪ Assigned To ▪ Execution Custom Fields ▪ Work Flow Status ▪ Total Execution Time ▪ Total Logged Time • Test Case information ▪ Issue Key ▪ Test Summary ▪ Project ▪ Component ▪ Version ▪ Priority • Test Steps information ▪ StepId ▪ OrderId ▪ Step ▪ Test Data ▪ Expected Result ▪ Step Result ▪ Comment ▪ Test Step Custom Field 4. Search Test Execution On export, the executions are exported with the respective cycle/folder information which includes the same fields as described above for Cycle Summary. | The UI allows users to export test cycles: • from reports that can be exported. • The supported export options are: Excel • The exported file includes a list of execution with: ▪ the corresponding test execution information (including custom fields) ▪ cycle information (including custom fields) for each of the execution ▪ test case information (including custom fields) for each execution |
Built-in Reports Including Traceability
Zephyr Squad | Zephyr Scale |
|---|---|
5 reports available for the users Squad reports can be reproduced with the use of filters for the reports in Scale. | 31 reports available for the users. Scale does not have an exact analog of the Squad reports |
Top Defect Report
The Top Defects report does not have an exact equivalent in Scale. With a bit of set up using filters, it could be reproduced using the Test execution results by coverage report.
The metric provides information about which defects are currently holding up the maximum number of tests from passing.
An option to select 5/10/15 maximum defects is provided, and users can select the statuses of defects they want to see in the chart.
Test Execution Chart
Test Execution Chart does not have an exact equivalent in Scale. A similar graph can be generated by Test execution results by test cycle or Test execution results by component or Test execution results by tester. The Test Execution widget displays totals of tests that have been executed for a particular project-fixVersion combination. This can be customized to show the data grouped either by Test Cycles, Folders, Components or Users.
Test execution details widget
The Test execution details widget have no equivalent in Scale. This gadget fetches data based on a saved ZQL search (i.e. a test execution filter) or a free-form ZQL query.
The following image shows Test Execution Details with test execution filter.
The following image shows Test Execution Details page and the search options.
Top Defects Impacting Testing widget:
The Top Defects Impacting Testing widget have no equivalent in Scale. This metric provides information about which defects are currently holding up the maximum number of tests from passing. An option to select 5/10/15 maximum defects is provided, and users can select the statuses of defects they want to see in the chart as seen in the following image.
The following image shows the defect details:
Executions By Date widget:
The Executions By Date page has no equivalent in Scale. The information is available in several reports in Scale and in a single page in Squad This metric provides information on the total test executions created for the selected fixVersion and the time interval applied.
Integration with Cucumber/Gherkin:
The main differences are as follows:
No feature editor in Scale.
No tagging capabilities
Scale generates one feature file per test cases when exporting feature file while Squad creates feature file including all the test cases since the feature concept exist.
Test case result view is better in Scale compared to Squad with more details and errors message being included.
In Scale, the results file can only be uploaded using the automation endpoint for cucumber.
Supported Automation Frameworks
The main differences are as follows:
No UI to create an automation task in Scale.
Some frameworks supported by Squad are only supported by Scale if the users transform their results in a Junit format.
Squad is allowing more in terms of Junit XML results attributes size.
CI/CD Script Integration:
The main differences are as follows:
• In Squad, the support on CI/CD tools is provided via the post-build actions. To do this, users can create a script file that works with Zephyr Squad through API.
In Scale, the support on CI/CD tools is supported in Zephyr Scale using the REST API or our Jenkins Plugin or Bamboo plugin via the post-build actions. There are 3 endpoints supporting Cucumber, JUnit 5, and JUnit 4 file formats.
SmartBear Tools Integration:
The main differences are as follows:
Zephyr Squad provides integration with the following tools of SmartBear: ▪ TestComplete ▪ ReadyAPI ▪ LoadNinja ▪ CrossBrowserTesting
No integration in Scale. Only capabilities to upload results using the Scale REST API.
Rich Set of API’s For Custom Integrations:
The APIs for Zephyr Squad and Zephyr Scale are not similar although they cover the same functionalities. Customers must review them before using it.
Zephyr Squad - Zephyr Squad Cloud REST API (formerly Zephyr for Jira) · Apiary
Zephyr Scale - Zephyr Scale for Jira Cloud API