Definition
Test execution is managed with test runs. A test run is a snapshot of your project that includes all the tests or just a subset of them. A test run creates a kind of branch of the project where current definitions of tests are stored. It means that you can continue working on your scenarios and modify them without impacting the test run and the execution. Of course your can synchronize your test run at any time with the new definitions of scenarios.
For each test of a test run, you can add one or more test results to keep track of your test execution progress.
Click the image to enlarge it.
Create a test run
When creating a new test run, you can choose between 4 options:
-
include all the tests,
-
select the same tests as a previous test run,
-
select a subset of tests (you can filter by folders and scenarios),
-
create an empty test run that will be used to integrate external test results.
Then click Create button at the bottom of the page.
Click the image to enlarge it.
Once your test run is created you can see the list of tests and metrics.
Click the image to enlarge it.
From this page, you can:
-
navigate between test runs,
-
select a test for execution,
-
select multiple tests for bulk status update,
-
select or unselect all tests in the test run for bulk status update,
-
select or unselect all tests in a folder for bulk status update,
-
show or hide tags of tests and folders,
-
synchronize all the tests with the latest definitions of scenarios,
-
add scenarios to the current test run,
-
export the data into an excel file,
-
archive the test run,
-
automate the test run,
-
reset test results,
-
delete the test run.
If the test run contains many execution cycles, you can see the history by clicking on the Executions history tab:
Click the image to enlarge it.
Assign a test to a user
You can assign a test to a user in the Assignees section. This user should be a project member.
Click the image to enlarge it.
You can search for assigned tests in a test run using the Search tests input by typing the assignee name.
Click the image to enlarge it.
Execute a test manually
When selecting a test, you can add one or more execution results. An execution result contains a description, a date, the author of the execution and the status. The author is automatically updated. You can add a description and attach files to an execution result (screenshot…).
You can also track execution step by step. The first click marks the step as passed, the second one as failed and last one as undefined.
Click the image to enlarge it.
You can use the Passed and next action to mark the test as Passed and pass to the next one:
Click the image to enlarge it.
Below the execution results, you have access to the execution results history:
Click the image to enlarge it.
From the test page you can as well:
-
add tags to your test
-
assign the test to one or many project members
Bulk status update
You can select multiple tests with the check boxes and update their status.
Click the image to enlarge it.
Automated execution
To see how to automate your tests and link CucumberStudio to your continuous integration server see the Automate your tests section.
Test run metrics
At any time you can click on the test run name (left column) to view the metrics. You can also export test runs into an excel file and then customize your metrics.
Click the image to enlarge it.
You can go back to your test runs list by clicking on the test run menu. From this list, you can export the content of all your test runs into a single excel file (one test run by tab), customize your metrics, and other actions on the existing test runs.
Click the image to enlarge it.
Add a new scenario to a test run
You can add a new scenario to a test run directly from the scenario page by selecting Add to test run from the menu.
Click the image to enlarge it.
Or you can add many scenarios from the folder page. Just select your scenarios and choose Add to test run.
Click the image to enlarge it.
Archive test runs
Because your test runs list can grow very quickly, you can hide some of your test runs thanks to the Archive option.
Click the image to enlarge it.
You can find archived test runs in your test runs list by selecting the dedicated filter.
Click the image to enlarge it.
An archived test run will not appear in the test results list of a scenario, and you will not be able to add a scenario in it or watch its metrics on the project dashboard. To unarchive a test run, select Unarchive from the menu.