Applies to BitBar SaaS, last modified on August 11, 2020

BitBar provides detailed results of each test run, as well as device performance date. You can view and analyze the results in several places:

Test run summary

You can access the Test Run summary by doing either of the following:

  • Start a test run.

  • Switch to the Projects tab, expand a project, and click any test run.

This view shows test run execution information, execution time, as well as a summary of test runs.

Test Run overview

Click the image to enlarge it.

In addition to summary information, you can download the application and test files, as well as all log files. To do this, click next to the needed files in the Input Files and Output Files sections.

Test run details

To get more infotmation about the test run and compare the test results by devices, click Go to results at the bottom of the test run summary view.

Opening the device session details

Click the image to enlarge it.

Device Session Browser

The Device Session Browser widget, shows the devices used in the test run. Each device shows the current device status: waiting, running, succeeded, warning, or failed. Additionally, the device icon shows the test method success rate for that device. Hover over the icons to get more information.

Device Session Browser

Control Panel

The Control Panel widget provides details about the test run.

Here, you can run the test on the selected device again. To do this, click  > Retry device.

BitBar records you test, so you can later replay the test run recording and view the log. To view the recording and log, click Timeline in the Control Panel widget.

Timeline

Click the image to enlarge it.

The Control Panel widget shows test run detail on the following tabs:

The Analytics tab shows important data about how the test run succeeded and how the selected device performed with respect to other devices in this run.

Analytics tab

Click the image to enlarge it.

If your test contains test steps or test functions, these are presented in order of execution. The Tests and Steps tab shows the order in which tests were executed and whether the functions executed successfully or no.

Tests and Steps tab

Click the image to enlarge it.

Also, you can compare the test case runs among the devices used in the test run.

Test case runs comparison

Click the image to enlarge it.

You can also view the device screenshots taken at certain test steps:

Viewing a screenshot

Click the image to enlarge it.

After the test execution we run some statistics against the devices in the test run. The Issues tab highlights some execution issues in this tab that we believe are interesting to you.

Issues tab

Click the image to enlarge it.

The Output Files tab contains all generated and captured files from the test execution. You can control which files get saved and picked to be presented here by configuring your run-tests.sh. Screenshots and video files are visible in the Media widget, while log files are also viewable here below with search and highlight support.

  •  – Click to view the file.

  •  – Click to download the file.

  •  – Click to open the file in a new tab.

Output Files tab

Click the image to enlarge it.

Media

The Media widget shows all captured screenshots and video during the selected test. Screenshots are presented in the order of capturing. The media files are downloadable.

Media widget

Click the image to enlarge it.

Logs

The logs widget provides line-by-line information about the test run. It allows showing the different types of log data available from the test run. These include logcat, Appium logs. The logs can be searched by browser search or the widget’s own search box. For easier debugging the view can be enlarged to full screen.

Logs widget

Click the image to enlarge it.

Performance

The Performance widget provides details of the test run for CPU and memory usage. You can click any given step/time to get more specific information about resource consumption.

Performance widget

Click the image to enlarge it.

See Also

BitBar Documentation

Highlight search results