Analyzing Results
BitBar provides detailed results of each test run as well as device performance data. You can view and analyze the results in several places:
In the test run summary view
In the test run details by devices
Test run summary
You can access the Test Run summary by doing either of the following:
Start a test run.
Switch to the Projects tab, expand a project, and click any test run.
This view shows test run execution information, execution time, as well as the summary of test runs.
 |
In addition to summary information, you can download the application and test files as well as all log files. To do this, click next to the needed files in the Input Files and Output Files sections.
Test run details
To get more information about the test run and compare the test results by devices, click Go to results at the bottom of the test run summary view.
 |
Device Session Browser
The Device Session Browser widget shows the devices used in the test run. Each device shows the current device status: waiting, running, succeeded, warning, or failed. Additionally, the device icon shows the test method success rate for that device. Hover over the icons to get more information.
 |
Control Panel
The Control Panel widget provides details about the test run.
Here, you can run the test on the selected device again. To do this, click > Retry device.
BitBar records your test, so you can later replay the test run recording and view the log. To view the recording and log, click Timeline in the Control Panel widget.
 |
The Control Panel widget shows the test run details on the following tabs:
Analytics
The Analytics tab shows important data about how the test run succeeded and how the selected device performed with respect to other devices in this run.
 |
Tests and Steps
If your test contains test steps or test functions, these are presented in order of execution. The Tests and Steps tab shows the order in which tests were executed and whether the functions executed successfully or not.
 |
Also, you can compare the test case runs among the devices used in the test run.
 |
You can also view the device screenshots taken at certain test steps:
 |
Issues
After the test execution, we run some statistics against the devices in the test run. The Issues tab highlights some execution issues in this tab that we believe are interesting to you.
 |
Output Files
The Output Files tab contains all generated and captured files from the test execution. You can control which files get saved and picked to be presented here by configuring your run-tests.sh. Screenshots and video files are visible in the Media widget, while log files are also viewable here below with search and highlight support.
– Click to view the file.
– Click to download the file.
– Click to open the file in a new tab.
 |
Media
The Media widget shows all captured screenshots and video during the selected test. Screenshots are presented in the order of capturing. The media files are downloadable.
 |
Logs
The logs widget provides line-by-line information about the test run. It allows showing the different types of log data available from the test run. These include logcat, Appium logs. The logs can be searched by browser search or the widget’s own search box. For easier debugging the view can be enlarged to full screen.
 |
Performance
The Performance widget provides details of the test run for CPU and memory usage. You can click any given step/time to get more specific information about resource consumption.
 |