Last modified on October 21, 2020

Release 2.108 (Oct 21st, 2020)

Improvements

  • The Projects screen is now updated automatically on changes made to the data it displays. No need to refresh the page manually anymore.

  • The API Key setting is now masked by default. To view it, click the new button next to it:

    Changes on the API settings

    Click this button quickly to preview the key, and click and hold it for a longer time to make the key visible permanently.

Bug fixes

  • Logs from iOS devices didn’t have highlighting.

  • You can export the “device usage” and “OS usage” statistics to a file. The resulting file didn’t contain all the records the Dashboard displayed.

  • Buttons on the Dashboard were misaligned on some screen resolutions.

  • We’ve also fixed a few typos in the UI.

Release 2.107 (Oct 7th, 2020)

  • In this minor release, we have fixed the error that occurred when deleting users and resources from access groups.

  • Also, we’ve updated internal algorithms and infrastructure to improve performance.

New features

  • Non-admin users can now get all device sessions that are in the running state with the state_eq_running filter parameter:

    curl -u <api-key>: https://cloud.bitbar.com/api/v2/users/{userId}/device-sessions?filter=state_eq_running

    See Get running device sessions for details.

  • Users can now get the maximum available test timeout period for a test run by sending the GET request to either endpoint:

    • curl -X GET https://cloud.bitbar.com/api/v2/users/{uerId}/runs/config

    • curl -X GET https://cloud.bitbar.com/api/v2/runs/config

    This will return the maxTestTimeout objects with the unit and value fields. See Get max available test run timeout.

  • The Live Testing view now shows only devices that support manual testing, so we removed the Supported Frameworks filter.

Bug fixes

  • Removed the possibility of opening output zip files in a new tab in the Control Panel widget as it duplicated the Download option.

  • Switching to the Live Testing view with a test run in the running state caused the Reconnect modal saying You have active connection to <device_name>. Do you want to reconnect? (otherwise device session will be released).

  • In the Admin mode, renaming a cluster caused creating a new one with the old name.

  • Misalignment of elements in the Device Instances view of the Admin mode has been fixed.

  • Improved error messages that appeared when a user logged in as an admin tried to make changes to other users, such as convert a user to a sub-user, add a plan to a user, disable a user.

  • Some redundant data was sent in POST requests when making changes in the My Account > My Preferences > Test Preferences dialog.

  • API level for iOS devices always showed 0 in the Live Testing > Past sessions > Control Panel widget.

In this release, we mostly worked on back-end improvements and bug fixes.

  • Fixed: BitBar Swagger UI could not execute the POST and DELETE requests.

  • Fixed: In Admin mode, the Test Timeout option accepted non-digit characters.

  • Fixed: Devices that user A was using in their test run were in the waiting state and not accessible for user B. This issue applied only to users with the same priority level.

  • Fixed: Setting None for Test time-out period caused unexpected behavior when creating automated tests.

New features

  • Admin users can now limit accepted file types to be uploaded per account via the UI. This gives account owners an opportunity to deny the possibility of uploading potentially harmful file types. This is especially interesting for private and enterprise device clouds where strict company policies need to be followed.

    Manage file permission

    Click the image to enlarge it.

Bug fixes

  • Minor UI fixes:

    • The Browse Device Models widget showed a number of devices in double brackets.

    • Too long file names overlapped the UI in the Projects > Test Run view.

  • The link to Swagger on the Fork the Code panel was outdated.

  • The BitBar left-side panel now shows the number of ongoing and available device sessions that can be run concurrently. See Parallel sessions for details.

    Parallel sessions
  • You can also use the /accounts/{accountId}/concurrency-status to get details about concurrent device sessions. See Get device concurrency slots for details.

New features

  • The name_notlike filter parameter now applies to filtering devices. You can use it to exclude the specified device filter group from the query result. For details, see Exclude a device filter group.

  • Admin users can now limit accepted file types to be uploaded per account. For teams or users with multiple sub-users, it is possible to limit which file types are allowed to be used when uploading files to BitBar. This gives account owners an opportunity to deny possibility to upload potentially harmful file types. This is especially interesting for private and enterprise device clouds where strict company policies need to be followed.

Bug fixes

  • When choosing devices for a test session from Projects > Add new test run > Choose devices, the selected devices were not highlighted.

  • The BitBar Demo project showed extra items: Add new test run, Retry, and Remove

In this minor release, we added the UI for setting a test timeout for dedicated devices in the Admin mode. See Dedicated device.

Test timeout settings

Click the image to enlarge it.

New features

  • New BitBar API timeout parameter. Use the testTimeLimit parameter to customize the timeout for a dedicated device, not just a global user timeout. For details, see Set the device timeout.

  • New users now have to finish the registration within 7 days, otherwise they will be automatically removed.

  • The Dashboard now shows test run performance by device. The data is shown in the Device Usage Statistics and Device OS Usage Statistics widgets.

    Device usage statistics

    Click the image to enlarge it.

    Device OS usage statistics

    Click the image to enlarge it.

Bug fixes

  • Issues with sorting and loading screenshots.

  • The Performance graph in Past sessions was not always properly displayed.

In this small release, we moved the My Subscriptions widget to the Billing page. On the  > Subscriptions page, you can change your subscription plan. For details, see Plans and Billing.

In this release, we mostly worked on back-end improvements and bug fixes.

  • Fixed: api/v2/admin/users/user_id/account/roles?limit=0&sort=name_a returned 500 Internal Server Error when the user had no roles assigned.

  • Fixed: Uploading files by specifying only a file name without extension assigned the .ipa extension to the file regardless of its original extension.

  • Fixed: Issues with Xcode 11.5 devices in Live Testing.

  • New Live Testing view:

    • The Screenshots and Device Log tabs were moved to the panel on the left.

    • The action box with live testing tools is now floating and has buttons to emulate the following Android controls:

      •  – Returns to the device's previous screen.

      •  – Opens the device settings.

      •  – Opens the app switcher.

    • The Actions panel shows all actions that you perform over a device during a live testing session.

    • The Files panel allows you to upload an application and additional files to a device.

      The Live Testing view

      Click the image to enlarge it.

  • After finishing a live test session, you are automatically moved to the Past Sessions view.

  • All actions performed over a device are now stored in the Media widget of the Past Sessions view. You can also download them as a JSON file from the Control panel widget.

  • New BitBar API exclusion filter. Use the name_notlike filter parameter for all output file resources to exclude the specified file from the query result. For details, see Filter device session files.

Bug fix

  • An internal server error occurred when uploading an .apk file.

  • BitBar Cloud introduces support for iOS 13.5.

  • Updated device tiles on the Devices screen. They now show display resolution and size and have buttons to quickly start a test using the selected device:

    Device tile

    Click the image to enlarge it.

  • The Admin panel now has the Files view which shows all files used in tests and projects and allows to download and mark the files as save or infected, as well as shows a file's state.

    The Files view

    Click the image to enlarge it.

Bug fix

  • The Forgot my password option was unavailable for some users.

  • Changes to the API. The device API provides now additional information on what testing methods are supported by each device. The testing types supported by BitBar device cloud include MANUAL, ROBOT and AUTOMATIC, which relate to live testing, AI Testbot, and automated testing. This information is especially useful when picking a device over API for automated or AI Testbot testing purposes. The user can avoid picking a device that does not support the wanted testing method.

  • Security fixes and improvements. There was an issue where a malicious user could have used our service to annoy a third-party email inbox. This is not possible anymore after this update.

  • We continue working on backend changes to allow further integration to other SmartBear services and tools. More on these later this year when we get these integrations out to our users!

This release is a small one that brings bug fixes and minor improvements to BitBar:

  • The Control Panel widget of the Past sessions screen has been improved to show essential device information.

    Past sessions: device info

    Click the image to enlarge it.

  • The value of the stateChengeTime property of the GET /api/v2/admin/devices operation is now passed to API.

  • Users were unable to rename projects.

  • Admins could not create new frameworks, and then save them.

  • The demo project was visible after the first project was created by a user.

  • Various minor UI fixes.

The highlights of this release are as follows.

Updates

  • New Projects view. It is now more informative and interactive:

    • The Last Test Runs panel now shows when a test run started, how many devices were used, and the test success ratio.

    • You can now view a test run summary by clicking anywhere in a test run row.

    • The test run summary view has also changed. See the image below for details.

    • The Projects panel has new Create new project and Add new test run buttons.

    Projects view

    Click the image to enlarge it.

  • You can now easily copy your test run configuration to later use it in BitBar API. To do this, expand a test run and click Show configuration.

    Viewing test run configuration

    Click the image to enlarge it.

  • Past manual sessions are now named after the device used in the session.

  • Email notifications now do not have user names in their text.

Bug fixes

  • Non-admin users can now get to the device session results.

  • The Dashboard now shows the shared project data for non-owners.

  • Performance improvements. We moved from AMD to Common JS, so the BitBar UI is now faster and more responsive.

  • The Dashboard now shows the results of shared projects along with the results of your own projects.

    Shared project results

    Click the image to enlarge it.

  • New Device Session view. The view shows live testing details, such as the device used during the session, files uploaded to the device, screenshots and a video recording of the session, device performance graph, and other details.

  • We strengthened up security. You can now set how long your BitBar session will last before you will be logged out. You can set the session timeout in My Account > Security > Change session timeout. The shorter your session is, the greater the security will be.

    Session timeout

    Click the image to enlarge it.

We moved from our own hosted Jira instance to cloud-hosted one during this release. We were not able to use any automated sync solution so the move was fun! No really, we’re still searching two instances for stories and related epics but already getting to work on our service again.

As part of SmartBear we don’t have to do everything on our own anymore. Instead of having to maintain and further develop our browser testing functionality, we can rely on Smartbear CrossBrowserTesting (CBT) to do so. For this reason, we added some links to CBT from our service. Go check them out for spectacular browser testing features.

BitBar + CBT

Click the image to enlarge it.

Since the acquisition of BitBar by SmartBear we have worked on integrating our service to Smartbear. This includes a lot of visual changes but also many changes on the backend that are not visible to you, the end-users.

Here are some highlights of what this release included.

  • Support for multiple Chromedriver versions. This is important for whoever does Appium browser or hybrid application testing on our devices. Now the latest Chromedriver is automatically setup for every device and version of Chrome.

  • We had to rewrite how we extract results from iOS test runs! XCode11 changed where and how test results are stored to which we had to adapt. Instead of directories with subdirectories and separate files, everything is now stored to one file.

  • Improvements to monitoring our internal servers. These include private cloud and public cloud servers on which you are running your tests.

  • We moved to the new Projects view for good now. We have received some improvement requests on the view that we’ll follow while modifying it.

    Projects view

    Click the image to enlarge it.

  • After some considerations, we decided to remove our integration with AWS Marketplace. It was possible to purchase the BitBar service through AWS Marketplace and have BitBar as part of your monthly AWS bill. Unfortunately, this did not really work as we expected.

  • We have started to try out changes in our trial user experience. We have shortened the path of trial users in getting to start their first test. Mostly the new funnel happens through our Dashboard view.

    Dashboard view

    Click the image to enlarge it.

  • ChromeDriver was updated on our devices allowing you to run Selenium and hybrid tests with the latest Chrome browser version on our Android devices.

This is a smaller release while we work on our back end improvements that are planned for later in the spring.

New features

  • We have installed and updated Mono, Nunit and nuget packages to our Android and iOS environments. This should help out all of you running C# tests in our service.

  • Additional improvements to the create automated test view, with more clues for users to follow. Our latest efforts on improving the visuals of our UI are really making a difference.

Bug fixes

  • On some iOS test runs video recording was not working properly and this got fixed.

Happy new year to all BitBar cloud users! We started this new decade by flushing out what we had done before leaving for the end of year holidays in December.

The content of this release has many internal changes that we needed to implement while unifying our processes with those of SmartBear tools. Also we’re going to do changes to our pricing in the coming weeks which we need to do some preparations for.

  • Our API was published on SwaggerHub so you can browse and try it out. Check the documentation here: https://app.swaggerhub.com/apis/smartbear/bitbar-cloud_api/2.88.0

    SwaggerHub

    Click the image to enlarge it.

  • With Apple updating their iOS environment, we again updated our Xcodes to the latest 11.3 version.

  • Sample applications directly in the tool to get started! We added application and test files for the most common frameworks. To get started just use the files we’ve already uploaded and click on start testing!

    Sample apps

    Click the image to enlarge it.

2019 releases

  • The new project view is brought out to the public! You should switch to this new view and start using it. Also, we’d love to hear from you about what you like or don’t in the new view.

    New Projects view

    Click the image to enlarge it.

    At the top of the screen, you can find the five last test runs followed by all your projects with all of their test runs. Opening a test run gives the basic run information with links to results, device run, and additional information about the run.

    Open test run

    Click the image to enlarge it.

  • Improvements in getting started with automated testing. E.g. when starting with Appium server-side tests, you can now start with ready samples straight from the new test run creator.

  • Android and iOS environments got updated with the latest Android SDK. We did this to make it easier to run Flutter based tests on these environments. We have the sample coming out any day now with the respect Flutter framework also becoming visible to the public cloud.

  • Our AI Testbot got updated to support the latest iOS versions! Give the bot a chance to exploration test your application!

  • We retired two unused frameworks from our UI, Calabash and AppCrawler. These are still available for some time over our API, but you can’t start runs with these using our UI.

    We are working on bringing two new more modern frameworks at the beginning of the year, but more on these as we get them done.

  • Server-side desktop browser runs now include execution video recordings. With this improvement you’ll see what actually happened during your test execution. Include the created video to bug reports for showing what really happened during the test execution!

  • Flutter is now supported on Android and iOS environments. If you already have Flutter test suites go ahead and try them out. We’re working on providing some samples to share for those who are still getting started with Flutter. More on these as we get them online!

  • Plenty of small changes on our test run creator view! There’s more step by step documentation, graphics and guides on how to proceed. We’re still adding extra guides so everyone gets to start a server-side test execution.

    Test run creator

    Click the image to enlarge it.

  • For all Ruby users, we have updated our Ruby client for easier interaction with the cloud.

  • A totally new server-side JavaScript sample. The steps to use the sample are the same as with other server or cloud-side tests. You need to zip your tests and upload them to BitBar for execution.

    zip sample-test.zip \
      package.json \
      package-lock.json \
      run-tests.sh \
      test/specs/main.js \
      wdio.conf.js

  • Python3 is now available on iOS and Android environments. If you haven’t started yet, you should start converting your Python tests to run under Python3 as the previous version is getting deprecated after the new year!

  • We have had remote USB connection available for some time already. In the public cloud, this is available for dedicated devices while in private and on-premise setups it is of course up to the cloud administrators to decide how this feature is made available.

    Today we introduced support for multiple ADB clients. This is now available for selected private cloud customers or public cloud customers with dedicated frameworks. Please get in contact with our sales if you need this.

    You can now in command line parameters define which version of ADB should be used for the connection, using the -adbversion command line parameter.

    For detailed information and the latest build, check out the remote client repository.

  • Improvements in our OWASP path. While we took Kubernetes into use some time ago, we noticed that some security features we already had implemented previously, were not there anymore. With this release, we brought those changes to our Kubernetes setup too.

  • Apple has kept us busy with new iOS updates, so we are bringing support for the latest iOS13.2 out with this release too.

  • We fixed some issues to continue supporting Akaze libraries on our latest Linux images on the public cloud.

  • Our Capabilities Creator now supports also configuring our Appium capabilities. No easier way of getting started with running client-side tests to real devices on our public cloud!

    Capabilities creator

    Click the image to enlarge it.

  • iOS test runs now support also Appium version 1.15.

  • We made the test run creator easier to use for first-timers! We’ll continue making changes to this view in the coming releases to make it super easy for everyone to start an automated test in our cloud!

    Test run center

    Click the image to enlarge it.

  • Whenever you create a new test run, you can see how you could do the same using our API. To make this easier we have a code block generator in the UI, from which you can copy-paste the required information to your command line or continuous integration service.

    Code presentation

    Click the image to enlarge it.

  • Our samples repository got updated with a guide on how to create a server-side desktop browser test run! This is the perfect approach for running fast, secure and robust desktop browser tests. Check out the samples on GitHub.

  • For some time we have been working on a new projects view that provides users with more information about the device test execution at first glance. You can start using it by enabling beta features from under “My account”. Here’s what’s waiting to come out as we iron out remaining issues.

    New Projects view

    Click the image to enlarge it.

  • Private and on-premise cloud admins get an easier way of maintaining team users. It’s now possible to move account users out of a managed account into own accounts that can be further managed.

  • Our Timeline functionality gets an overhaul to showing information tooltips on what has happened during the test execution and thumbnails of the screenshots taken during the execution.

    BitBar Timeline

    Click the image to enlarge it.

  • Android Appium gets updated to version 1.15.

During the past weeks, we’ve been fixing plenty of issues related to our transition to a Kubernetes hosted service! Unfortunately, this is visible from the smaller number of new features we have been able to push out lately.

Still, with every release, we bring you out something to make your testing easier!

  • iOS testing saw the support for Mojave and the latest Xcode! We’ve had iOS13 support on the public cloud already after last release for the native testing frameworks. Now we bring support to the remaining frameworks.

  • We have Kubernetes on production! All private and on-premise customers, as you’ll upgrade to newest versions of our cloud, you’ll get a Kubernetes installation if only supported by your infrastructure.

    This will provide you with easier and faster upgrades in the future. Also you’ll be able to run a newer version next to your existing installation to assure everything is working as you wish, before making the shift to the new instance!

    With this update, we also continue being the only cloud-native real device cloud service out there supporting the setup of device clouds in any customer environment.

We got bought by SmartBear! This is great news as we’ll be able to focus on our great service even more! We’re eager to see how to make the most of this new situation!

  • With Apple’s yearly new device publication, we started on working new iOS device support. Unfortunately with this release we only got the environment working so new devices will come with the next release.
  • Switching of our payment service provider to Stripe. We were thinking of this already for some time and finally did the change over the summer. Customers will benefit from improved public cloud prices where we’ll be able to provide better deals on real devices and browsers.
New features
  • New location supported for our desktop browsers. Besides having desktop browsers in Europe, we now also serve browsers in US West. This makes for a huge speed improvement for test runs started from the US!

    The best part is, this is completely transparent for the end-user. As a user, you don’t need to make any changes to your scripts and still, you’ll get browsers closest to you!

  • Continued A/B testing of our upgrade button. Some of you get to see some nice out of the box upgrade invitations in our service! ;-)

  • With the rest of the Internet mostly already using https, we moved these docs to under https too!

  • Improved iOS testing speed. We improved our iOS cloud side test execution to be faster than our competition!

  • We’re in the process of moving our service to run using Kubernetes. With this release, we’re moving our internal service to a new Kubernetes based environment.

Bug fixes
  • Our Run in Cloud Jenkins plugin did not correctly support setting the test timeout value. It does now.

BitBar development team typically has a longer internal development cycle during the summer over the month of July. This year was the same and a lot of effort was put into improving our service’s scalability and security. Although these are internal development efforts all our users get to benefit from these updates as they are pushed out to production.

Here are the highlights of this sprint:

  • We have implemented a number of security improvements again to our service. With our environment already one of the most secure enterprise grade cloud solution, we improved things like password strength requirements, support for multi factor authentication (MFA), upgrading to newer libraries and removing old versions.

    Many of the added features are also such that they can be enabled in whole or in part for enterprise installations. For example, multifactor authentication might be required only for administrators but not for other users.

    Public cloud users can enable MFA on their accounts under ‘My Account’ and Security tab.

    Enabling multi-factor authentication
  • UI preparations for bringing a new UI layout. More of this when the new UI gets out in following releases.

  • Bringing automated setup support for other cloud service providers like Google Cloud, Alibaba Cloud, Azure etc. While AWS is definitely the cloud mostly used by our customers, we see other cloud providers gaining traction and want to be ready supporting customers scale their testing cloud on other cloud providers. Obviously here too, customer specific security requirements are taken into account!

  • We have enabled client-side Selenium desktop browser testing support to all our customers on the public cloud. To get started, take a look how to update your desired capabilities with the capability creator. Once that is good, simply launch your tests against BitBar browsers!

This is our last release before we start a month long internal development sprint. We have plenty of tasks that we want to get done during this time so as to be able to provide new goodies in the fall again. Plenty of technology related upgrades and changes to make our cloud solution even more flexible.

This release is a small one that brings bug fixes and minor improvements to cloud. We wanted to push these out before we start our one month sprint for internal improvements. Some keywords for July include Dockers, Kubernetes, test automation, Azure cloud, OWASP standard, making Python3 default etc. It’s going to be superb!

Due to some Private cloud customer setups we decided to postpone our public cloud release a few weeks.

  • AI Testbot is now available in the UI as the default fully automated testing solution. Just import your application, create new test run and wait for results!

    AI testbot
  • Release of the Selenium desktop browser capability creator in our GitHub repo. This gives you a glimpse of what is coming out very shortly to general availability! Selected customers are already testing this.

    Capabilities creator

    Click the image to enlarge it.

  • Improved support for LDAP integration. This is of special interest for on-premise cloud setups integrated to customers’ LDAP.

  • We restructured our samples repository to make it easier to find appropriate samples. There are so many different programming languages and frameworks for both Android and iOS that having these all constantly available for new users is time consuming. We’ll continue refactoring to make the samples still better.

  • Some might have already observed we’re working on bringing desktop browser support to our online service. For this purpose we created acapabilities creator. Check it out and get in touch if you want to run your browser tests using our cloud.

  • We did an integration to Ranorex Automation tool to run both Android or iOS tests against our cloud. We don’t have this enabled on our public cloud but please contact our sales if you are interested in this.

  • Introduction of AI Testbot for iOS, but over API only. This allows you to start an AI test run using iOS devices.

  • With the previous release we introduced API for device-specific cleanup functionality where admins are able to specify by device what steps and procedures are taken for device cleaning. While the API supported all features, we now added the missing UI to simplify making changes.

  • A boost in strengthening our back end to better support third-party solution providers to integrate with the BitBar device cloud. For tool and service providers that want to offer some add-on service to customers running tests on BitBar have now a smoother way of publishing their service for BitBar.

  • Some may have noticed we have introduced new devices to the device list. Even though everyone is now able to see these, access to these is enabled during May.

Bug fixes
  • There was a bug where skipped tests were wrongly calculated into success ratio and giving incorrect percentage information. This has been fixed now. From now on skipped tests are NOT calculated to total test case count.

Small release pushing out some back end changes required by upcoming features.

  • Our remote USB feature, which was released last January for on-premise customers is now available also for Private and Public clouds. For more information on how to use this check out theGitHub repository. To get remote USB access to public cloud devices, you’ll need to request a dedicated device.

    Remote USB CLI

    Click the image to enlarge it.

  • File library allows you to upload multiple files to cloud at the same time.

This release included multiple fixes to our release process but also a number of features that should make several of our customers happy.

  • Security update with new login approach when session to service times out. Previously when user’s session timed out, user’s page and test results stayed visible under a login dialog. After this release user is moved to the login screen when the session times out.

  • Project and device group sharing were brought to the UI. Also for all resources shared with you, you’ll see a sharing icon showing the resource was shared by someone to you. To not disclose cloud users’s emails, when sharing resources you’ll always receive the same message regardless if you mistyped the share recipient email.

  • We’ve had since the beginning of times support for running tests on the first available device. Until now the used device may have been hidden behind all the skipped devices. Not anymore! The selected device is presented first whereas skipped devices are presented after the used devices.

  • Our company branding and web site have been updated. This branding update is also visible in the receipts we send out from our cloud service!

    Receipt

    Click the image to enlarge it.

  • Our Appium client-side broker now supports additional capabilities that allow you to use either different frameworks on BitBarbitbar_frameworkId or a different version of the Appium server bitbar_appiumVersion. For more information on how these work, please check our Appium capabilities.

  • Sharing resources between cloud users. It is now possible to share e.g. files to other cloud users. This is perfect when a new application is uploaded to the cloud and needs to be tested by a team of exploratory testers or even manual testers. Once you share a file with other users, every user is able to use the same resource and be sure testing is carried on the correct version of an application.

    File sharing

    Click the image to enlarge it.

  • Until now if you wanted to view any text files you needed to download the file to local machine and view it there. There’s now a new button in the UI that allows to open the files in a new browser tab and view log files there. This is a lot easier approach to going through and searching the logs.

    Opening files in a new tab

    Click the image to enlarge it.

  • Client-side Appium uses now the same apps as visible in the file browser UI. There is now only one approach to uploading files to cloud and it works for both client and server-side executions. For more information check Appium client side documentation.

A minor release while we prepare our new service to be release ready.

New features
  • Android side Appium was updated to Appium 1.10.1. This applies to all default Appium runs on BitBar cloud.

  • Client-side Appium runs can now make use of files uploaded over the UI. To use an existing app in your client-side tests, find the application’s file ID from the file browser.

    File ID

    Click the image to enlarge it.

    When you know the file ID, you can use that in your client-side Appium test bitbar_app = '126948355'.

    Of course instead of using the UI, you can upload your application (or test file) over the API from the command:

    curl -X POST -u "${API_KEY}:" -F 'file=@./BitbarSampleApp.apk' "${CLOUD_URL}/api/v2/files" | jq '.id'
    <code>

  • We removed integration to HipChat. HipChat was for a long time trying to rival with Slack as the best chat platform but last year Atlassian decided to cancel HipChat project. There is no purpose of keeping support for it. You should still configure your Slack integration

Bug fixes
  • We had an issue with manual testing when users were using Retina displays with devices that had either low or very high resolution. The displayed device screen did not look great and was difficult to read. We did some refactoring and re-plumbing to fix that. Now the manual testing looks good regardless of the device you try.

Second release of the year with important updates.

  • We’re bringing 3D secure to our online purchase process. As an online service provider we want our customers to feel safe about their online purchases. 3D secure is an added security protocol created on top of credit and debit card purchases.

  • We also tightened security on the count of allowed failed login attempts to the public cloud. We know forgetting passwords happens, but now we’ll lock your account after five failed attempts. You’ll then need to either request your administrator to unlock your account, or proceed to changing your password. Keep your email credentials safe!

  • There are some other important security changes we did on the back end side that are transparent to the end-users, but greatly improves your safety!

  • You can now enable receiving as notification your monthly plan receipt on the public cloud (or you can have it sent directly to someone else).

    Receipt email

    In addition to getting notifications on the receipt, you can also get notified when you are about to go over your monthly plan. Note that this same is available as Slack notification!

  • We are trying out some changes in our registration process. New users, be prepared for whole new experience! ;-)

  • Plenty of UI changes especially for admin users. Better handling of tables and dashboard!

Happy New Year!!! Starting off for a great year 2019

  • There are new API resources allowing users to share resources in cloud. Using the API, users are able to create access groups and include users and resources to that group. Users in the same group can access the resources in that group.

    For now shareable resources include: projects, files, file sets and device groups.

    Looking at the usage, we’ll probably implement some UI for this so all cloud users will be able to use this without having to use an API.

  • With last release we added support for remote USB connection to our devices. You are now able to get our latest remote client from the public GitHub repository.

  • Finally but not least there are plenty of UI tweaks that make the handling of sub users nicer and just looks good!

2018 releases

  • There is a new filtering criteria for Public cloud devices. You can now search for devices based on their physical location. Currently available locations are EU or US. When doing eg. manual or client side Appium testing, the physical location of the device may be important as you want to avoid extra network latencies to the device.

  • Cloud is now able to display the device screen and logs while test execution is ongoing for Appium server side test runs. When a test run starts a new button appears on the device test view allowing you to see the test execution. To enable this functionality, you need to enable the Beta functionality on, from under your preferences.

    Live view

    Click the image to enlarge it.

    Clicking on that button, opens you new view showing the device screen and the ongoing test execution. Note that you are not able to interact with the device but only view the execution.

    Live view

    Click the image to enlarge it.

  • Our device cloud now supports creating remote USB connection from your local machine to dedicated Android or iOS devices in BitBar cloud. Using the remote device client, you can connect to any of your dedicated device in the cloud and access it like it would be connected locally.

    This allows you to eg. run tests against your dedicated devices in cloud using proprietary testing tools.

    The client is built using Java, so you’ll need to have a working Java runtime environment and as we’re using the latest ADB server on our side, you might need to update yours to get the connection up.

  • Invite users to your account to make your team more effective in mobile application testing. In right corner of you

    Inviting users

    Click the image to enlarge it.

  • Back end preparations for our new remote connection service. This will allow you to connect to our devices in the cloud as if these were connected directly to your own machine. This will be available for all dedicated devices, iOS and Android.

  • iOS12 support on all public cloud devices. There were already some devices we upgraded to iOS12 and made them available. Remaining i-devices are being updated to iOS12 and enabled on cloud.

  • Public cloud moved inside a more secure AWS private cloud environment. Public cloud is now running with same setup as our private clouds!

New features
  • Remote connection to devices is currently available for Private cloud customers (new installations). This allows you to make remote connections to your Android devices from your host machine to control the selected device using ADB commands.

  • Appium upgraded to 1.9.1 for Android runs client and server side. iOS had this support already released earlier.

  • Okta authentication support for Private and On-premise clients. This allows customers to own and control user access and permissions straight out of Okta!

  • Run in Cloud Jenkins plugin was updated to support Jenkins pipeline script snippet generation. This update eases your transition into Jenkins pipeline scripted CI.

Bug fixes
  • There was an annoying bug in Android side test videos where the video was pixelated and had strange colors appearing. This is now fixed and videos are clean again.

As mentioned with last release, we’re in the middle of building some great new stuff that’ll come out in following releases.

This release consists mostly of updates and changes mandatory for smoothly running a device cloud. You know, updating all the stuff you don’t think of when thinking “we could build a device cloud on our own”. But don’t worry, we’re keeping our cloud up to date so you don’t have to think of this stuff.

Bug fixes
  • Users registered through AWS had issues on buying a plan :-( This is now fixed, so go ahead and use BitBar device cloud and get charged on your AWS account.

We are in the middle of building some cool stuff that takes a bit longer than our typical two weeks release cycle. For this reason we’re having some smaller releases now. The great stuff is coming out in a month or so. Stay tuned for those!

Here are some of the new changes that were pushed out:

  • New users get a landing page to get started with using our cloud solution. This is an easy approach to starting AI testing, Manual testing or an automated test script run in cloud.

  • Many users didn’t notice the single sign on buttons we had on the login page so we made them more explicit with specific buttons.

  • BitBar Gradle plugin was updated to allow setting test run timeout values. By default test timeout is set to 30 minutes. When creating a test run manually you can set this timeout in the advanced settings of the test run.

  • To ease the use of uploading and managing files you upload to BitBar cloud, we added a new button at the top bar where you can at any time go and edit the files you have in the cloud.

    File browser

This release didn’t go as smoothly as we had planned it. Database changes were planned but during release things got off track, causing a cloud downtime. We are sorry about this.

There was a small release a few weeks back but there was little changes visible to end users so we didn’t write about it.

Last week we releases our AI powered automation for exploratory Android application testing. AI Testbot uses machine learning and artificial intelligent to explore your application for as long as you give it time. By default all our test runs are time limited to 30 minutes (this is configurable in your test run’s advanced settings). The results are really impressive and you should definitely check our AI Testbot. Currently only Android is supported but iOS support is under way too!

New features
  • iOS environment update. We’re taking the latest XCode9 into use to give you devices with iOS11.4 (we’ll update our devices following today’s release).

  • The iOS environment upgrade also brings the support for iOS manual testing support for premium customers. Manual testing works just the same as with Android devices. Select ‘Manual Testing’ from the menu, pick one of the iPhones and wait for the screen to get connected.

    iOS manual testing

    Click the image to enlarge it.

  • Appium gets updated to 1.8.x. We wanted to have version 1.9, but it got release a day after we had fixed our release content. Remember that you can still use which ever version of framework you want, by just installing it at the beginning of your test run.

This is last release until we head to our yearly automation month to hack and code some bigger improvements to our system for a whole month! But more on this in August!

Here are the changes we’re bringing out now.

  • iOS performance data is available for all iOS test runs now.

    iOS performance
  • Admin dashboard customization gets out of beta for all admins to enjoy. Go remove or add the widgets you want to remove or bring back!

  • As we believe the best way to use our service is through our API, we’re now providing an even easier access to using it by providing API access with Swagger. You need to be logged in to access the API descriptions. Once in you can even try out the API commands in the web UI. Access Swagger from My Account > API Integration > Open Swagger.

    Swagger link
  • SSO using your Azure Active Directory credentials!

    Azure SSO

During the last days this release’s regression testing after all our automation tests had passed successfully, our QA team found bugs. Fixing and re-testing the issues took some days postponing the release until Monday 18th of June.

Unfortunately we were not able to fix all issues within our Android environment forcing us to keep video recording disabled for some Android test runs.

New features
  • The admin dashboard has new beta feature (needs to be turned on under user’s account settings) to allow user to select which admin dashboard widgets are visible.

    Admin dashboard

    Click the image to enlarge it.

    For what ever reason you might not wish to have all the widgets visible filling your dashboard. All reports are not equal so make the dashboard your own!

    Dashboard Plugins List
  • Cloud administrators will also enjoy the new possibility to define testing framework support at device level. Device level testing framework selection allows deciding which devices support manual testing, Appium automation or XCUITests. End users are able to select devices based on the required device properties or supported frameworks as shown in below image.

    Supported frameworks
  • Creating a new project just got easier as we’ve removed the need to define the project type. To create a new project you just need to give the project a new name. It is during new test run creation the used framework needs to be defined. After first run creation, same framework is used if not explicitly changed.

    Creating a new project
  • Jenkins Run in Cloud (RiC) plugin has been updated to support Jenkins Pipelines. Running mobile tests from within Jenkins is now a simple pipeline step. For more information, checkout the RiC documentation.

  • We decided to increase the default test timeout to 30 minutes. With customers relying more and more on our platform the previous timeout was too restricting. Users can still configure test timeout value in test run’s advanced settings.

  • Remote USB connectivity to cloud devices for on-premise setups. Ever wanted to debug that one device on which tests always fail? Now you can do this by taking a remote connection to the device and seeing the device on your local laptop. Run ADB commands, install or remove apps and do what you need to get you app working. This is available for on-premise setups with own devices for now. For more information, get in contact with your BitBar account executive.

We’re currently buried with some larger updates coming up as soon as we get them implemented on the back end side of our service. Nevertheless we have some things that I’m sure you’ll find interesting and useful for your testing!

We also have a sad news, as with this release we removed the support for Gamebench reporting in our public cloud. It’s still something that we offer as an addition for our private and on-premise customers.

New features
  • We brought the new Test Run Creator out of beta and made it the default one. We believe creating new runs is now simpler and faster than before. Also with the new test run creator menu item it’ll be easier than before to start whole project with test run at one shot!

    You need to provide the same basic information as before, environment (iOS/Android), used framework, files to upload and finally the devices to use. It’s also possible to provide advanced settings (like project and test run names etc.). Do check out the advanced settings too!

    Test run creator
  • New device run view with control panel! We’re really exited about this and it’ll be covered in more details in the user manual. But let’s check the main views here.

    • Control panel – A way to quickly check how this device and test run performed against other devices in this run.

      Control panel

      Click the image to enlarge it.

    • Tests and Steps – To view the test steps that were executed as part of the execution. In case of some execution error this will let you to see until where did the execution proceed before some error happened.

      Tests and Steps panel

      Click the image to enlarge it.

    • Issues – As part of results processing we check a number of things from the logs to try to figure out if something might have gone wrong under the hood or if there is something that you should try to verify.

      Issues panel

      Click the image to enlarge it.

    • Output Files – Similar to what there was previously we have gathered all available files to this view. You are here able to download all generated files. Of course these are also still available over API.

      Output files panel

      Click the image to enlarge it.

  • iOS execution performance and robustness improvements. We improved our iOS side test frameworks’ robustness quite a bit during the past month. You should definitely try out Appium or XC(UI)Test test executions to check out how much faster and more robust the environment is!

    As an additional perk, it’s easier than ever for us to dedicate iOS devices for manual testing in public cloud. If you need iOS manual testing just get in contact with our sales@bitbar.com.

With various public holidays during the past weeks we decided postponing our release to May 9th. Plenty of small and bigger fixes and improvements in the back end but also some user facing changes. Enjoy the updates!

New features
  • Update to our “client side” Appium runs. When running your Appium tests locally against our devices in cloud you can now quit your session inside of the test and start a new test session without having to find and wait for a new device. In more details, you can do something like this:

    driver = new IOSDriver<>(new URL("http://appium.bitbar.com/wd/hub"),
        capabilities);
    // some testing here...
    driver.findElement(By.className("XCUIElementTypeSearchField")).sendKeys("Helsinki");
    driver.quit();
    // make what ever changes you need to the test setup or initial configuration
    // and start new session
    driver = new IOSDriver<>(new URL("http://appium.bitbar.com/wd/hub"),
        capabilities);
    // new test steps with new test setup
    // ...

  • Improvement for on-premise and private cloud administrators. You are now able to follow which devices keep getting long waiting times and which users have been using the cloud most during the last week.

  • User Dashboard got links to your latest test runs, allowing you to quickly jump into the latest runs.

  • API default response mode was changed to JSON (used to be XML). So no need to necessarily include the -H "Accept: application/json" to your API calls. Response will be in JSON regardless.

    # both below calls response is in JSON
    $ curl -s -H "Accept: application/json" -u "${API_KEY}": -X GET "${CLOUD_URL}api/v2/me/projects" | jq '.'
      ...
    $ curl -s -u "${API_KEY}": -X GET "${CLOUD_URL}api/v2/me/projects" | jq '.'

  • Those users that are trying out our Beta features, know we have a new test run creator. New improvements were made here, so we’ll be ready to release the creator in the next coming releases.

  • We removed GB from our cloud test runs. We’ll continue with removing the related UI parts in the coming releases. Unfortunately there wasn’t the kind of request from our customers as we expected to justify having the integration on our public cloud. GB integration is still available as an add-on for private and on-premise customers requiring it!

  • Improvements to our iOS test execution that are being rolled out during the following weeks.

We updated our Appium broker (the part that enables client side Appium) in our cloud. In addition to it getting faster and improved internals we also added new capability for allowing you to have multiple Appium test sessions during a single broker session.

It is currently possible to control the duration the broker stays waiting for new Appium sessions after the test calls driver.quit(). A new capability testdroid_multiSessionWait was introduced. This capability takes the duration in seconds for how long the broker session needs to listen for new tests. More on the capability here.

A lot of things going on… Customer specific private cloud updates and improvements but also some bigger changes cooking for the public cloud.

New features
  • For private cloud setups, Appium was updated to latest version 1.7.2.

  • Back end work on restructuring test execution logic and order. This should improve especially iOS side runs.

  • Enable GitHub single sign on to BitBar cloud. In addition to users being able to sign in with Google, now also GitHub is supported.

    Sign in with GitHub
  • Main domain was changed to use cloud.bitbar.com and appium.bitbar.com. Old testdroid based domains are still up but are being phased out progressively.

Fixes and improvements
  • The new Test run creator had errors that our QA team noticed and were fixed now before the general availability of the creator.

  • Files library didn’t work on Mac properly and one was not able to select multiple files at the same time for upload to device.

We don’t believe in big bang releases but prefer to regularly push out any changes we have. This allows us to have some features partly out even if not yet released for the public. One such feature is the new test run creator that is released as a “beta feature”.

New features
  • New test run creator to simplify the creation of new runs. The old creator is already fairly old and starts to be a bottle neck for new functionality. As said this is now a beta feature and will be released in coming releases after we fix some findings and get some feedback.

    Test run creator
  • Framework version upgrades. Appium got to version 1.7.2 and Calabash (calabash-android 0.9.2, Cucumber 3.1.0 and calabash-ios 0.21.4). Default Ruby version stays as 2.3.1 but with the use of RVM version 2.5.0: rvm use 2.5.

  • Initial speed up of iOS manual testing on dedicated devices on public cloud. This is first step in a series of improvements for iOS manual and automation testing.

Fixes and improvements
  • There was a bug where video recording did not record whole screen when device screen was rotated. This is now fixed.

  • Improvements to user dashboard layout. Project widget was reformatted to give more space for project names over success bars.

We are moving to having no-downtime releases in our public cloud. During last release (Feb 22nd) we forgot to announce that manual testing and client side Appium runs weren’t available during the release. This time around they still won’t work during release but we remembered to notify about it :-).

Cloud notification
New features
  • New toolset for video recording for iOS test runs. We tried using an iOS library for recording test runs earlier but it had all kinds of issues. Additionally this new approach allows us to implement later some cool new features that will be first available on Private clouds. iOS video recording is enabled on limited number of devices for now.

  • Updates on the Android testing frameworks. Appium updated to version 1.7.2. New Appium version has plenty of new fixes and improves robustness. Appium upgrade on iOS side comes with next release.

  • Calabash, Cucumber and Ruby are being updated now and made available latest with next release in two weeks.

Fixes and improvements
  • We have had some issues with Android server side test runs not getting broken videos. With release 2.54 we brought a fix to remove the problem. Let us know if you still get some test run where the video is broken.

  • We speed up the start of iOS manual testing on dedicated devices. We still have some ideas up our sleeves to further improve the iOS manual testing experience.

We are preparing to roll out some new services in our private and public clouds. Meanwhile we improve our current service usability and provide better tools to run and debug test runs.

  • A long waited change for testers using Ruby, eg. many Calabash testers. When you load your Gemfile with your test package for server side execution, our system will handle gems installation for you. This allows you to use the same gems and versions you are using locally.

    Gemfile
  • We added timestamps to our console logs - this helps you to find possible errors easier as you know when something went wrong. While debugging can be a pain this should make it just a little easier.

  • A ton of bigger and smaller UI changes and improvements! Overall we have improved spacing and locations of icons in the UI. It’s small changes but really makes a difference. Additionally we added some new perks that we think you’ll love.

    • Did you know you could use avatars in our service? Now it’s brought more visible to under your account information. Go crazy and make the service look like you!

    • Is there something you require help with, do you think something could be improved or would like to send us some nice comments? You can do that through the feedback dialog under the life buoy icon.

      Send feedback
      Send feedback form
    • Access your files! You need to run tests on that file you uploaded the previous day? No problem, you can access these files through the file browser.

      My files library
    • There’s a new timeline view in the device run view. This view collects different test artifacts and shows them combined. You’ll be able to view screenshots with logs and performance data.

      Timeline

      Click the image to enlarge it.

      Timeline view

      Click the image to enlarge it.

The second release of the year, time really flies when you’re having fun! In this release the front end team managed to continue moving forward with taking Vue.js framework into use. For end users this shows as an updated UI layout. Also this new layout will give us more room to provide you with more direct links and add some additional services to come.

New layout

Click the image to enlarge it.

More on the updated UI in our blog.

Here are some other changes that came out with this release.

  • iOS XCUITests are again automatically taking screenshots. With previous Apple Xcode update automatic screenshot capturing was removed from Xcode, but many of our customers were really liking that feature. For this reason we have now enabled it for XCUITest runs.

  • Users can delete their accounts from our cloud. This will remove any user related data that was uploaded by the user to our cloud. For more information check Delete Account.

    Delete account dialog

    Click the image to enlarge it.

  • Improvements to Public Cloud iOS Appium test runs. We had some issues with runs not starting that should get fixed with this change. This does not affect private or on-premise setups where this was not an issue. The approach to run tests is slightly different in these environments.

BitBar wants to wish you all a great and successful new year! This year brings multiple great changes for mobile developers and testers using the BitBar cloud.

Before the end of 2017 we cleaned out some smaller stories and hold larger changes for this beginning of the year. The latest release brings out the changes we worked on before leaving for the holidays.

  • BitBar Build service gets released for dedicated environments. This is an API service for creating and launching iOS (or Android) builds on dedicated hardware with the possibility of running simulator tests after a successful build. End user has access to all build artifacts and logs for possible trouble shooting.

  • Our Monitoring service environment was updated with same Appium version as used on the testing side. Moving existing tests for monitoring purposes is now straight forward.

  • For Private clouds we updated the admin dashboard with some new analytics about cloud device distribution with respect to market device distributions. This is a great way for admins to follow which devices need to be updated with newer versions of operating system to follow market shares.

    Admin dashboard

    Click the image to enlarge it.

    This shows our staging environment’s Android, iOS and OpenGL distributions in the cloud and in the wild.

  • Appium broker is now served from https://appium.bitbar.com/hub/wd. Make sure to update your client side tests with the new endpoint!

  • API documentation was updated. The online documentation now shows which values can be queried and sorted.

  • Public cloud is updated to support latest iOS 11.2.1 and the flagship iPhoneX device. The device is added during the following days.

  • Manual testing session supports now filtering of the logs. Only logs visible can be later downloaded.

    Log filtering

    Click the image to enlarge it.

2017 releases

Last release got delayed a few days after our QAs found some blocking bug. Once that fixed we were able to get out a great release with some interesting new features. Something for the end users and something for the admins! We believe in providing our users with as much data from their test runs as possible to make it easier for them to do their job. Either report progress, verify new features in an application or fix some bug. For this reason there are now additional dashboard widgets to bring additional data to our users.

  • With our new manual testing released last time we forgot to include the button for changing the device orientation. Thanks for the feedback, we now brought it back.

    Rotate button

    Click the image to enlarge it.

  • iOS manual testing is available for on-premise setups. It behaves pretty close to same as the Android manual testing.

  • Plenty of UI updates that are best described in our blog by our front end developers. Just go check that out to see all cool stuff.

  • New user dashboard. If you read the above blog post, you must have noticed the new dashboard.

    Dashboard

    Click the image to enlarge it.

  • As cloud admin it is important to know how the devices in cloud are distributed by eg. operating system or API versions. New admin dashboard widgets give you exactly this!

    Admin widgets

    Click the image to enlarge it.

Great Thanksgiving! Hopefully you managed to get your apps tested and are now enjoying a well deserved break with your app delivery cycle! We have some great new features to make your app testing a lot more fun.

  • Manual testing has been redone from scratch. We are true believers of test automation, but there are valid cases for doing manual testing too. With our new manual testing implementation you’ll want to do manual checks just because it’s so much fun. Even when using shared public cloud devices app feedback is great. For on-premise customers, responsiveness can further be improved. We are increasing the number of supported devices gradually while we make sure each device supports the new solution.

    Our manual testing has the extra benefit it can be integrated into other tools like eg. IDEs. If you are interested in knowing more about such an option please get in touch with us through feedback.bitbar@smartbear.com

    Currently manual testing is enabled for Android devices. For iOS device support a dedicated device is required. For getting access to iOS devices please contact our sales@bitbar.com.

  • Devices view and device groups. Last release brought a new devices view and now we updated the device groups handling page. Full documentation is described at Device Groups.

    Device groups

    Click the image to enlarge it.

  • iOS test runs are now showing performance data similar to what has been available for Android test runs. This shows CPU and RAM usage of the device during test execution.

    iOS performance data

    Click the image to enlarge it.

We are working on introducing an improved manual testing solution that is just awesome. But that’ll come in the next releases. With this release we brought some general improvements to the public cloud.

  • Introduced Appium 1.7.1 to Android side test runs. On iOS we are running version 1.7.0. There is no major difference between the two. If for any reason you need to run Appium 1.6 or 1.5, it is possible on our Android server side runs. You need to select which version to use in you run-tests.sh script. If you want to take 1.7.1 into use check your run-tests.sh script is updated to call new version!

    # start the Appium server
    echo "Starting Appium ..."
    appium-1.7 --log-no-colors --log-timestamp

  • Customers with dedicated devices see these as first devices in the devices view. It’s easier to start using those.

    Devices view
  • A whole new devices view! You probably noticed that from previous small screenshot, but it now looks fabulous. Select the filters to suite your test requirements and create a new device group by dragging the selected devices on top of the device groups on right hand side.

    Device groups

Release was mostly focused on getting last hurdles with remaining testing frameworks to properly work with iOS11. Also we improved cleaning of iOS devices after test runs.

  • iOS11 support on all iOS side test frameworks. Appium support was already there but now also Calabash support for iOS11 was introduced. Of course you can also use any other (or own) test framework by installing it at the beginning of the test run. More informationhere.

  • Cleanup of Safari browser (history and cache) after test runs. Even if we do these cleanups you should not leave any open sessions from your test sessions. For improved security you should go for dedicated devices. Sales can help you with these at sales@bitbar.com.

  • We brought a place holder for our Monitoring service to the cloud UI. This page will get updated during the end of the year to include creating and viewing basic checks.

A lot of things brewing in our development pipeline for new features and performance improvements. There was release on Wed 4th of October that brought out some neat improvements and new features.

New features
  • We are rolling out support for iOS11 with latest Appium 1.7.0. We first introduced a limited number of dedicated devices with this setup and now bringing additional devices online. These devices support XCUITest, XCTest and Appium test frameworks. We’re bringing Calabash support also during the next two weeks.

  • BitBar cloud is moving forwards with iOS build support! We have had support for iOS builds but have not really made a number of it. We are now introducing dedicated iOS build environments to our private customers. The solution is targeted for companies with well defined build environments requiring a fast and always available iOS build environment.

  • We have introduced improvements to cloud usage activity logging. The list of actions is now more detailed on who has used the cloud and how with cleaner logs.

  • Users coming from external marketplaces likeBluemix or AWS Marketplace have now a more streamlined user interface.

  • We created a new tool enabling Android device network management and traffic. On-premise and Private customers can now monitor the device network availability in their test environments.

Bug fixes
  • We fixed some bugs related to our Android device cleanup mechanism. Cleanup happens now faster and it is more robust.

  • GB integration was brought back to Android UIAutomator runs. We had to limit GameBench usage with some of our test frameworks but were now able to bring it back. This particularly affects the AppCrawler runs.

During the past three months we have had several releases of which some were minimal and for this reason did not put anything here. There was a major release that could have gone better but kept us so busy we waited until now to write about.

The last release in August was a huge one, where we updated most of our infrastructure to support a more flexible and service oriented architecture. This included the switch to using Docker images for running our tests. There have been multiple blog and email posts on this earlier.

This week’s release was mostly a preparation release for upcoming Xcode9 support with iOS11 and test framework upgrades at the end of September. Also we’ll be available on some new marketplaces in the coming days which is also cool.

  • Additional options for Slack and HipChat notifications. In addition to getting notification from test runs, you can now also get notifications about cloud status or news about new devices included into cloud.

    Notification options
  • Private and On-premise administrator get more control on what and how to inform end users. The admin menu has a totally new menu item to control to whom, what information and how is sent. This is a great way of letting users know about the addition of new devices or frameworks to run tests on.

  • iOS updates rolling out after release!

  • Fastlane – Fastlane is now available as pre-installed on the test machines.

  • Xcode  –  was updated to Xcode8.3. We’ll start upgrading devices to support the latest available iOS version 10.3.2 during next week.

  • We decided on not updating Appium version as the new stable version came very late and we didn’t know how stable it would be.

  • Calabash upgraded to latest version 0.20.5

  • Support for Gemfile-file in uploaded test zip files. This should make it easier to setup special Ruby test environments.

  • New registration and activation process. If you don’t yet have an account for BitBar Testing go and create a new trial account!

  • Video recording is now available and set on for all Android test runs, except manual test sessions. For now, videos of test runs are available on the device test run view next to log files. Note some devices do not support video recording and getting the videos into the result files can take up to a minute longer than the other files.

    Resulting files
  • HipChat integration. Last time we brought support for posting test run results to Slack, this time the same is available for HipChat. The integration is described at HipChat notifications.

    HipChat integration
  • iOS device availability was improved significantly. We have improved the way how our iOS devices are visible and available in the public cloud. All existing users were notified about this change beforehand as it affected all existing iOS device groups. As a summary this update broke existing device groups with iOS devices and these groups need to be recreated.

  • Making management of the main users easier for BitBar Testing admins - especially useful for on-premise installations or private cloud installations with admin rights - main admin can now convert a project sub-user into a main user. This action will make the original main user into a sub user.

  • Lastly, we hope you’ll enjoy the updated colors in our UI. We think these bring a breeze of freshness.

  • All users wishing to get in contact with us, can do so directly from the cloud, using the get in touch window. This is now moved to the top bar next to system notifications. You’ll be able to send us screenshots and files through this feedback form too.

    Feedback

    Click the image to enlarge it.

Bug fixes
  • We were wrongly presenting some Android Appium server side test runs as iOS UIAutomation tests. This had stayed from when we disabled support for UIAutomater altogether.

  • To get a better overview of the cloud usage for on-premise customers we’ve added some reporting about the cloud to Admin - Dashboard. You’ll here be able to check which users and teams are most using the cloud, which devices are being used (you might want to add additional instances) etc.

    Admin dashboard

    Click the image to enlarge it.

  • On-premise customers get improved tracking on project and user creations so admins know who are using which parts of the product!

  • Slack notifications! We too love our Slack and thought that you should be able to get your test notifications to the channels that you use in your daily work. We’ll be adding some additional chat integrations here in the coming releases.

    Slack integration
  • A new widget for getting email notifications was also added while working on the Slack notifications. You just need to add your email here, and select what type of notifications you want to receive into your mailbox!

    Email notifications
  • Support for running Gamebench analytics on the client side Appium runs. To get started with this you just need to have Gamebench configured and use thetestdroid_gamebench capability in your script. Check out more on the capability here.

    Gamebench capability
  • And a plethora of other things under the hood that is not directly visible on the UI.

A few big changes in today’s release. First the change in our free trial plan and the second in our API. Additionally some fun stuff for manual testers.

  • Free plan changes to a free limited time trial period.

  • API change for file uploading. We wrote about this in ablog post last week. In practice this affects all users using theJava client, Run in Cloud Jenkins plugin, Gradle plugin.

  • Access to BitBar Monitoring. As you may have read we are working on a new productBitBar Monitoring. This is a mobile application synthetic monitoring solution that can be accessed with BitBar Testing credentials.

  • Possibility to set key bindings to your manual testing session. When testing some application that needs the user clicking on particular buttons or regions in the app multiple times. Doing this by mouse can be cumbersome but assigning keybindings for these make testing really simple. To try this out, you’ll need to enable Beta features under ‘My account’. When a manual session is started at the bottom of the device widget there will be an option to create key bindings.

    Key bindings

While previous release 2.36 did not bring much visible improvements to public cloud, there are some major ones with this release. This is also the reason there is no release notes for the previous release.

  • Notable changes relate to using manual testing. In short whole of the manual session was rewritten to make it faster, more usable and easier for the user to install all the apps that are needed during the testing session. For a whole description on how the manual testing works, please check theuser manual page.

  • The used notification system got also updated. This is the reason cloud now asks at first login to allow showing of notifications.

    Allow cloud notifications

This will get used more in the future as we implement support for it. Our main developer wrote about it a blog post.

In addition to the above UI changes there were several back end fixes that improve the security and availability of devices.

This release consisted mostly of technical improvements to our iOS test runs infrastructure. With the last releases of Xcode there has been a lot of changes in mobile testing behavior.

Old Apple UI Automation framework got removed from the supported test frameworks as it hasn’t been supported by Apple either for years.

There are visible UI changes on the My Account page. For those interested in the reasons behind the changes can be read here.

Support for integrating to Gamebench service directly from the UI. Also note that the other possible integrations are now grouped here under My account.

My integrations
  • iOS AppCrawler enabled again. Due to the iOS side updates that were done since the beginning of the year, now the AppCrawler was done from scratch. Because of this the AppCrawler won’t work exactly the same as the old one. Also, for now, it won’t be able to fill in login or other forms.

    iOS AppCrawlr
  • UI modifications and improvements. Notably changes in the My Account page which now contains available and enabled integrations and the list of account users. Account users are users that are allowed to run test runs billed to the current user.

    My integrations
  • Interested in getting deeper hardware statistics about how apps behave on various devices in BitBar Testing? It’s now possible by providing existing Gamebench credentials to get device and app statistics gathered from each test run in BitBar Testing. This same Gamebench integration is also supported on On-premise and private cloud setups.

    GameBench integration

Once a testrun is executed and Gamebench has been enabled then a link to the results is presented in the test run results page.

GameBench results

This link will take you directly to the Gamebench portal where the results for the used devices are presented.

GameBench

Click the image to enlarge it.

  • A ton of small fixed here and there, especially on our iOS side Appium execution.

During the past month there have been multiple updates done to BitBar public cloud. Some have been visible to end users and some not. The below list of enhancements are available on the public cloud but also available to our On-Premise and Private cloud customers. To get all latest features whole or parts of On-Premise or Private cloud needs to be updated. Please contact your dedicated support contact.

GameBench integration
New features
  • XCTest and XCUITest frameworks are available to all users of BitBar Testing cloud! These have been available to a limit number of users already since December but now all public cloud users get access to these. BitBarGitHub samples present an example test project with both XCTest and XCUITest test steps.

    The documentation on how to prepare your Xcode project for these frameworks is available here.

    Projects
  • Latest stable Appium version 1.6.3, supporting newest iOS and Android devices has been installed to public cloud. This is the default Appium available for client side and server side test executions. On Android server side it is still possible to select some older version if required. For iOS, this is the only supported version.

    iOS side Appium Safari testing and iOS AppCrawler support are introduced to public cloud in few weeks with separate updates.

    Attention Users using older versions of Appium (before 1.5) may need to update their test scripts. Appium has made changes to default desired capabilities and deprecated some methods since Appium version 1.4. For example with Android tests, the method find_element_by_name(<somestring>) has been deprecated and needs to be updated with find_element_by_android_uiautomator('newUiSelector().text(<somestring>)). Please check our sample tests online for updated tests or Appium release notes.

    Next major Appium version upgrade on BitBar Testing is done towards end of Q1 2017.

  • Public cloud iOS devices are upgraded to latest available and Apple supported versions. As with earlier version of iOS most users have already updated their devices to the latest iOS 10.2 version. In public cloud, only a limited number of devices with older iOS versions 6, 7 or 8 remain.

iOS 10 adoption trends

Click the image to enlarge it.

  • Support for Java 8 is available for test runs.

  • Chromedriver was updated to 2.27.440175. This update was done already before Christmas, but we had a bug and the update didn’t go through everywhere as it should have.

2016 releases

  • XCTest for iOS10 devices available to all paid plans. iOS10 devices were introduced to public cloud about a month ago and now the support for running XCTest test on them is available to all users. For more information on how to build the app IPA and how to package the tests is available inour documentation.

BitBarGitHub sample repository also contains an example XCTest project that can be used as a starting point with the above documentation, when starting out.

With the support for XCTest project type, it is now also possible to run KIF tests in BitBar Testing public cloud. The same has been available for a long time in Private and Enterprise installations.

Android instrumentation

XCTest is available only on newer iOS devices with iOS version 10 or above. Support for older devices is coming in the following weeks.

  • Calabash project type available to all new users regardless of the registered plan. Free users are limited by number of available free devices. Existing free users get access to Calabash projects by upgrading to apaid plan.

  • iOS manual testing support is now part of default BitBar Testing. Manual testing with iOS devices was until now enabled per device for dedicated devices and for Private and Enterprise installations. Testing how the app looks and behaves on real Apple devices is now easier than ever.

  • Jenkins Run in Cloud plugin updated to version 1.0.17. This fixes issues with downloading results and starting Appium server test runs.

  • Calabash environment variables. For better control over Calabash test run executions environment variables are typically used. In BitBar Testing it is possible to give values to the two pre-existing environment variablesCALABASH_TAGS and CALABASH_PROFILE, in the advanced settings of a test run. These can be used to better orchestrate in which order tests get executed.

    Calabash environment variables

For private and on-premise installations it is also possible to allow users create their own variable and value pairs.

  • Project type names were renamed for better clarity. BitBar Testing solution is very test framework agnostic but we have a number of predefined common project types available. Below is how we have renamed the project types.

  • Android > Android Instrumentation

    Android’s own unit testing framework meant to run on the device for functional testing.

  • Appium Android > Appium Android client side

    To unify the naming convention with Appium client and server side projects. Earlier naming was not explicit about the client side approach.

  • Appium iOS > Appium iOS client side

    Uniformity to project naming conventions between client and server side projects.

    Project naming
  • Chromedriver gets a version upgrade to latest version of 2.25.

Technical release with reliability updates and preparations for upcoming features.

  • For easier screenshot step recognition in device test run view, the screenshot names are attached below the screenshots.

    Screenshot names
  • Appium server sideimage recognition sample test gets a total rework. The sample project shows step by step how to do image recognition testing using Appium for testing eg. games.

Setting up needed environment is much easier to earlier example with less external dependencies and libraries. Used libraries were updated to the latest ones.

Launching server side tests is simplified by introducing the launch-tests.sh script. This script is able to create the test zip, handle uploading of the required files as well as creating a new test project, when needed. Ideally the script is used by CI systems, but manual usage works just as well.

$ bash launch-tests.sh
Either -a or -i flag must be provided, but not both!

  launch-tests.sh - create and upload test project to Testdroid Cloud and run it

  Usage: launch-tests.sh -a/i -g <DEVICE_GROUP_NAME> -k <API_KEY>
   -a for Android test
   -i for iOS test
  Optional: -p <PROJECT_NAME> to choose a specific profile. If not given, a new project will be created
  Optional: -t for creating and uploading a new test zip file
  Optional: -f <APP_FILE_PATH> for uploading a new app file
  Optional: -e <API_ENDPOINT> for private cloud instances

The launch-tests.sh script is already available in image recognition samples.

  • We setup our Maven central proxy making any Maven installs more robust and a lot faster. This is great news for all and especially Java server side mobile test runs. For more information about Maven mirrors checkhere.

  • Sample demo project is removed in BitBar testing after user creates first new project.

New features
  • Improvements to device filters. The back end behavior for handling various device properties was improved to make sure the search criteria are better up to date and results more accurate than before. It’s now easier to add new device labels (for On-Premise installations) and to search for devices from the UI.

  • Small changes make a difference. Uploaded apps to manual testing now show the upload date and time making it easy to select the latest uploaded app for testing.

    Manual testing timestamp
  • Filtering of tests inside of test run view. To make it easier to debug the cause of failures in apps, it’s now possible to filter device runs to show only those where tests have failed. It is possible to have multiple filters enabled at a time making it possible to filter out failed devices and look only at successful test runs.

    Test filters
  • The documentation to integrate JIRA with BitBar Testing is updated and can be found in docs atIntegrations/jira

  • Big updates in BitBar image recognition samples onGitHub samples. In addition to library updates, there are a lot of new improvements. iOS screenshots speed was dramatically improved by changing screenshots mechanism to use idevicescreenshot. Image recognition does not need the testdroid-appium-driver anymore, but uses the native Appium driver.

Bug fixes
  • Improvements in iOS server side test runs. There were multiple reasons with this of which OSX upgrade was one. Work on further stabilization is in progress.

  • Appium client side runs were terminated if a test could not find some element. This is now fixed.

New features

Single Sign On (SSO) using Google accounts has been enabled to public cloud. This is especially useful for teams and enterprise customers using Google mail services. It is enough to invite a co-worker to own project in BitBar Testing cloud after which the invited user is able to SSO and instantly start contributing in testing or debugging of mobile apps.

Bug fixes
  • Device run logs widget fixed.

  • Improvements to iOS test run stability. There are still issues but improvements are being rolled out in phases.

New features

Dedicated devices - a new service for getting customers devices connected to our public cloud but also available to all other clouds. Use BitBar Testing’s large device range for manual and automated testing, but run regression and continuous integration tests on named devices hosted and managed by BitBar. To get and pick the dedicated devices, please get in contact with sales@bitbar.com.

Enterprise customers can use this functionality to create device pools visible and accessible only to particular teams.

Known issues
  • Since the past week there have been issues in running tests on our iOS devices. We are investigating the issue and are fighting to get it fixed. We are really sad about this.

  • Testdroid Cloud supports video recording of Android test runs. This is a premium feature that is available for Team and Business plans. To enable video recording of your test runs, please contact your named service representative or sales@bitbar.com. Support for iOS test run recording is coming shortly.

  • AppCrawler quick project creator includes credential fields. Instead of having to go through creating a new AppCrawler test run from the create test run menu, it’s now possible to include app credentials also in the quick AppCrawler creator.

    AppCrawler
  • Enterprise customers get yet more admin capabilities to manage their dedicated device cloud. Admins can now easily create teams inside of Testdroid Cloud which allows reporting per team based usage statistics. Getting an answer to which team uses device cloud most is then straight forward.

  • Find the user that needs to be added as sub user and on user information page click on the button to set main user

    Sub-user button
  • It’s then possible to select the main user

    Converting the main user to a sub-user
  • Enterprise users can define how many times test runs get retried or not. There’s now a new API call that can be used for changing the number of times that a test is retried. There are multiple cases when a test should not be re-run even if there’s a problem with the test or device. If you have an Enterprise or Private cloud setup please be in touch with your customer success contact person to get more information about this.

A small back end update focused on upgrading the Appium version used in server side test runs. Default for server side runs is still Appium 1.4.16. Modifying the run tests script run-tests.sh, allows using of Appium version 1.5.2.

This is the old approach of starting Appium in the script.

# on iOS
/opt/appium/bin/appium.js -U ${UDID} --command-timeout 120 >appium.log 2>&1 &

# on Android
/opt/appium/appium/bin/appium.js >appium.log 2>&1 &

To use Appium 1.5, the above lines need to be updated to as below. It is suggested to update the script even if older Appium version is used. The launch procedure has been improved with this new approach.

# on iOS
#appium-1.4 -U ${UDID} --command-timeout 120 >appium.log 2>&1 &
appium-1.5 -U ${UDID} --command-timeout 120 >appium.log 2>&1 &

# on Android
#appium-1.4 -U ${UDID} >appium.log 2>&1 &
appium-1.5 -U ${UDID} >appium.log 2>&1 &

Due to how Appium is started there is no more a need for a constant sleep after the launch. The following line should be commented out in the run-tests.sh as unnecessary.

sleep 20 # Sleep for appium to launch properly #<- this is not needed anymore

For now Appium client side runs continue to use older Appium 1.4 version.

  • New project types for server side runs available to all users with SOLO plan or up. To create a server side Appium project, simply select Appium iOS or Android Server side project type from the project drop down list.

    Server-side project
  • Appium client side testing picks similar available device by default

We updated our client side Appium to work in a way where the user does not need to provide an exact device name. The Appium client will search for a device that matches the requested device but does not guarantee that the requested device is used. The used device is visible in Testdroid Cloud test run view.

For example by passing “iPhone” as testdroid_device capability name, Testdroid Appium client will pick any of the available iPhone devices, regardless of iOS or iPhone version. The test could be run on an iPhone 5s or iPhone 6.

If the user wants to turn off this feature and use a specific device then the testdroid_findDevice Appium desired capability needs to be set to True like in below Python example. By default this is False.

```
desired_capabilities_cloud['testdroid_findDevice'] = True
```

  • Enterprise cloud admin users are now able to manage project ownership through the admin panel. Go to Admin - Projects, select the project that you want to move to other user and click on the “Project owner” icon.

    Project owner

    Click the image to enlarge it.

    You can then change the owner by selecting new user from the list.

    Project owner
  • Support for iOS 9.3.x devices using XCode 7.3.

  • Python version 3.5.1 is available for server side runs. To use it, just call your Python scripts with python3 your_test.py.

  • A touch or personalization is always fun. Testdroid Cloud now supports avatars through the Gravatar service.

New features
  • AppCrawler - Testdroid Cloud has had since its very early days an automated app tester that crawls through an app, checking for correct behavior. AppCrawler can be accessed directly from the main menu, or by creating a new project. When creating a new project the user has more options on guiding AppCrawler forward.

    AppCrawler

    Click the image to enlarge it.

To smoke test an application the app needs to be uploaded, then choose project to include the results or create a new one and select the number of devices the app needs to be tested on. If device groups are defined then these can also be used. Clicking ‘Go’ starts the test run. Free users are limited to running on only one free device at a time.

  • The view to read and debug a test run’s test steps or error logs can now be enlarged for better clarity. Once the view enlarged it’s easy to use the search box to find keywords. Of course browser’s own search is also supported.

    Expanded steps

    Click the image to enlarge it.

    Expanded errors

    Click the image to enlarge it.

  • A better support for presenting results as test steps for users using Cucumber with Appium.

Bug fixes
  • Results processing in some cases was taking a long time or even failed. Test results processing problem was solved and solution improved.

New features
  • Enterprise and Private cloud administrators get a new menu element for managing used test frameworks. This gives Testdroid Cloud administrators the possibility of sharing test projects configurations between teams and modify the configurations of a specific test project.

  • Calabash test framework is updated to latest version in Enterprise Clouds. Available versions are 0.19.0 for iOS and 0.7.3 for Android Calabash.

  • Users are now able to cancel a test run with multiple devices and keep the results for the finished devices. Earlier when canceling a test run, the execution and results processing was canceled for all devices. Even those that had already finished test execution. With this release cancellation cancels only tests for those devices that have not yet started or finished running the tests. Results are presented for device runs that already have results.

    Aborting a test run

    Click the image to enlarge it.

  • Throughout the spring there has been updates to better visualize the test results. The last part (for now) comes to end with the updates to the device run view. The view has changed to give a better overall view of the device run. The page is divided in four. First view lets the user to switch which device from the run is inspected and present general relevant information on that device run.

    Device test run view steps

    Click the image to enlarge it.

Logs are in their own widget. Different logs have their own syntax highlighting and everything can be text searched. Switching between various gathered logs is easy. The test framework specific logs are ighlighted (eg. Appium and Calabash) and additional logs can be selected from the drop down.

Device run view logs

Click the image to enlarge it.

Known issues
  • There is an issue with post processing of some test results in our Public Cloud. This release fixes part of the problem but there can still be issues with some runs. A robust solution is being implemented. More on this when final solution is coming out.

Bug fixes
  • Getting always the same device from device group. Some customers had an issue when running sequential tests on a device group where the test + first device configuration failed. In future the devices from a selected group are proposed in a randomized order to increase success of sequential test runs.

Release has mostly changes important to Enterprise Cloud users and internal to Testdroid Cloud handling of project related data. More on this with the upcoming releases, with more visible changes.

New features
  • Jenkins plugin, BitBar TestingRun in Cloud Jenkins Plugin was published to Jenkins Wiki. Also an update release (1.0.12) was made available and can be installed through the Jenkins Manager.

    Jenkins manager

    Click the image to enlarge it.

  • Improved the presentation of test steps in View Screenshots by Test Step view. This functionality is available for Calabash users but also for AppCrawler (automated test runner) and Android UiAutomation tests. Now the screenshot comparison view presents erroneous steps and devices.

The comparison view also got some new icons next to device titles. These show whether there were errors in the tests or running the tests. As elsewhere when everything is good we give it green color, otherwise orange or red when things got really bad.

Shots by test steps

Click the image to enlarge it.

Bug fixes
  • Performance fixes to creating new device groups. There was an annoying bug when creating a new device group by dragging new devices to the “New group” icon. The group got created but the correct number of devices in the group was slow to get updated.

    Device groups

    Click the image to enlarge it.

Technical release mostly intended to fixing some back end issues. Release targeted for Enterprise installations.

New features
  • UI improvements to device run view. To ease access to the project information, like downloading log files, screenshots etc. we moved the project information up above the test run status view.

    New device run view
  • Bread crumbs to improve navigation. To help navigation inside of projects we added bread crumbs under top menu so it’s easy to navigate back to where you came from.

    Navigation
Bug fixes
  • Unable to use function buttons in manual testing: Using Firefox 44 some UI elements on the manual testing view did not react correctly.

  • Our resign-ipa script did not handle app Bundle display name with spaces correctly: Fixed.

Technical release with improvements for Private Cloud users and bug fixes.

New features
  • Improved usage details reporting.

Bug fixes
  • Changes to IBM Marketplace integration

  • Performance issues for administrator users (Enterprise and Private Clouds)

Release for Private and Enterprise installations in mind.

New features
  • Administration interface was upgraded. Administrators can now use the same look and feel throughout the whole service.

  • Integration to IBM Marketplace. IBM Marketplace users can now get started using Testdroid Cloud.

    TestDroid Cloud in IBM
  • Possibility to re-prioritize test runs in queue (Enterprise & Private Clouds). While an other team is running long regression tests reserving all the devices, you might want to prioritize your bug fix test run higher to get your fix tested. You can now do so by setting the priority of your test run in the admin panel of your Enterprise Cloud installation. Clicking the “edit pencil” admin is able to set the wanted priority to the test run.

    Reprioritizing test runs

See Also

What's New in BitBar
About BitBar

Highlight search results