Do we need to share the source code of the app in the Cloud?
Do we need to share the source code of the app in the Cloud?
Do we need to share the source code of the app in the Cloud?No you don’t unless you choose to use a test framework that specifically requires this.
How does BitBar manage the cleaning of devices?
How does BitBar manage the cleaning of devices?How does BitBar manage the cleaning of devices?Each phone is cleaned of apps that are not expected to be there.
Android browser
We do remove any browser app local data implying there will not be any navigation data on the next session. We also uninstall all apps that are possible to uninstall and remove files we know are created during test execution.
iOS browser
We rely on iOS’s own browser history cleaning services to revert Safari to an initial state.
Each device is put through a cleanup phase after each test run. Regardless of all these efforts, some files or data may stay hidden somewhere on the device in Public Cloud. For a truly private environment, we recommend Private or Enterprise Cloud installation.
What are the requirements for signing my iOS applications to use on BitBar’s iOS devices?
What are the requirements for signing my iOS applications to use on BitBar’s iOS devices?What are the requirements for signing my iOS applications to use on BitBar’s iOS devices?The app needs to be an ad-hoc distribution developer debug build. To learn more, see Preparing for testingYou can find more information here.
How do you support BitBar devices communicating with servers behind our firewall?
How do you support BitBar devices communicating with servers behind our firewall?How do you support BitBar devices communicating with servers behind our firewall?For Public Cloud, the only way is for the customer servers to put our public IPs on the allow list to establish connections from our devices. For private clouds or dedicated devices, we can implement a VPN or proxy connection. In on-premise setups, the customer has complete freedom of implementing networking as needed.
Do you support turning off and on Wi-Fi on BitBar devices?
Do you support turning off and on Wi-Fi on BitBar devices?Do you support turning off and on Wi-Fi on BitBar devices?This is not supported in Public Cloud. There is support for this in Private and Enterprise environments.
My tests require a Bluetooth (BLE) connection between the device and my external HW. What should I do?
My tests require a Bluetooth (BLE) connection between the device and my external HW. What should I do?My tests require a Bluetooth (BLE) connection between the device and my external HW. What should I do?For Public Cloud, you could try to mock the BLE connection data as required by the tests. Otherwise, we can pair devices to required Bluetooth devices in Private or Enterprise Cloud environments.
Do you use emulators or real devices?
Do you use emulators or real devices?Do you use emulators or real devices?Real devices only.
A lot of our content will not work outside the X country. How can this be solved in the Cloud?
A lot of our content will not work outside the X country. How can this be solved in the Cloud?A lot of our content will not work outside the X country. How can this be solved in the Cloud?For Public Cloud, the only way is for the customer servers to whitelist our public IPs allowing our devices to connect. For Private Cloud, we can implement a VPN or proxy connection. In Enterprise Cloud, it is up to the customer to define as the installation is in their own premise.
Do you support in-app purchases?
Do you support in-app purchases?Do you support in-app purchases?In Public Cloud, it is complicated to manage the Google account cache. If a test account can be used to log in and do the purchase completely inside the tested app, then we support it. For Private and Enterprise environments, we can set accounts controlled by the customer to enable such purchases.
Do you support testing of push notifications?
Do you support testing of push notifications?Do you support testing of push notifications?If the used test framework supports testing push notifications, then we can support them too. Triggering of push notifications is up to the customer to handle.
Can we change device settings on BitBar?
Can we change device settings on BitBar?Can we change device settings on BitBar?In Public Cloud, it is not allowed to access device settings at all. For Private and Enterprise environments, settings can be changed through remote access to the device.
What types of device performance data does BitBar provide for test runs?
What types of device performance data does BitBar provide for test runs?What types of device performance data does BitBar provide for test runs?We provide CPU and Memory usage data.
Can we do remote manual testing with BitBar devices?
Can we do remote manual testing with BitBar devices?Can we do remote manual testing with BitBar devices?Yes, for most Android and iOS devices. Some models are incompatible with the used technology. Manual testing on iOS devices is supported on Public Cloud through dedicated devices. Enterprise Cloud and Private Cloud users can decide which users get access to manual testing.
Does BitBar support parallel or concurrent test runs?
Does BitBar support parallel or concurrent test runs?Does BitBar support parallel or concurrent test runs?Yes. See Running cloud side Appium tests for details.
Can I use my own Google accounts on BitBar devices?
Can I use my own Google accounts on BitBar devices?Can I use my own Google accounts on BitBar devices?In Public Cloud, setting a Google account on the device isn’t allowed. Using a Google API/Service to log in to an app with your own account can be done. For Private and Enterprise environments, a customer’s own Google account can be set on the devices.
Which Google Play account is used on your devices?
Which Google Play account is used on your devices?Which Google Play account is used on your devices?In BitBar Public Cloud, a number of Google accounts are in use. These Google accounts may appear when running tests in BitBar Public Cloud:
-
play@bitbar.com
-
play-x@bitbar.com, where x is in 1…18
-
play-USx@bitbar.com, where x is either 1 or 2
-
play.us1.public@gmail.com
-
patchcordX@gmail.com, where X > 18
Where can I see a list of your devices? How often do you update it?
Where can I see a list of your devices? How often do you update it?Where can I see a list of your devices? How often do you update it?The complete list of devices with all details can be found in BitBar Cloud. We are adding new devices to increase device availability on existing models and add new models as they come out.
Can BitBar provide dedicated private devices for my testing?
Can BitBar provide dedicated private devices for my testing?Can BitBar provide dedicated private devices for my testing?Dedicated devices are available through Private Cloud installations or as part of Public Cloud. Private Cloud installation allows the customer to freely select the number and type of devices, and these are managed by BitBar. In Public Cloud, users can reserve one or multiple devices to their use only to assure device availability and security. For more information, please contact our Sales team.
Can I change or choose the OS on the devices? If not, how do you choose what OS should be on there?
Can I change or choose the OS on the devices? If not, how do you choose what OS should be on there?Can I change or choose the OS on the devices? If not, how do you choose what OS should be on there?No, devices have a certain version of OS and cannot be changed. We do the change when time goes on and if OEM releases relevant OS update. The purpose is to keep up with the actual use of certain OS versions globally and provide a versatile setup of different devices and OSs as possible.
Can the handsets under test receive an email via Wi-Fi?
Can the handsets under test receive an email via Wi-Fi?Can the handsets under test receive an email via Wi-Fi?Yes, but your application needs to be configured to receive emails. A regular email application on a device cannot be currently configured for sending/receiving emails.
Where can I find a free test trial, and how can I get started with BitBar Cloud?
Where can I find a free test trial, and how can I get started with BitBar Cloud?Where can I find a free test trial, and how can I get started with BitBar Cloud?You can create yourself a free account at bitbar.com. After leaving your email address, you’ll get an activation email with a link guiding you through the registration process. You’re now able to log in to BitBar Cloud using your new credentials and access our trial device group.
My test run failed on most devices. Why was that?
My test run failed on most devices. Why was that?My test run failed on most devices. Why was that?There are several reasons why test runs can fail on BitBar Cloud. The first and the most typical case is that there is something wrong with the application, and instrumentation makes it crash. A good rule of thumb is that if AppCrawler run crashes with your app, then the problem is in the app itself. Device problems are typically seen as pending test runs.
Are the devices jailbroken/rooted?
Are the devices jailbroken/rooted?Are the devices jailbroken/rooted?No. None of our devices is jail broken or rooted.
Our app requests Device Administrator privileges from the user. After the user grants the app Device Admin privileges, these privileges cannot be removed without entering a password first. It is conceivable that some tests may leave an app on the device. Does any of this cause a problem for your test environment?
Our app requests Device Administrator privileges from the user. After the user grants the app Device Admin privileges, these privileges cannot be removed without entering a password first. It is conceivable that some tests may leave an app on the device. Does any of this cause a problem for your test environment?Our app requests Device Administrator privileges from the user. After the user grants the app Device Admin privileges, these privileges cannot be removed without entering a password first. It is conceivable that some tests may leave an app on the device. Does any of this cause a problem for your test environment?No, this does not cause problems to our devices or environment. Our system automatically cleans, reboots, and hard resets devices before any new test run.
The app connects to our internal server. Our IT department will only allow connections from known IP address ranges. Is it possible for you to tell us what IP ranges are used to originate traffic from your test devices?
The app connects to our internal server. Our IT department will only allow connections from known IP address ranges. Is it possible for you to tell us what IP ranges are used to originate traffic from your test devices?The app connects to our internal server. Our IT department will only allow connections from known IP address ranges. Is it possible for you to tell us what IP ranges are used to originate traffic from your test devices?We have several public cloud data centers with the following IP address ranges:
- 205.234.8.112/28
- 157.25.46.232/29
- 157.25.61.24/29
- 157.25.63.200/29
Public Cloud users can also implement a test app to find the current IP of the used device/connection and communicate it to an external service that can open that IP for connections. For Private and Enterprise Cloud installations, most special network configurations are possible.
Which port ranges are used when running Appium client-side tests?
Which port ranges are used when running Appium client-side tests?Which port ranges are used when running Appium client-side tests?When you are running an Appium client-side test, your host will be making a connection to a dynamic IP address and a port range of 10010 to 10090.
Where are your devices located?
Where are your devices located?Where are your devices located?BitBar devices are located in our data centers in Mountain View, CA and Wrocław, Poland.
How safe are you?
How safe are you?How safe are you?All our user data, applications, and results are stored in Amazon Cloud and are very safe. We also use SSL and encrypt the data as seen appropriate. Only users with valid accounts can access their own data and results. We also track all IPs connecting to our BitBar.
How long is usually the queue? Does the priority group matter? Do I have to wait for 2 hours or a day?
How long is usually the queue? Does the priority group matter? Do I have to wait for 2 hours or a day?How long is usually the queue? Does the priority group matter? Do I have to wait for 2 hours or a day?The time between the test run start and delivering results depends entirely on the test case length, current queue, time of the day, and the desired device set. Typically, we’re talking about 2 to 15 minutes. Sometimes, it can be longer if you start multiple test runs on the same devices. We have multiple copies of devices at BitBar, so some devices can ensure results in a few minutes (depending on the test run length). The customers with higher priority plans (platinum > gold > silver) can get their app running first in the case of a queue.
How long will our projects be online?
How long will our projects be online?How long will our projects be online?We periodically clean unused projects and project files from our public cloud. For unused projects (where tests have not been run), after three months, the project files are removed. After this, the whole project is removed after the next three months.
Can I take pictures, sound recording (mic) with the phones?
Can I take pictures, sound recording (mic) with the phones?Can I take pictures, sound recording (mic) with the phones?Yes, you can. Our devices are not positioned for any specific photo target and recording of audio can give you arbitrary recording. However, our devices are fully functioning Android and iOS devices, and both mentioned functions are enabled on the devices.
What iOS test automation frameworks do you support?
What iOS test automation frameworks do you support?What iOS test automation frameworks do you support?We support KIF, XCTest, XCUITest, Jasmine, and Appium for iOS test automation with BitBar.
Is iOS Test based on UI Automation? Does this mean it can only test UIKit components?
Is iOS Test based on UI Automation? Does this mean it can only test UIKit components?Is iOS Test based on UI Automation? Does this mean it can only test UIKit components?As of the XCode8 release, UI Automation is no longer supported in BitBar testing Public Cloud, as Apple has dropped support for it.
How are you handling provisioning profiles to be able to test on devices?
How are you handling provisioning profiles to be able to test on devices?How are you handling provisioning profiles to be able to test on devices?We replace your provisioning profile with our provisioning profile. No need to root or jailbreak devices to get this done.
Keyboard input into password boxes is not visible. Why?
Keyboard input into password boxes is not visible. Why?Keyboard input into password boxes is not visible. Why?You can notice that when testing applications on devices using iOS version 13 or later. Once you put the insertion point into a password box, the screen keyboard disappears, and you cannot enter a value into that password box. The box remains blank when you are pasting a value into it or when you are typing text on your computer’s keyboard.
This happens due to the newer security policy these iOS versions use. The entered text is hidden from the video stream that BitBar receives from the device, so you see blank password boxes. Another effect of the newer security policy is that some security popups are also hidden.
You notice these issues when doing live testing and in the video recordings of automated tests.
To work around the issue in live testing, we recommend entering passwords with your computer’s keyboard.
See instructions
See instructionsSee instructions
-
Click within the password box to put the insertion point there.
-
Type the password on your computer’s keyboard, and press Enter.
It’s important to press Enter to confirm the input.
-
In the application, you can then click “Sign in”/“Log in” to continue.
As for automated tests, the issues don’t affect their functioning because these tests simulate user actions by calling specific methods of the objects they find in the tested app. The only issue is that user actions are absent from the video recordings of automated tests.
Can I test Cocos2d- or C++ OpenGL- based games?
Can I test Cocos2d- or C++ OpenGL- based games?Can I test Cocos2d- or C++ OpenGL- based games?Yes, you can. To get this done effectively, you have to think of what test automation framework fits your needs best and whether the framework will work with your app/game.
Is there support for automatically handling iOS dialogs?
Is there support for automatically handling iOS dialogs?Is there support for automatically handling iOS dialogs?Currently, handling iOS dialogs is performed through the used test framework. We have internal tools using which we are able to avoid and dismiss OS dialogs outside of test runs.
How can I start an iOS AppCrawlr run using BitBar RiC?
How can I start an iOS AppCrawlr run using BitBar RiC?How can I start an iOS AppCrawlr run using BitBar RiC?iOS AppCrawlr isn’t available at the moment in the Run-in-Cloud plugin.
How can I start an iOS AppCrawlr using API?
How can I start an iOS AppCrawlr using API?How can I start an iOS AppCrawlr using API?You can do it on an iOS project of the UI Automation type.
API Calls:
-
Get current project config: GET /api/v2/projects/{projectId}/config
-
Set Project Config: POST /api/v2/projects/{projectId}/config
projectId, {projectId}
mode, "IOS_CRAWLER"
usedDeviceGroupId, {deviceGroupId} (mandatory, or test run start fails)
-
Start new Test Run: POST /api/v2/runs
Can I cluster my BitBar Enterprise device set over various locations (e.g. 15 Android devices in the UK, 10 iOS in India)? And is that one installation?
Can I cluster my BitBar Enterprise device set over various locations (e.g. 15 Android devices in the UK, 10 iOS in India)? And is that one installation?Can I cluster my BitBar Enterprise device set over various locations (e.g. 15 Android devices in the UK, 10 iOS in India)? And is that one installation?Yes, of course. This depends on how you want to set up your environment. You can have one server connecting a certain device group (e.g. Android devices in the UK) and another one in India for iOS devices. Those two servers can be grouped together in BitBar Enterprise under your firewall, and results can be viewed anywhere in your premises/locations.
For more information about the BitBar Integration API, see BitBar API.
The Java source code and client can be found in the Github repository:
https://github.com/bitbar/testdroid-api
Do you have an API? Where can I find documentation on it? Can I integrate Jenkins or my system into it?
Do you have an API? Where can I find documentation on it? Can I integrate Jenkins or my system into it?Do you have an API? Where can I find documentation on it? Can I integrate Jenkins or my system into it?Yes, the documentation with the full API description can be found at BitBar API. You can integrate your CIs or any scripts with BitBar using the API’s JSON-based calls.
Do you provide Jenkins and other CI plugins for BitBar?
Do you provide Jenkins and other CI plugins for BitBar?Do you provide Jenkins and other CI plugins for BitBar?We provide a Jenkins plugin, which is available at JenkinsCI Github.
Where is your support located? What are the business hours? Is it email support or also phone support?
Where is your support located? What are the business hours? Is it email support or also phone support?Where is your support located? What are the business hours? Is it email support or also phone support?Our support is located in the USA, Finland, and Poland. We provide 24/5 (business days) support for all test runs on devices at BitBar as well as email answers for any questions related to BitBar products. Business hours – as defined in Pricing – apply to the European CET time zone 9 a.m. - 5 p.m.
Do you offer devices with pre-release OS versions of Android and iOS?
Do you offer devices with pre-release OS versions of Android and iOS?Do you offer devices with pre-release OS versions of Android and iOS?Not for regular app testing. Maintaining a pre-release OS is problematic as such releases can be very unstable and out of our hands to handle.
What is 3D Secure?
What is 3D Secure?What is 3D Secure?You can buy the BitBar service online by starting the registration process at BitBar.com.
Our service is purchased at cloud.bitbar.com using popular credit cards. For security reasons, we verify each purchases using 3D Secure (sometimes, also known as “Verified by Visa”, “Mastercard SecureCode”, or “American Express SafeKey”). In some cases, you may need to enable 3D Secure for your credit card if your bank does not support it by default.