Applies to BitBar SaaS, last modified on September 17, 2020

Cloud-side Appium testing is when you upload your existing Appium tests to BitBar Cloud and then, either using the API or the cloud UI, select and start test execution on multiple devices. The benefits of this approach are that you do not need to modify your Appium tests, the execution of tests can be parallelized on multiple devices, and you are informed when the execution is finished.

The below sample code can be downloaded from our git repository. Here, we use Python. You can also find test suites in other supported languages like Java, Ruby, C#.

Run the sample in the cloud

Zip the tests

All the files needed for testing should be uploaded to the cloud as a zip package. This package must contain test files, data files and needs to have a Shell script to launch test execution at the root level of the package. So, there are a few easy steps to be taken before you can upload your current tests to the cloud for the Server Side execution. Find them below.

Note that this is different from the Client-Side execution. To learn more, see Running Client-Side Appium tests.
1. Prepare your app

One important file is of course your application. This needs to be uploaded alongside the test suite zip. Make sure it is an application made for testing. If it is an .ipa application, you can read about the packaging here.

2. Important files

For the cloud-side execution to work properly, it is important to have the test suite zip with the right files. Here, we will go through the files and also the correct makeup of the test zip. The most important is the run-tests.sh file, it is responsible for starting the test execution in the cloud and also installing requirements. Depending on whether you are testing iOS or Android, you’ll need to rename run-tests_android.sh or run-tests_ios.sh to run-tests.sh considering the environment you want to test.

In the run-tests.sh file, make sure that the ${TEST} variable points to the correct file name. This is the name of the main Python test script file. Other modifications are not needed. In this python example, we use the BitBar Sample Test script, so our TEST variable is the following:

TEST=${TEST:="BitbarSampleAppTest.py"} #Name of the test file

For the sake of this example, let’s decide we want to run an Android test, so we need to rename run-tests_android.sh to run-tests.sh.

$ cp run-tests_android.sh run-tests.sh

Sample zip contents (Python)

run-tests.sh
BitbarSampleAppTest.py
requirements.txt

  • run-tests.sh – This file is responsible for starting the test execution in the cloud.

  • BitbarSampleAppTest.py – The actual test files written in Python.

  • requirements.txt – Lists the required Python packages that need to be installed for the test to be executable, e.g. AppiumPythonClient or RobotFramework.

You can find Python sample files here:

https://github.com/bitbar/test-samples/tree/master/samples/testing-frameworks/appium/server-side/python

Sample zip contents (Java)

run-tests.sh
Pom.xml
src/ folder

  • run-tests.sh – This file is responsible for starting the test execution in the cloud.

  • Pom.xml – The Maven configuration file.

  • src/ folder – The folder containing tests and other test files, note that the test files themselves are not at the root level.

You can find Java sample files here:

https://github.com/bitbar/bitbar-samples/tree/master/samples/testing-frameworks/appium/client-side/java

README has a more detailed explanation of the repository structure and the way to use it.

3. Create your test suite zip file

The above python sample has the following content:

$ ls -lrt
-rwxr-xr-x 1 user staff   60 Feb 28 16:31 requirements.txt
-rw-r--r-- 1 user staff 4667 Mar 9 15:38 TestdroidAppiumTest.py
-rw-r--r-- 1 user staff 2687 Mar 13 11:04 BitbarSampleAppTest.py
-rwxr-xr-x 1 user staff 1263 Mar 14 16:10 run-tests_ios.sh
-rwxr-xr-x 1 user staff 1382 Mar 14 16:11 run-tests_android.sh

Now, you can create the test zip (called test.zip here) for uploading as follows:

$ zip -r tests.zip *
  adding: BitbarSampleAppTest.py (deflated 67%)
  adding: TestdroidAppiumTest.py (deflated 70%)
  adding: requirements.txt (deflated 2%)
  adding: run-tests.sh (deflated 47%)
  adding: run-tests_android.sh (deflated 47%)
  adding: run-tests_ios.sh (deflated 44%)

run-tests.sh needs to always be at the root of the package, and it is copied and renamed from run-tests_android.sh.
4. Upload and Execute

Now, it is time to use your newly created files and start a test run. Navigate to cloud.bitbar.com and log in with your credentials.

  • Start by selecting Test Run Creator or create a New Project in the project view.

  • Create a new test run for this project.

  • Upload the tested application (apk/ipa) at the Application step.

  • Upload the above-created zip using the Upload test file step.

  • Choose the device group to use or create a new group for this run.

  • Select optional advanced options.

  • Start your test run.

Once the test execution has finished, test results become available in the cloud.

Additional considerations

To get all your test artifacts presented in BitBar Cloud, you should take the following additional steps into consideration:

  • To get screenshots and test results visible in the cloud, make sure your test scripts store screenshots in the screenshots directory at the root of the test files directory.

  • Also, if your tests create JUnit result files, name them as TEST-all.xml and locate it at the root of the test files directory.

After a test run of this sample test, the output should look as follows:

$ ls -lrt
-rw-rw-r-- 1 user staff    4667 Jun 8 01:05 TestdroidAppiumTest.py
-rwxrwxr-x 1 user staff    1263 Jun 8 01:05 run-tests_ios.sh
-rwxrwxr-x 1 user staff    1382 Jun 8 01:05 run-tests_android.sh
-rwxr-xr-x 1 user staff      60 Jun 8 01:05 requirements.txt
-rw-rw-r-- 1 user staff    2687 Jun 8 01:05 BitbarSampleAppTest.py
-rwxrwxr-x 1 user staff    1382 Jun 8 01:06 run-tests.sh
-rw-rw-r-- 1 user staff 1595408 Jun 8 01:09 get-pip.py
-rw-rw-r-- 1 user staff    5572 Jun 8 01:09 TestdroidAppiumTest.pyc
drwxrwxr-x 2 user staff    4096 Jun 8 01:09 screenshots
-rw-rw-r-- 1 user staff    3129 Jun 8 01:09 TEST-all.xml
drwxrwxr-x 2 user staff    4096 Jun 8 01:09 test-reports

Run the Cloud sample locally

The changes needed to make local tests run in BitBar Cloud are minor. Therefore, it is possible to execute the cloud test zip locally as well.

This example works for both Android and iOS devices. Depending on the device you want to use, rename run-tests_android.sh or run-tests_ios.sh to run-tests.sh. To get started, you need to have cURL, Unzip, Appium, and ADB for Android or Xcode for iOS installed on your test machine. The rest of the required dependencies are listed in the requirements.txt file of this sample directory. run-tests.sh takes care of installing the remaining tools and libraries. To learn more about how to set up the Environment, see Environment Setup.

Change Settings

  • In run-tests.sh, remove or comment out the zip extraction, requirements, installations, and starting of the Appium server as these are meant for the cloud only, or parametrize to fit your local environment needs. The cloud dependencies can be found between the lines:

    ##### Cloud testrun dependencies start

    ##### Cloud testrun dependencies end.

  • Launch the Appium server as usual when running local Appium tests. Appium for iOS needs the device ID in launch parameters.

  • run-tests.sh assumes the tested apk/ipa file is named as application.apk or application.ipa and is located at the root of the working directory. Alternatively, update run-tests.sh to use some other path.

  • Launch tests by running the run-tests.sh script.

Tips and tricks

Use the samples as a base for your test suite, this way, you already have the correct folder makeup, replace or modify test files according to your needs.

See Also

Appium Support

Highlight search results