Running Server-Side Appium Tests
Server-side Appium testing is when you upload your existing Appium tests to BitBar Cloud and then, either using the API or the cloud UI, select and start test execution on multiple devices. The benefits of this approach are that you do not need to modify your Appium tests, the execution of tests can be parallelized on multiple devices, and you are informed when the execution is finished.
The below sample code can be downloaded from our git repository. Here, we use Python. You can also find test suites in other supported languages like Java and Ruby. For C# we suggest running client-side execution.
Run the sample in the cloud
Zip the tests
All the files needed for testing should be uploaded to the cloud as a zip package. This package must contain test files, data files and needs to have a Shell script to launch test execution at the root level of the package. So, there are a few easy steps to be taken before you can upload your current tests to the cloud for the Server Side execution. Find them below.
Important
Note that this is different from the Client-Side execution. To learn more, see Running Client-Side Appium Tests.
1. Prepare your app
One important file is of course your application. This needs to be uploaded alongside the test suite zip. Make sure it is an application made for testing. If it is an .ipa application, you can read about the packaging here.
2. Important files
For the server-side execution to work properly, it is important to have the test suite zip with the right files. Here, we will go through the files and also the correct makeup of the test zip. The most important is the run-tests.sh file, it is responsible for starting the test execution in the cloud and also installing requirements. Depending on whether you are testing iOS or Android, you’ll need to rename run-tests_android.sh or run-tests_ios.sh to run-tests.sh considering the environment you want to test.
In the run-tests.sh file, make sure that the ${TEST} variable points to the correct file name. This is the name of the main Python test script file. Other modifications are not needed. In this python example, we use the BitBar Sample Test script, so our TEST variable is the following:
TEST=${TEST:="BitbarSampleAppTest.py"} #Name of the test file
For the sake of this example, let’s decide we want to run an Android test, so we need to rename run-tests_android.sh to run-tests.sh.
$ cp run-tests_android.sh run-tests.sh
Specifying Appium Versions
In your test script, you have the flexibility to select a particular Appium version by using appium-1.xx which corresponds to Appium 1.xx. If you use appium alone, it will execute with our default Appium version, presently set at Appium 2.0.1. This feature is only available for Android devices.
Sample zip contents (Python)
run-tests.sh BitbarSampleAppTest.py requirements.txt
run-tests.sh– This file is responsible for starting the test execution in the cloud.BitbarSampleAppTest.py– The actual test files written in Python.requirements.txt– Lists the required Python packages that need to be installed for the test to be executable, e.g. AppiumPythonClient or RobotFramework.
You can find Python sample files here:
Sample zip contents (Java)
run-tests.sh Pom.xml src/ folder
run-tests.sh– This file is responsible for starting the test execution in the cloud.Pom.xml– The Maven configuration file.src/ folder– The folder containing tests and other test files, note that the test files themselves are not at the root level.
You can find Java sample files here:
README has a more detailed explanation of the repository structure and the way to use it.
3. Create your test suite zip file
The above python sample has the following content:
$ ls -lrt -rwxr-xr-x 1 user staff 60 Feb 28 16:31 requirements.txt -rw-r--r-- 1 user staff 4667 Mar 9 15:38 TestdroidAppiumTest.py -rw-r--r-- 1 user staff 2687 Mar 13 11:04 BitbarSampleAppTest.py -rwxr-xr-x 1 user staff 1263 Mar 14 16:10 run-tests_ios.sh -rwxr-xr-x 1 user staff 1382 Mar 14 16:11 run-tests_android.sh
Now, you can create the test zip (called test.zip here) for uploading as follows:
$ zip -r tests.zip * adding: BitbarSampleAppTest.py (deflated 67%) adding: TestdroidAppiumTest.py (deflated 70%) adding: requirements.txt (deflated 2%) adding: run-tests.sh (deflated 47%) adding: run-tests_android.sh (deflated 47%) adding: run-tests_ios.sh (deflated 44%)
Important
run-tests.sh needs to always be at the root of the package, and it is copied and renamed from run-tests_android.sh.
4. Upload and Execute
Now, it is time to use your newly created files and start a test run. Navigate to cloud.bitbar.com and log in with your credentials.
Start by selecting Test Run Creator or create a New Project in the project view.
Create a new test run for this project.
Upload the tested application (apk/ipa) at the Application step.
Upload the above-created zip using the Upload test file step.
Choose the device group to use or create a new group for this run.
Select optional advanced options.
Start your test run.
Once the test execution has finished, test results become available in the cloud.
Additional considerations
To get all your test artifacts presented in BitBar Cloud, you should take the following additional steps into consideration:
To get screenshots and test results visible in the cloud, make sure your test scripts store screenshots in the screenshots directory at the root of the test files directory.
Also, if your tests create JUnit result files, name them as
TEST-all.xmland locate it at the root of the test files directory.
After a test run of this sample test, the output should look as follows:
$ ls -lrt -rw-rw-r-- 1 user staff 4667 Jun 8 01:05 TestdroidAppiumTest.py -rwxrwxr-x 1 user staff 1263 Jun 8 01:05 run-tests_ios.sh -rwxrwxr-x 1 user staff 1382 Jun 8 01:05 run-tests_android.sh -rwxr-xr-x 1 user staff 60 Jun 8 01:05 requirements.txt -rw-rw-r-- 1 user staff 2687 Jun 8 01:05 BitbarSampleAppTest.py -rwxrwxr-x 1 user staff 1382 Jun 8 01:06 run-tests.sh -rw-rw-r-- 1 user staff 1595408 Jun 8 01:09 get-pip.py -rw-rw-r-- 1 user staff 5572 Jun 8 01:09 TestdroidAppiumTest.pyc drwxrwxr-x 2 user staff 4096 Jun 8 01:09 screenshots -rw-rw-r-- 1 user staff 3129 Jun 8 01:09 TEST-all.xml drwxrwxr-x 2 user staff 4096 Jun 8 01:09 test-reports
Run the Cloud sample locally
The changes needed to make local tests run in BitBar Cloud are minor. Therefore, it is possible to execute the cloud test zip locally as well.
This example works for both Android and iOS devices. Depending on the device you want to use, rename run-tests_android.sh or run-tests_ios.sh to run-tests.sh. To get started, you need to have cURL, Unzip, Appium, and ADB for Android or Xcode for iOS installed on your test machine. The rest of the required dependencies are listed in the requirements.txt file of this sample directory. run-tests.sh takes care of installing the remaining tools and libraries. To learn more about how to set up the Environment, see Environment Setup.
Change Settings
In
run-tests.sh, remove or comment out the zip extraction, requirements, installations, and starting of the Appium server as these are meant for the cloud only, or parametrize to fit your local environment needs. The cloud dependencies can be found between the lines:##### Cloud testrun dependencies start ##### Cloud testrun dependencies end.
Launch the Appium server as usual when running local Appium tests. Appium for iOS needs the device ID in launch parameters.
run-tests.shassumes the tested apk/ipa file is named asapplication.apkorapplication.ipaand is located at the root of the working directory. Alternatively, updaterun-tests.shto use some other path.Launch tests by running the
run-tests.shscript.
Tips and tricks
Use the samples as a base for your test suite, this way, you already have the correct folder makeup, replace or modify test files according to your needs.