Last modified on October 21, 2020

The instructions below describe how to create a test run with minimal requirements. Optionally, you can also configure advanced settings.

  1. Click Projects in the left-side menu, expand a project in the Projects section, and click Create your first test run, if the project has no test runs. Click Add new test run, if the project has test runs. Alternatively, click Create your first test run in the Last Test Runs section.

    Show image

    -- or --

    On the Dashboard, click Create Automated Test in the Automation Testing widget.

    Show image

    -- or --

    Select Automation > Create Automated Test on the left sidebar.

    Show image

  2. Select the target operating system in the Select a target OS type step:

    • iOS – Select for testing iOS applications.

    • Android – Select for testing Android applications.

    • Desktop – Select for browser testing.

  3. Depending on the previously selected operating system, select one of the available frameworks in the Select a framework step. Currently, the supported frameworks are:

    Note: Depending on your cloud setup, the names and avail able frameworks may differ from the ones listed below.

     iOS

    • Appium iOS Server Side

    • XCTest

    • XCUITest

    • AI Testbot iOS

    • Mobile Browser Testing

     Android

    • Android Instrumentation

    • Appium Android Server Side

    • AI Testbot

    • Mobile Browser Testing

  4. Upload an application file in the Choose files step. You can upload up to 3 files.

    Tip: If you do not have a test app file yet, click Use our samples to upload sample apps.

    Select what action to perform over the uploaded files:

    • Install on the device

    • Copy to the device storage

    • Use to run the test

  5. Select a device or a device group in the Choose devices step:

    • Use existing device group – Select this option if you want to use trial devices or have an existing device group.

    • Use chosen devices – Select this option if you want to choose devices from the list of available devices.

    • Run on currently idle devices – Select this option if you want to run your test on random devices that are not currently used.

  6. Click Create and run automated test.

Advanced settings

In addition to the basic settings described above, there are plenty of additional configuration options for the test run.

To open advanced settings, click the area under the Additional settings (optional) step.

Additional settings

Click the image to enlarge it.

Project name

The name of the project to store the test run. If the specified project name does not exist, a new project will be created. Omitting this option will create a new project with a default name, for example Project 1.

Test run name

The name of the test run that describes what you are testing with this test run, for example, a build number, fixed bug ID, date and time, and so on.

Language

The language to be set on the selected devices before starting the test run.

Test time-out period

The timeout of the test run start. The default value is 10 mins. Possible values are: none, 5, 10, 15, 20, 30, 60, and 120 minutes.

Scheduling

Select how and when to run tests on the selected devices: simultaneously, on one device at a time, or on available devices first only.

  • Simultaneously – The test is started on all available devices at the same time. If the devices are not currently available, the test will start when they become available.

  • One device at a time – The test is started sequentially on all of the selected devices.

  • First available device only – The test is run on the first available device of the selected device group.

Use test cases from

Applies to Android Instrumentation test runs. Select a test class of the package to be executed, if you do not want to run the whole test suite.

Test case options

The test case options to be included or excluded.

Test finished hook

As the test run is finished, it is possible to make a POST call to the specific URL at the end of the test run. Note that in addition to this hook URL, you can also use email or Slack integrations to get notified of finished test runs.

Screenshots configuration

By default, screenshots are stored on the device SD card at /sdcard/test-screenshots/. Here, you can change the storage location.

Custom test runner

The test runner to be used. The default value: android.test.InstrumentationTestRunner.

Test user credentials

The user name and password combination that should be used during the AppCrawler test run.

Custom test run parameters

Public cloud supports a number of Shell environment variables that are made available to each test run. These can be used for test case sharding or selecting an execution logic for runs.

For Espresso test sharding, variables numShards and shardIndex are supported out of the box. Use these the same way as you would locally.

Xcode-based test suites can be controlled with XCODE_SKIP_TESTING and XCODE_ONLY_TESTING keys.

  • XCODE_SKIP_TESTING should take the value of the -skip-testing command-line flag. This allows you to skip some named test case or class.

  • XCODE_ONLY_TESTING takes the value of the command-line flag -only-testing. This flag allows you to define whether to run a single test method or all tests from a test class.

In On-Premise and Private Cloud setups, users can create their own key-value pairs. For customers with advanced plans, it is possible to create keys for specific tasks to be done before, during, or after a test run also in Public Cloud.

See Also

Managing Test Runs

Highlight search results