Preparing for Testing

Test planning checklist

Plan your testing process before you start testing:

  • Define the test goal

    Decide which application functionality you want to test. The clearer the goal and the simpler the test, the better. Large tests that deal with various aspects of the application’s behavior are difficult to create and maintain. We recommend that you create a simple test that will test only certain functionality of your application.

  • Plan testing steps

    Decide which actions the test will perform. The testing steps depend on the test purpose and the nature of the application under test. Testing steps may include actions that prepare the application for the test (that is, they put the application to some initial state), or that enter some data in the application.

  • Plan checking actions

    Usually, the actions a user performs over an application cause some changes in the application. These can be visual changes (new values in forms, new windows or dialogs are displayed), or changes in the application's data (new objects, database entries, new or deleted files). You should decide what criteria should be used to check whether a test has passed or failed.

  • Log the results

    Make sure to save the full test results to a file or track the latest test run data. See Analyzing Results for details.

App requirements

The maximum app size that a user can upload is 4096 MB.

Create iOS app package

Regardless of whether testing is performed with XCTest or XCUITest, the application under test needs to be built and imported to BitBar Testing Cloud. The below steps present how to create the appropriate Ad Hoc Distribution IPA build.

Create test IPA

  1. Open your project in Xcode. In the Xcode Product menu, select Build for, and then select Testing.

    Creating a test .ipa

  2. The application still needs to be packaged into an IPA zip. For this, the location of the app is needed. In Xcode, select the built .app file, right-click it, and select Show it in Finder.

    Creating a test .ipa

  3. In Finder, right-click the .app file and copy its location for later use. This location does not change from build to build so this step needs to be performed only once.

    Creating a test .ipa

  4. The IPA package needed for XCTest in BitBar Testing is a zip package and its creation can be automated and made part of a CI integration. Open a terminal and create the IPA.

    $ mkdir /tmp/Payload
$ cd /tmp/Payload
$ cp -r /Users/username/Desktop/Build/Products/Debug-iphoneos/LocalizationDemo.app . # app path can be pasted
$ cd ..
$ zip --symlinks -qr "LocalizationDemo.ipa" Payload
$ ls -lrt LocalizationDemo.ipa
-rw-r--r-- 1 username staff 0 Dec 16 12:42 LocalizationDemo.ipa

Note on signing iOS apps

We re-sign iOS applications so that they could run on BitBar Cloud. By doing so, we remove support for the following capabilities:

  1. Access WiFi Information

  2. App Attest

  3. App Groups

  4. Apple Pay Payment Processing

  5. Associated Domains

  6. AutoFill Credential Provider

  7. ClassKit

  8. Communication Notifications

  9. Custom Network Protocol

  10. Data Protection

  11. Extended Virtual Address Space

  12. Family Controls

  13. FileProvider TestingMode

  14. Fonts

  15. Game Center

  16. Group Activities

  17. HealthKit

  18. HealthKit Estimate Recalibration

  19. HLS Interstitial Previews

  20. HomeKit

  21. Hotspot

  22. iCloud

  23. In-App Purchase

  24. Increased Memory Limit

  25. Inter-App Audio

  26. Low Latency HLS

  27. Mac Catalyst (Existing Apps Only)

  28. Maps

  29. MDM Managed Associated Domains

  30. Multipath

  31. Network Extensions

  32. NFC Tag Reading

  33. Personal VPN

  34. Push Notifications

  35. Sign In with Apple

  36. SiriKit

  37. System Extension

  38. Time Sensitive Notifications

  39. User Management

  40. Wallet

  41. Wireless Accessory Configuration

The removed capabilities are replaced with the following ones:

  1. Data Protection

  2. iCloud

  3. Inter-App Audio

  4. Maps

  5. NetworkExtensions

  6. SiriKit

  7. Wallet

We can also handle the process in other way. We can omit re-signing within BitBar Private Cloud setups or with dedicated devices in BitBar Public Cloud. This way, you have the possibility to use your own distribution certificate.

If you are interested in one of these solutions, contact us.

Symbolication

Symbolication is the process of translating iOS application crash logs to a human-readable file using the dSYM file. dSYM or “debug symbols file” is a file generated when building the application when the Strip Debug Symbols setting in Xcode is set to true. Without the dSYM file, any crash logs generated by a failing application will be unreadable by end users. If the dSYM file is part of the uploaded application package, then BitBar automatically processes the crash logs for the following frameworks: iOS XCUITEST, iOS Appium Client Side, iOS Appium Server Side.

The crash report is collected from the iOS device, it is symbolicated and sent to BitBar Cloud - where it is made available via UI or API as a file named symbolicated.crash.

Good to know

Armv7

If building for iPhone 5 or iPhone 5C (having the Armv7 32-bit processor) devices, then an additional step is needed before creating the build. Starting from Xcode7 onwards, armv7s is no more part of the default$(ARCHS_STANDARD) and so should be added as target build architecture.

Xcode Armv7 build option

iOS 9.3.5

By default, Xcode sets the iOS target version to the latest available iOS version for new projects. For testing against devices running previous iOS versions, the app and the tests need to be built for these separately.

For example, the project-level Deployment Target can be set to 9.3.

Deployment target version

Then, the same needs to be done for the app target (Calculator in these example screenshots)

Deployment info

and still for the tests (CalculatorUITests).

iOS Deployment Target

See Also

In this section:
© 2024
Publication date: