Running Flutter Tests for iOS

iOS simulator

iOS simulator test runs spend actual real device test time, the real iOS device is just idling while the simulator is running tests. iOS tests are run on the Mac OS. Flutter is pre-installed on virtual machines. Cocoapods need to be installed, it can be done using the run-tests.sh file.

Create a shell script file called run-tests.sh. This file will contain commands to execute the test.

  1. Unzip test files:

    unzip tests.zip

  2. Install cocoapods:

    sudo gem install cocoapods
pod setup

  3. Run Flutter doctor:

    flutter doctor

  4. (Optional) Go to the flutter project directory (sample command):

    cd my_app

  5. (Optional) Clean the project folder (build/packages):

    flutter clean
rm -r .packages
rm pubspec.lock

  6. Install an iOS simulator:

    npm install -g ios-sim

  7. Start the simulator:

    ios-sim start --devicetypeid "iPhone-8"

  8. Find the simulator device from Flutter devices:

    flutter devices > devices.txt
cat devices.txt | while read line
do
case "$line" in
*simulator*)
  TEST_DEVICE=$line
  RUN_DEVICE=`echo ${TEST_DEVICE} | awk '{print $1, $2}'`
  echo $RUN_DEVICE > run_device.txt
  echo "device"
  cat run_device.txt
  ;;
  *)
  ;;
esac
done

  9. Run Flutter integration tests using the simulator (sample command):

    flutter drive -d "`cat run_device.txt`" --target=test_driver/main.dart

  10. (Optional) If test results (a file called TEST-all.xml) or screenshots should be displayed in the cloud UI:

    • Go back to the root directory (test result files are searched in the root directory):

      cd ...

    • Move TEST-all.xml to the root directory:

      mv my_app/TEST-all.xml TEST-all.xml

    • Create the screenshots directory and move screenshots there (sample command):

      mkdir -p screenshots
mv /tmp/screenshots/test/ screenshots

Create a zip file containing the run-tests.sh file and your Flutter project directory. Note that the “run-tests.sh” file must be in the root of the zip file, otherwise, it is not found. Here is a sample command to create a zip file: zip -r ios-test-files run-tests.sh my_app. Upload this file to the cloud and use it as the “Use to run the test” file.

Create a Flutter iOS simulator test run

To create a Flutter iOS simulator test run in BitBar Cloud:

  1. Create a Appium iOS Server Side-type test run.

    iOS test run
    Appium iOS Server Side test run

  2. Upload the test zip file and the actual app .ipa file or a small sample app .ipa file (it needs to be able to get installed to the actual real iOS device, no simulator builds), the test does not use this app.

    Uploading files

  3. Select devices (the actually selected device is not used to run tests on).

  4. Set a test timeout period. Make sure it is long enough.

  5. Run tests.

iOS real device

BitBar Cloud supports running tests created with the following versions:

  • Flutter 1.22.6

  • Flutter 2.2.0

Before you start your testing, you need to build your iOS Flutter application locally, and then upload it to the BitBar cloud. You should also remember to move the app to the required directory during the test run.

Use steps from the Flutter user guide on how to integrate a Flutter module into your iOS project to add debug functionalities to the debug version of your app.

You can prepare the package in two ways.

Option 1

Create a shell script file called run-tests.sh. This file will contain commands to execute the test.

  1. Open your project in Xcode. In the product menu, select Build for > Testing.

  2. Locate your output from the compilation in the build/ios/Debug-iphoneos directory.

  3. Prepare the .ipa file using the following steps in the command line interface.

    cd build/ios/Debug-iphoneos
mkdir Payload
mv Runner.app Payload/
zip --symlinks -qr application.ipa Payload

  4. Upload the app to the BitBar cloud with the default option Install on the device.

For detailed explanation, see how to create .ipa from .app.

Option 2

Run the integration test locally and archive your application:

  1. Test the execution example:

    flutter drive -v --target=test_driver/app.dart

    Note

    Remember that the target may be different and depends on your project naming convention.

  2. Archive your application using the following steps in the command-line interface.

    cd build/ios/iphoneos
mkdir Payload
mv Runner.app Payload/
zip --symlinks -qr application.ipa Payload

  3. Upload the app to the BitBar cloud.

Prepare test.zip

Create a shell script file called run-tests.sh. This file contains commands to execute the test. Also, there are two Flutter versions pre-installed inside a VM. You should specify the version you want to use in the run-tests.shfile. See the examples of run-tests.sh:

  • Flutter 1.22.6

    Use the --no-build value to flutter drive command execution.

    #!/bin/bash#echo 'Extracting tests.zip...'
yes | unzip tests.zip

mv application.ipa application.zip

unzip application.zip

mv Payload/Runner.app Runner.app

# make sure build folder exists
mkdir -p build/ios/iphoneos

# move app to build folder
mv Runner.app build/ios/iphoneos/Runner.app

echo 'starting flutter doctor ...'
flutter-1.22.6 doctor

echo 'Flutter devices ...'
flutter-1.22.6 devices

# Install flutter packages
flutter-1.22.6 pub get

flutter-1.22.6 drive -v --no-build --target=test_driver/app.dart

  • Flutter 2.2.0

    Use the --use-application-binary option to point to your Runner.app application:

    #!/bin/bash#echo 'Extracting tests.zip...'
yes | unzip tests.zip

mv application.ipa application.zip

unzip application.zip

mv Payload/Runner.app Runner.app

# make sure build folder exists
mkdir -p build/ios/iphoneos

# move app to build folder
mv Runner.app build/ios/iphoneos/Runner.app

echo 'starting flutter doctor ...'
flutter-2.2.0 doctor

flutter-2.2.0 devices

# Install flutter packages
flutter-2.2.0 pub get

flutter-2.2.0 drive -v --use-application-binary build/ios/iphoneos/Runner.app --target=test_driver/app.dart

  • Flutter 3.3.8

    The 3.3.8 Flutter version integration_test rebuilds an application from scratch. In the workspace, include all the needed dependencies to build the app for the chosen OS type. For more information, refer to the Flutter integration testing documentation page.

    #!/bin/bash#echo 'Extracting tests.zip...'
yes | unzip tests.zip

mv application.ipa application.zip

unzip application.zip

mv Payload/Runner.app Runner.app

# make sure build folder exists
mkdir -p build/ios/iphoneos

# move app to build folder
mv Runner.app build/ios/iphoneos/Runner.app

echo 'starting flutter doctor ...'
flutter-3.3.8 doctor

flutter-3.3.8 devices

# Install flutter packages
flutter-3.3.8 pub get

flutter-3.3.8 test integration_test/main.dart

Add the run-tests.sh to your Flutter project directory.

Create a tests.zip file containing run-tests.sh and the content of your Flutter project directory.

Important

Remember that therun-tests.sh file must be in the root of the zip file, otherwise, it is not found.

Ensure that you executed the flutter clean command before archive creation. See the example:

$ flutter clean
Cleaning Xcode workspace... 5.9s
Deleting build... 213ms
Deleting .dart_tool... 19ms
Deleting .packages... 2ms
Deleting Generated.xcconfig... 2ms
Deleting flutter_export_environment.sh... 2ms

The contents of the tests.zip archive should look something like this:

Test.zip contents

(Optional) View screenshots or TEST-all.xml file

If you want to display screenshots or the file TEST-all.xml in the cloud UI, follow the steps:

  1. Create the screenshots directory and move there your prepared screenshots.

  2. Generate TEST-all.xml based on the console output of flutter drive execution.

    Note

    You need changes in run-tests.sh starting from flutter drive execution, but remember that everything above the test execution must remain unchanged. Here is an example of how you can achieve it:

    # show logs on the cosnole output and additionally create log file, for parsing purposes.
flutter-2.2.0 drive -v --use-application-binary build/ios/iphoneos/Runner.app --target=test_driver/app.dart 2>&1 | tee testconsole.log

flutterExitCode=$?5
while read line; do
case "$line" in
*setUpAll*)
TEST_CLASS=$line
;;
*)
;;
esac
case "'$line'" in
*All*tests*passed*)
TESTS_PASSED=true
;;
*)
;;
esac
case "'$line'" in
*Some*tests*failed*)
TESTS_PASSED=false
;;
*)
;;
esac
done <testconsole.log

# parse results to TEST-all.xml
CLASS_NAME_TEST=`echo ${TEST_CLASS} | awk -F"0m:" '/0m/{print $2}' | sed 's/[][]//g' | sed 's/(setUpAll)0m//'`

if [ "$TESTS_PASSED" = "true" ]
then
sed -i.bu "s/default-testsuitename/$CLASS_NAME_TEST/" TEST-pass.xml
sed -i.bu "s/default-testcase-class/$CLASS_NAME_TEST/" TEST-pass.xml
sed -i.bu "s/default-testcase-1/All tests passed/" TEST-pass.xml
sed -i.bu "s/default-testcase-result/All tests passed/" TEST-pass.xml
mv TEST-pass.xml TEST-all.xml
else
sed -i.bu "s/default-testsuitename/$CLASS_NAME_TEST/" TEST-fail.xml
sed -i.bu "s/default-testcase-class/$CLASS_NAME_TEST/" TEST-fail.xml
sed -i.bu "s/default-testcase-1/Some tests failed/" TEST-fail.xml
sed -i.bu "s/default-testcase-result/Some tests failed/" TEST-fail.xml
for i in {1..4}
do
sed -i.bu "s/0/1/" TEST-fail.xml
done
mv TEST-fail.xml TEST-all.xml
fi

exit $flutterExitCode

Create a Flutter iOS real device test run

To create a Flutter iOS real device test run in BitBar cloud, follow these steps:

  1. Create the Flutter iOS test run.

    iOS test run
    Appium iOS Server Side test run

  2. Upload a test.zip file and the application.ipa file to the BitBar cloud and select them:

    Uploading files

  3. Select devices to run test on.

  4. Click Create and run automated test.

See Also

In this section:
© 2024
Publication date: