• Local development: We will use Expo Go app together with Detox so that we can refresh the app and update our tests easily.
• CI pipeline: We will build standalone artefacts and use CircleCI MacOS and Android executors to run the same test suites

At the time of writing the team is using Expo 46 and Detox 19.

Set Up Detox

On your project, run the following command to install required dependencies

npm install --save-dev @config-plugins/detox detox@19.9.0 detox-expo-helpers jest

@config-plugins/detox is used to help generate detox compatible artifacts when expo runs the build.

detox-expo-helpers is needed when we use Expo Go locally to communicate with detox, more on this later.

jest is required because detox does not have its own test-runner.

Now, open app.json and add the @config-plugins/detox plugin to your plugins list (this must be done before prebuilding). This will automatically configure the Android native code to support Detox.

{
  "expo": {
    // ...
    "plugins": ["@config-plugins/detox"]
  }
}

Create Detox configuration files using the following command:

npx detox init -r jest

Add .detoxrc.json to project root dir. Note that ios-expo is to support local development

{
    "testRunner": "jest",
    "runnerConfig": "e2e/config.json",
    "skipLegacyWorkersInjection": true,
    "apps": {
        "ios": {
            "type": "ios.app",
            "binaryPath": "ios/build/Build/Products/Release-iphonesimulator/MyApp.app",
            "build": "eas build --platform ios --profile detox-test --local --clear-cache"
        },
        "android": {
            "type": "android.apk",
            "binaryPath": "android/app/build/outputs/apk/release/app-release.apk",
            "testBinaryPath": "android/app/build/outputs/apk/androidTest/release/app-release-androidTest.apk",
            "build": "eas build --platform android --profile detox-test --local --clear-cache"
        },
        // Local configuration
        "ios-expo": {
            "type": "ios.app",
            "binaryPath": "bin/Exponent.app"
        }
    },
    "devices": {
        "simulator": {
            "type": "ios.simulator",
            "device": {
                "type": "iPhone 14"
            }
        },
        "emulator": {
            "type": "android.emulator",
            "device": {
                "avdName": "pixel_4"
            }
        }
    },
    "configurations": {
        "ios": {
            "device": "simulator",
            "app": "ios"
        },
        "android": {
            "device": "emulator",
            "app": "android"
        },
        // Local configuration
        "ios-expo": {
            "device": "simulator",
            "app": "ios-expo"
        }
    }
}

Update eas.json to include detox test details

{
    // ...
    "build": {
        "detox-test": {
            "distribution": "internal",
            "ios": {
                "simulator": true
            },
            "android": {
                "gradleCommand": ":app:assembleRelease :app:assembleAndroidTest -DtestBuildType=release",
                "withoutCredentials": true
            }
        },
        // ...
    },
    // ...
}

Local Development

For local development we will use Expo app instead of building standalone apps because it will be faster to reflect changes and run tests by following TDD practice.

So in package.json we could add a helper command like following inside scripts section:

{
  // ...
  scripts: {
    // ...
    "test:detox": "DETOX_ENV=expo detox test -c ios-expo",
  }
}

And we also added a scripting file to help running the test easier

#!/bin/sh

EXPO_APP_DIR=bin/Exponent.app

if [ ! -d $EXPO_APP_DIR ]; then
    mkdir -p $EXPO_APP_DIR
    curl https://dpq5q02fu5f55.cloudfront.net/Exponent-2.28.7.tar.gz -o bin/expo-app.tar.gz
    tar xvzf bin/expo-app.tar.gz -C $EXPO_APP_DIR
fi

npm run test:detox

Because Expo app and standalone apps open and execute the tests in a different way, at the moment we choose to accommodate our tests by using an environment variable (We use expo for local development and standalone for running in CircleCI). Hopefully this will be improved in future. An example test would look like the following:

const { reloadApp } = require("detox-expo-helpers");

const detoxEnv = process.env.DETOX_ENV;

describe("Example", () => {
    beforeAll(async () => {
        if (detoxEnv === "standalone") {
            await device.launchApp();
        }
    });

    beforeEach(async () => {
        if (detoxEnv === "standalone") {
            await device.reloadReactNative();
        } else if (detoxEnv === "expo") {
            await reloadApp();
            await waitFor(element(by.id("loginMessage")))
                .toBeVisible()
                .withTimeout(20000);
        }
    });

    // test cases
    // ...
});

CircleCI pipeline

In CircleCI we will build the standalone apps first, start up the simulators then execute detox tests. By the time of writing, Android emulator is very unstable even with hardware acceleration on (see problems below), so we opt to run detox tests only on iOS simulator which is quite stable.

iOS step:

detox-test-ios:
    executor: ios
    steps:
      - attach_workspace:
          at: ~/
      - node/install:
          install-yarn: true
          node-version: '16.9.1'
      - run: echo -n $ARTIFACTORY_NPMRC | base64 -d  > ~/.npmrc
      - restore_cache:
          name: Restoring npm cache
          keys:
            - npm-ios-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}}-{{ checksum "package-lock.json" }} # Primary cache
            - npm-ios-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}} # Fallback cache
      - run:
          name: Install Apple sim utils
          command: |
            brew tap wix/brew
            brew install applesimutils
      - run:
          name: ios build
          command: |
            npm install -g expo-cli eas-cli detox-cli
            npm ci
            detox build -c ios
            mkdir -p ios/build/Build/Products/Release-iphonesimulator/
            mv *.tar.gz ios/build/Build/Products/Release-iphonesimulator/app-build.tar.gz
            cd ios/build/Build/Products/Release-iphonesimulator/ && tar xvzf app-build.tar.gz && cd -
      - run:
          name: ios test
          command: DETOX_ENV=standalone detox test -c ios --headless --take-screenshots failing --record-videos failing
      - store_artifacts:
          path: artifacts/
      - save_cache:
          name: Saving npm cache
          paths:
            - ~/.npm
          key: npm-ios-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}}-{{ checksum "package-lock.json" }}

Android step:

detox-test-android:
    machine:
      image: android:2023.02.1
    resource_class: xlarge
    steps:
      - attach_workspace:
          at: ~/
      - node/install:
          install-yarn: true
          node-version: '16.9.1'
      - run: echo -n $ARTIFACTORY_NPMRC | base64 -d  > ~/.npmrc
      - restore_cache:
          name: Restoring npm cache
          keys:
            - npm-android-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}}-{{ checksum "package-lock.json" }} # Primary cache
            - npm-android-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}} # Fallback cache
      - run: echo -n $PLAY_STORE_SERVICE_ACCOUNT_KEY | base64 -d  > serviceAccountKey.json            
      - run:
          name: Android build and test
          command: |
            sudo apt-get --quiet update --yes
            sudo apt-get --quiet install --yes \
              libc6 \
              libdbus-1-3 \
              libfontconfig1 \
              libgcc1 \
              libpulse0 \
              libtinfo5 \
              libx11-6 \
              libxcb1 \
              libxdamage1 \
              libnss3 \
              libxcomposite1 \
              libxcursor1 \
              libxi6 \
              libxext6 \
              libxfixes3 \
              zlib1g \
              libgl1 \
              pulseaudio \
              socat

            yes | sdkmanager --licenses || if [[ $? -eq 141 ]]; then true; else exit $?; fi
            sdkmanager --install "system-images;android-32;google_apis;x86_64"
            avdmanager --verbose create avd --force --name "pixel_4" --device "pixel_4" --package "system-images;android-32;google_apis;x86_64"            

            $ANDROID_SDK_ROOT/emulator/emulator @pixel_4 -no-audio -no-boot-anim -no-window -use-system-libs -wipe-data -accel on -gpu host 2>&1 >/dev/null &

            max_retry=10
            counter=0
            until adb shell getprop sys.boot_completed; do
              sleep 10
              [[ counter -eq $max_retry ]] && echo "Failed to start the emulator!" && exit 1
              counter=$((counter + 1))
            done

            npm install -g expo-cli eas-cli detox-cli
            npm ci
            detox build -c android
            mkdir -p android/app/build/outputs/apk/
            mv *.tar.gz android/app/build/outputs/apk/app-build.tar.gz
            cd android/app/build/outputs/apk/ && tar xvzf app-build.tar.gz && cd -
            adb shell input keyevent 82
            DETOX_ENV=standalone detox test -c android --headless --take-screenshots failing --record-videos failing
      - run:
          name: clean up
          when: always
          command: adb emu kill &
      - store_artifacts:
          path: artifacts/
      - save_cache:
          name: Saving npm cache
          paths:
            - ~/.npm
          key: npm-android-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}}-{{ checksum "package-lock.json" }}

Problems Occurred During Set Up

• Android build library conflicts

When building Android under the versions we use, we came across duplicates/conflicts for some of the libraries. In order to fix this, we used a custom expo plugin to automatically pick the first library during eas build process. So under plugins folder, create a file named withAndroidPickFirst.js with the following content:

const { withAppBuildGradle } = require("@expo/config-plugins");

function addPickFirst(buildGradle, paths) {
    const regexpPackagingOptions = /\bpackagingOptions\s*{/;
    const packagingOptionsMatch = buildGradle.match(regexpPackagingOptions);

    const bodyLines = [];
    paths.forEach((path) => {
        bodyLines.push(`        pickFirst '${path}'`);
    });
    const body = bodyLines.join("\n");

    if (packagingOptionsMatch) {
        console.warn(
            "WARN: withAndroidPickFirst: Replacing packagingOptions in app build.gradle"
        );
        return buildGradle.replace(
            regexpPackagingOptions,
            `packagingOptions {
                ${body}`
        );
    }

    const regexpAndroid = /\bandroid\s*{/;
    const androidMatch = buildGradle.match(regexpAndroid);

    if (androidMatch) {
        return buildGradle.replace(
            regexpAndroid,
            `android {
                packagingOptions {
                    ${body}
                }`
        );
    }

    throw new Error(
        "withAndroidPickFirst: Could not find where to add packagingOptions"
    );
}

module.exports = (config, props = {}) => {
    if (!props.paths) {
        throw new Error("withAndroidPickFirst: No paths specified!");
    }
    return withAppBuildGradle(config, (config) => {
        if (config.modResults.language === "groovy") {
            config.modResults.contents = addPickFirst(
                config.modResults.contents,
                props.paths
            );
        } else {
            throw new Error(
                "withAndroidPickFirst: Can't add pickFirst(s) because app build.gradle is not groovy"
            );
        }
        return config;
    });
};

Then inside app.json file, add the following to plugins section:

"plugins": [
   // ...
   [
     "./plugins/withAndroidPickFirst",
     {
       "paths": [
          "lib/**/libc++_shared.so",
          "lib/**/libreactnativejni.so",
          "lib/**/libreact_nativemodule_core.so",
          "lib/**/libglog.so",
          "lib/**/libjscexecutor.so",
          "lib/**/libfbjni.so",
          "lib/**/libfolly_json.so",
          "lib/**/libfolly_runtime.so",
          "lib/**/libhermes.so",
          "lib/**/libjsi.so"
       ]
      }
   ]
 ],

• Android emulator performance issues

Initially we aimed to use both iOS and Android for CircleCI pipeline but later on found out that Android emulator is quite unstable due to the error System UI isn't responding popup. This causes the tests to fail because we would need to click Dismiss from the popup. Even though we tried to add hardware acceleration to the emulator, it still doesn’t fix the issue completely.

• detox-expo-helpers local communication issues

The version we are using has an issue where it could not properly communicate with Expo app to execute the tests. After some investigation it seems that we have to manually patch the library to get around. Fortunately there is patch-package which allows us to apply a diff to the library. Install patch-package using npm:

npm install --save-dev patch-package

Then create the following diff detox-expo-helpers+0.6.0.patch under patches folder (you can also use patch-package to generate this for you):

diff --git a/node_modules/detox-expo-helpers/index.js b/node_modules/detox-expo-helpers/index.js
index 864493b..e9165dd 100644
--- a/node_modules/detox-expo-helpers/index.js
+++ b/node_modules/detox-expo-helpers/index.js
@@ -70,7 +70,13 @@ const reloadApp = async (params) => {
     newInstance: true,
     url,
     sourceApp: 'host.exp.exponent',
-    launchArgs: { EXKernelDisableNuxDefaultsKey: true, detoxURLBlacklistRegex: formattedBlacklistArg },
+    launchArgs: {
+      EXKernelDisableNuxDefaultsKey: true,
+      detoxURLBlacklistRegex: formattedBlacklistArg,
+      detoxEnableSynchronization: 0,
+      ...(params && params.launchArgs),
+    },
+    ...(params && params.args),
   });

Finally execute the patch after npm post install by specifying this in package.json file:

{
  // ...
  scripts: {
    // ...
    "postinstall": "patch-package",
  }
}

References

• Running E2E tests on EAS Build

Expo EAS

Expo provides a React Native build tool called Expo Application Services (EAS), it allows you to build and submit applications easily. By default when you run eas build, it will create a build on their cloud services and you will have to pay for priority pipeline execution. The Free plan at the time of writing only allows you to build one pipeline at a time. In our case we already have CircleCI subscription so it would be better if we can build app bundles there. Fortunately there is a --local option which allows building app bundles locally. You will still need to set up EAS on Expo, for detail guide, please refer to their setup guide. Here is an example command we use to build our app bundles using CircleCI MacOS and JVM executor.

eas build --profile preview --platform ios --clear-cache --local --non-interactive --output "my-app.ipa"

We use eas submit to automatically push master/production build to app store and play store after manual approval. The key here is that for BrowserStack we need to use internal distribution profile so that we can install on their devices. You don’t have to add their remote devices to Expo as BrowserStack re-signs the bundles upon upload so that they can be installed on their devices.

BrowserStack

To make current manual testing easier I decided to introduce BrowserStack App Live, it allows the team to test the app using remote real devices and it offers quite a lot of iOS and Android options. It also comes with local testing which allows us to connect to our backend services hosted behind our vpn. There are some options for getting access to the app. You can upload app .ipa and .apk bundles, or install via Apple TestFlight and Google Play. To test push notification, please refer to guide here. If you are uploading the .ipa Apple bundle, unfortunately it is not supported at the time of writing, the only way to test push notification on iOS is installation via TestFlight.

Overall Pipeline View

Here is what our overall pipeline looks like:

For branch build we have a manual trigger to push bundles to BrowserStack as we don’t want every commit to go through manual QA process. We only trigger this once the PR has been reviewed and ready for QA. The bundles are also uploaded to S3 so that we can retrieve them and install locally easily. You will need to register your test physical devices with Expo using eas device:create so that the bundles can be installed. See internal distribution setup here.

For master/main branch build we went through the similar process with the only difference that builds are automatically pushed to BrowserStack with a manual trigger to push to production following production profile build. We use standard-version to automatically increase our app version so that every build is able to be released. The BrowserStack preview here is quite useful during our review session and following that we will decide whether the candidate version is good enough for release.

CircleCI config

Here are some useful CircleCI steps for the pipeline described above:

Build Android Using OpenJDK executor

eas-build-android:
    executor: android
    parameters:
      profile: 
        description: distribution profile
        type: string
        default: preview
    environment:
      PROFILE: << parameters.profile >>
    steps:
      - attach_workspace:
          at: ~/
      - node/install:
          install-yarn: true
          node-version: '16.9.1'
      - run: echo -n $PLAY_STORE_SERVICE_ACCOUNT_KEY | base64 -d  > serviceAccountKey.json        
      - run:
          name: android build
          command: |
            mkdir android_sdk
            curl -O https://dl.google.com/android/repository/commandlinetools-linux-8512546_latest.zip
            unzip commandlinetools-linux-8512546_latest.zip
            mv cmdline-tools android_sdk
            mkdir -p android_sdk/cmdline-tools/tools
            mv -t android_sdk/cmdline-tools/tools android_sdk/cmdline-tools/lib android_sdk/cmdline-tools/bin android_sdk/cmdline-tools/source.properties android_sdk/cmdline-tools/NOTICE.txt
            export ANDROID_SDK_ROOT=$HOME/project/android_sdk
            yes | $ANDROID_SDK_ROOT/cmdline-tools/tools/bin/sdkmanager --licenses || if [[ $? -eq 141 ]]; then true; else exit $?; fi
            npm install -g expo-cli eas-cli
            npm ci
            eas build --profile << parameters.profile >> --platform android --clear-cache --local --non-interactive --output "${CIRCLE_BRANCH//\//-}-${PROFILE}-${CIRCLE_SHA1:0:7}.apk"

Build iOS using MacOS executor

  eas-build-ios:
    executor: ios
    parameters:
      profile: 
        description: eas profile
        type: string
        default: preview
    environment:
      PROFILE: << parameters.profile >>
    steps:
      - attach_workspace:
          at: ~/
      - node/install:
          install-yarn: true
          node-version: '16.9.1'
      - run:
          name: ios build
          command: |
            npm install -g expo-cli eas-cli
            npm ci
            eas build --profile << parameters.profile >> --platform ios --clear-cache --local --non-interactive --output "${CIRCLE_BRANCH//\//-}-${PROFILE}-${CIRCLE_SHA1:0:7}.ipa"

We are using CircleCI NodeJS Orbs to install NodeJS on the executors. For Android we pass in service account key from pipeline secret and need to download Command Line Tools too.

Overall the pipeline works quite well for us and we have significant improvements on development process. Now product manager/designer and manual QA could easily download the bundle and play with the app on either BrowserStack or our test physical devices.