Recently our Frontend team started to think about automated testing for our Expo managed project. Following discussion we decided to use Detox as our E2E testing tool (We evaluated Appium and Detox). Since we already have a CircleCI pipeline running based on Expo EAS local build and automatic submission, it makes sense to integrate Detox into our pipeline. In additional we also want to enable easy local development when writing E2E tests. After research we have reached the following conclusions/setup:
- Local development: We will use Expo Go app together with Detox so that we can refresh the app and update our tests easily.
- CI pipeline: We will build standalone artefacts and use CircleCI MacOS and Android executors to run the same test suites
At the time of writing the team is using Expo 46 and Detox 19.
Set Up Detox
On your project, run the following command to install required dependencies
npm install --save-dev @config-plugins/detox detox@19.9.0 detox-expo-helpers jest@config-plugins/detox is used to help generate detox compatible artifacts when expo runs the build.
detox-expo-helpers is needed when we use Expo Go locally to communicate with detox, more on this later.
jest is required because detox does not have its own test-runner.
Now, open app.json and add the @config-plugins/detox plugin to your plugins list (this must be done before prebuilding). This will automatically configure the Android native code to support Detox.
{
"expo": {
// ...
"plugins": ["@config-plugins/detox"]
}
}Create Detox configuration files using the following command:
npx detox init -r jestAdd .detoxrc.json to project root dir. Note that ios-expo is to support local development
{
"testRunner": "jest",
"runnerConfig": "e2e/config.json",
"skipLegacyWorkersInjection": true,
"apps": {
"ios": {
"type": "ios.app",
"binaryPath": "ios/build/Build/Products/Release-iphonesimulator/MyApp.app",
"build": "eas build --platform ios --profile detox-test --local --clear-cache"
},
"android": {
"type": "android.apk",
"binaryPath": "android/app/build/outputs/apk/release/app-release.apk",
"testBinaryPath": "android/app/build/outputs/apk/androidTest/release/app-release-androidTest.apk",
"build": "eas build --platform android --profile detox-test --local --clear-cache"
},
// Local configuration
"ios-expo": {
"type": "ios.app",
"binaryPath": "bin/Exponent.app"
}
},
"devices": {
"simulator": {
"type": "ios.simulator",
"device": {
"type": "iPhone 14"
}
},
"emulator": {
"type": "android.emulator",
"device": {
"avdName": "pixel_4"
}
}
},
"configurations": {
"ios": {
"device": "simulator",
"app": "ios"
},
"android": {
"device": "emulator",
"app": "android"
},
// Local configuration
"ios-expo": {
"device": "simulator",
"app": "ios-expo"
}
}
}Update eas.json to include detox test details
{
// ...
"build": {
"detox-test": {
"distribution": "internal",
"ios": {
"simulator": true
},
"android": {
"gradleCommand": ":app:assembleRelease :app:assembleAndroidTest -DtestBuildType=release",
"withoutCredentials": true
}
},
// ...
},
// ...
}
Local Development
For local development we will use Expo app instead of building standalone apps because it will be faster to reflect changes and run tests by following TDD practice.
So in package.json we could add a helper command like following inside scripts section:
{
// ...
scripts: {
// ...
"test:detox": "DETOX_ENV=expo detox test -c ios-expo",
}
}And we also added a scripting file to help running the test easier
#!/bin/sh
EXPO_APP_DIR=bin/Exponent.app
if [ ! -d $EXPO_APP_DIR ]; then
mkdir -p $EXPO_APP_DIR
curl https://dpq5q02fu5f55.cloudfront.net/Exponent-2.28.7.tar.gz -o bin/expo-app.tar.gz
tar xvzf bin/expo-app.tar.gz -C $EXPO_APP_DIR
fi
npm run test:detoxBecause Expo app and standalone apps open and execute the tests in a different way, at the moment we choose to accommodate our tests by using an environment variable (We use expo for local development and standalone for running in CircleCI). Hopefully this will be improved in future. An example test would look like the following:
const { reloadApp } = require("detox-expo-helpers");
const detoxEnv = process.env.DETOX_ENV;
describe("Example", () => {
beforeAll(async () => {
if (detoxEnv === "standalone") {
await device.launchApp();
}
});
beforeEach(async () => {
if (detoxEnv === "standalone") {
await device.reloadReactNative();
} else if (detoxEnv === "expo") {
await reloadApp();
await waitFor(element(by.id("loginMessage")))
.toBeVisible()
.withTimeout(20000);
}
});
// test cases
// ...
});CircleCI pipeline
In CircleCI we will build the standalone apps first, start up the simulators then execute detox tests. By the time of writing, Android emulator is very unstable even with hardware acceleration on (see problems below), so we opt to run detox tests only on iOS simulator which is quite stable.
iOS step:
detox-test-ios:
executor: ios
steps:
- attach_workspace:
at: ~/
- node/install:
install-yarn: true
node-version: '16.9.1'
- run: echo -n $ARTIFACTORY_NPMRC | base64 -d > ~/.npmrc
- restore_cache:
name: Restoring npm cache
keys:
- npm-ios-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}}-{{ checksum "package-lock.json" }} # Primary cache
- npm-ios-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}} # Fallback cache
- run:
name: Install Apple sim utils
command: |
brew tap wix/brew
brew install applesimutils
- run:
name: ios build
command: |
npm install -g expo-cli eas-cli detox-cli
npm ci
detox build -c ios
mkdir -p ios/build/Build/Products/Release-iphonesimulator/
mv *.tar.gz ios/build/Build/Products/Release-iphonesimulator/app-build.tar.gz
cd ios/build/Build/Products/Release-iphonesimulator/ && tar xvzf app-build.tar.gz && cd -
- run:
name: ios test
command: DETOX_ENV=standalone detox test -c ios --headless --take-screenshots failing --record-videos failing
- store_artifacts:
path: artifacts/
- save_cache:
name: Saving npm cache
paths:
- ~/.npm
key: npm-ios-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}}-{{ checksum "package-lock.json" }}Android step:
detox-test-android:
machine:
image: android:2023.02.1
resource_class: xlarge
steps:
- attach_workspace:
at: ~/
- node/install:
install-yarn: true
node-version: '16.9.1'
- run: echo -n $ARTIFACTORY_NPMRC | base64 -d > ~/.npmrc
- restore_cache:
name: Restoring npm cache
keys:
- npm-android-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}}-{{ checksum "package-lock.json" }} # Primary cache
- npm-android-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}} # Fallback cache
- run: echo -n $PLAY_STORE_SERVICE_ACCOUNT_KEY | base64 -d > serviceAccountKey.json
- run:
name: Android build and test
command: |
sudo apt-get --quiet update --yes
sudo apt-get --quiet install --yes \
libc6 \
libdbus-1-3 \
libfontconfig1 \
libgcc1 \
libpulse0 \
libtinfo5 \
libx11-6 \
libxcb1 \
libxdamage1 \
libnss3 \
libxcomposite1 \
libxcursor1 \
libxi6 \
libxext6 \
libxfixes3 \
zlib1g \
libgl1 \
pulseaudio \
socat
yes | sdkmanager --licenses || if [[ $? -eq 141 ]]; then true; else exit $?; fi
sdkmanager --install "system-images;android-32;google_apis;x86_64"
avdmanager --verbose create avd --force --name "pixel_4" --device "pixel_4" --package "system-images;android-32;google_apis;x86_64"
$ANDROID_SDK_ROOT/emulator/emulator @pixel_4 -no-audio -no-boot-anim -no-window -use-system-libs -wipe-data -accel on -gpu host 2>&1 >/dev/null &
max_retry=10
counter=0
until adb shell getprop sys.boot_completed; do
sleep 10
[[ counter -eq $max_retry ]] && echo "Failed to start the emulator!" && exit 1
counter=$((counter + 1))
done
npm install -g expo-cli eas-cli detox-cli
npm ci
detox build -c android
mkdir -p android/app/build/outputs/apk/
mv *.tar.gz android/app/build/outputs/apk/app-build.tar.gz
cd android/app/build/outputs/apk/ && tar xvzf app-build.tar.gz && cd -
adb shell input keyevent 82
DETOX_ENV=standalone detox test -c android --headless --take-screenshots failing --record-videos failing
- run:
name: clean up
when: always
command: adb emu kill &
- store_artifacts:
path: artifacts/
- save_cache:
name: Saving npm cache
paths:
- ~/.npm
key: npm-android-detox-{{.Environment.CIRCLE_PROJECT_REPONAME}}-{{ checksum "package-lock.json" }}Problems Occurred During Set Up
- Android build library conflicts
When building Android under the versions we use, we came across duplicates/conflicts for some of the libraries. In order to fix this, we used a custom expo plugin to automatically pick the first library during eas build process. So under plugins folder, create a file named withAndroidPickFirst.js with the following content:
const { withAppBuildGradle } = require("@expo/config-plugins");
function addPickFirst(buildGradle, paths) {
const regexpPackagingOptions = /\bpackagingOptions\s*{/;
const packagingOptionsMatch = buildGradle.match(regexpPackagingOptions);
const bodyLines = [];
paths.forEach((path) => {
bodyLines.push(` pickFirst '${path}'`);
});
const body = bodyLines.join("\n");
if (packagingOptionsMatch) {
console.warn(
"WARN: withAndroidPickFirst: Replacing packagingOptions in app build.gradle"
);
return buildGradle.replace(
regexpPackagingOptions,
`packagingOptions {
${body}`
);
}
const regexpAndroid = /\bandroid\s*{/;
const androidMatch = buildGradle.match(regexpAndroid);
if (androidMatch) {
return buildGradle.replace(
regexpAndroid,
`android {
packagingOptions {
${body}
}`
);
}
throw new Error(
"withAndroidPickFirst: Could not find where to add packagingOptions"
);
}
module.exports = (config, props = {}) => {
if (!props.paths) {
throw new Error("withAndroidPickFirst: No paths specified!");
}
return withAppBuildGradle(config, (config) => {
if (config.modResults.language === "groovy") {
config.modResults.contents = addPickFirst(
config.modResults.contents,
props.paths
);
} else {
throw new Error(
"withAndroidPickFirst: Can't add pickFirst(s) because app build.gradle is not groovy"
);
}
return config;
});
};Then inside app.json file, add the following to plugins section:
"plugins": [
// ...
[
"./plugins/withAndroidPickFirst",
{
"paths": [
"lib/**/libc++_shared.so",
"lib/**/libreactnativejni.so",
"lib/**/libreact_nativemodule_core.so",
"lib/**/libglog.so",
"lib/**/libjscexecutor.so",
"lib/**/libfbjni.so",
"lib/**/libfolly_json.so",
"lib/**/libfolly_runtime.so",
"lib/**/libhermes.so",
"lib/**/libjsi.so"
]
}
]
],- Android emulator performance issues
Initially we aimed to use both iOS and Android for CircleCI pipeline but later on found out that Android emulator is quite unstable due to the error System UI isn't responding popup. This causes the tests to fail because we would need to click Dismiss from the popup. Even though we tried to add hardware acceleration to the emulator, it still doesn’t fix the issue completely.
- detox-expo-helpers local communication issues
The version we are using has an issue where it could not properly communicate with Expo app to execute the tests. After some investigation it seems that we have to manually patch the library to get around. Fortunately there is patch-package which allows us to apply a diff to the library. Install patch-package using npm:
npm install --save-dev patch-packageThen create the following diff detox-expo-helpers+0.6.0.patch under patches folder (you can also use patch-package to generate this for you):
diff --git a/node_modules/detox-expo-helpers/index.js b/node_modules/detox-expo-helpers/index.js
index 864493b..e9165dd 100644
--- a/node_modules/detox-expo-helpers/index.js
+++ b/node_modules/detox-expo-helpers/index.js
@@ -70,7 +70,13 @@ const reloadApp = async (params) => {
newInstance: true,
url,
sourceApp: 'host.exp.exponent',
- launchArgs: { EXKernelDisableNuxDefaultsKey: true, detoxURLBlacklistRegex: formattedBlacklistArg },
+ launchArgs: {
+ EXKernelDisableNuxDefaultsKey: true,
+ detoxURLBlacklistRegex: formattedBlacklistArg,
+ detoxEnableSynchronization: 0,
+ ...(params && params.launchArgs),
+ },
+ ...(params && params.args),
});Finally execute the patch after npm post install by specifying this in package.json file:
{
// ...
scripts: {
// ...
"postinstall": "patch-package",
}
}