Files
software-mansion__argent/scripts/e2e/drive-device.sh
Ignacy Łątka 05e619e55d ci: device E2E smoke tests for every hosted-CI platform (#371)
## What

`wayland-e2e.yml` boots an Android AVD on Linux under headless Weston
and asserts the full pipeline works without a display. It guards exactly
one cell of the support matrix. This adds the same
boot→screenshot→gesture-tap smoke test for every other (target × host)
cell that can run on GitHub-hosted runners.

| target | host | job | status |
|---|---|---|---|
| Android emulator | Linux (KVM) | `wayland-e2e.yml` (existing) | ✅ |
| iOS simulator | macOS | `ios-sim-macos` | ✅ |
| Chromium / Electron | Linux (Xvfb) | `chromium-linux` | ✅ |
| Chromium / Electron | macOS | `chromium-macos` | ✅ |

## How

- **`scripts/e2e/drive-device.sh`** — shared driver. Boots one device
through the tool-server, then asserts `booted:true` → screenshot has
real (non-blank) pixels → `gesture-tap` round-trips. Every tool takes
the device id as `udid` and screenshots come back as the same
`data.image.hostPath` envelope on all platforms, so the body is uniform;
each job just supplies the cell-specific boot JSON + device id.
- **`packages/tool-server/test/fixtures/electron-smoke-app/`** — minimal
self-contained Electron app the Chromium jobs point `electronAppPath`
at. It vendors its own electron via its own lockfile (kept out of the
repo root lockfile); CI `npm ci`s it so `boot-electron` resolves
`./node_modules/.bin/electron`.
- **`.github/workflows/e2e-device-smoke.yml`** — three jobs,
`workflow_dispatch` + path-filtered `pull_request`. iOS downloads the
darwin native binaries (injection must succeed for `bootIos` to report
`booted:true`); Chromium-Linux runs under Xvfb with `--no-sandbox`. The
tool-server is started with `TS_NODE_TRANSPILE_ONLY=1` so cold ts-node
startup doesn't flakily exceed the readiness poll on loaded macOS
runners (types are still enforced by the typecheck job).

## Known gaps (supported on real machines, not runnable on hosted
runners)

- **Android emulator on macOS** — the arm64 emulator needs HVF
(`-enable-hvf`), but hosted macOS runners are themselves VMs with no
nested virtualization. Confirmed by running it: qemu dies with `HVF
error: HV_UNSUPPORTED` ~18s into boot, regardless of GPU mode or RAM.
The Android boot/screenshot/tap path is already regression-guarded on
Linux via KVM (`wayland-e2e.yml`), so macOS adds no runnable coverage.
- **Physical devices** — real iOS (CoreDevice HID) and real Android
(adb-USB) have no attached hardware on hosted runners.

Both would need self-hosted runners (a Mac exposing HVF / wired-up
devices). Documented in the workflow header.

## Verification

- All checks green: the 3 device jobs + ESLint + Prettier + Unit tests.
- Locally (before CI): ran `drive-device.sh` end-to-end against a real
tool-server → boot `booted:true`, screenshot **142 KB** (floor 20 KB),
tap `tapped:true`, exit 0; and confirmed via the Argent stack that a tap
at (0.5, 0.5) lands on the fixture's centered button and increments its
counter.
2026-06-26 17:44:17 +02:00

132 lines
5.6 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
# Generic tool-server E2E driver shared by every cell of e2e-device-smoke.yml
# (iOS sim / Android emulator / Chromium-Electron, on macOS or Linux). It boots
# ONE device through an already-running tool-server, then asserts the core
# interaction pipeline works headlessly:
# 1. boot-device reports booted:true
# 2. screenshot returns real (non-blank) pixels
# 3. gesture-tap round-trips
# This mirrors the three assertions inlined in wayland-e2e.yml, but parameterised
# so each platform job stays a few lines. Every tool takes the device id as
# `udid` and screenshots come back as the same ArtifactHandle envelope
# (data.image.hostPath) regardless of platform, so the body is uniform.
#
# Inputs (env):
# BOOT_JSON required JSON body for POST /boot-device
# DEVICE_ID required id/udid/serial for screenshot + gesture-tap
# (e.g. emulator-5554, an iOS UDID, chromium-cdp-<port>)
# BASE_URL optional default http://127.0.0.1:3033/tools
# MIN_SHOT_BYTES optional default 20000 (all-zero framebuffer PNGs are ~3-7 KB)
# SLEEP_BEFORE_SHOT optional default 0 (Android needs ~15s for SurfaceFlinger)
# BOOT_CURL_TIMEOUT optional default 900 (curl -m seconds; > the boot budget)
# ARTIFACT_DIR optional default ${RUNNER_TEMP:-/tmp}; screenshot copied here
: "${BOOT_JSON:?BOOT_JSON is required}"
: "${DEVICE_ID:?DEVICE_ID is required}"
BASE_URL="${BASE_URL:-http://127.0.0.1:3033/tools}"
MIN_SHOT_BYTES="${MIN_SHOT_BYTES:-20000}"
SLEEP_BEFORE_SHOT="${SLEEP_BEFORE_SHOT:-0}"
BOOT_CURL_TIMEOUT="${BOOT_CURL_TIMEOUT:-900}"
ARTIFACT_DIR="${ARTIFACT_DIR:-${RUNNER_TEMP:-/tmp}}"
mkdir -p "$ARTIFACT_DIR"
echo "::group::boot-device ($DEVICE_ID)"
echo "POST ${BASE_URL}/boot-device body=${BOOT_JSON}"
T0=$(date +%s)
BOOT_RESP=$(curl -sS -m "$BOOT_CURL_TIMEOUT" -X POST "${BASE_URL}/boot-device" \
-H 'Content-Type: application/json' -d "$BOOT_JSON")
T1=$(date +%s)
echo "boot-device ($((T1 - T0))s): $BOOT_RESP"
echo "::endgroup::"
echo "$BOOT_RESP" | python3 -c "import json,sys; sys.exit(0 if json.load(sys.stdin).get('data',{}).get('booted') is True else 1)" || {
echo "::error::boot-device did not report booted:true"
exit 1
}
if [ "$SLEEP_BEFORE_SHOT" -gt 0 ]; then
echo "Waiting ${SLEEP_BEFORE_SHOT}s for the compositor to paint the first frame..."
sleep "$SLEEP_BEFORE_SHOT"
fi
echo "::group::screenshot"
SHOT_JSON="$ARTIFACT_DIR/smoke-shot.json"
# Retry a few times: a fresh-spawn streaming-screenshot race (issue #391) can
# briefly return an error envelope before the first frame is available. A tiny
# loop is cheap belt-and-suspenders for this one-shot smoke.
HOST_PATH=""
for attempt in 1 2 3; do
curl -sS -m 60 -X POST "${BASE_URL}/screenshot" \
-H 'Content-Type: application/json' \
-d "{\"udid\":\"${DEVICE_ID}\"}" >"$SHOT_JSON" || true
# Surface a tool-side error as a clean annotation instead of letting the
# hostPath extraction blow up with a raw python KeyError traceback.
ERR=$(python3 -c "import json,sys;
try:
d=json.load(open('$SHOT_JSON'))
except Exception as e:
print('unparseable screenshot response: %s' % e); sys.exit(0)
print(d.get('error','') if isinstance(d,dict) else '')")
if [ -n "$ERR" ]; then
echo "screenshot attempt ${attempt}/3 failed: ${ERR}"
sleep 2
continue
fi
# The screenshot tool returns an ArtifactHandle ({ image: { hostPath, ... } });
# the job runs co-located with the tool-server so hostPath is readable here.
HOST_PATH=$(python3 -c "import json;print(json.load(open('$SHOT_JSON')).get('data',{}).get('image',{}).get('hostPath',''))")
[ -n "$HOST_PATH" ] && break
echo "screenshot attempt ${attempt}/3 returned no hostPath"
sleep 2
done
if [ -z "$HOST_PATH" ]; then
echo "::error::screenshot did not return a readable hostPath after 3 attempts (last response: $(cat "$SHOT_JSON"))"
exit 1
fi
SHOT_PNG="$ARTIFACT_DIR/smoke-shot.png"
cp "$HOST_PATH" "$SHOT_PNG"
# `wc -c` instead of `stat` so the script is identical on macOS and Linux
# (BSD stat uses -f%z, GNU stat uses -c%s).
SZ=$(wc -c <"$SHOT_PNG" | tr -d '[:space:]')
echo "screenshot=${SHOT_PNG} size=${SZ}B (floor ${MIN_SHOT_BYTES}B)"
echo "::endgroup::"
# An all-zero / uniform framebuffer PNG-compresses to a few KB; a real painted
# screen is reliably larger. Guards against the boot "succeeding" but the
# device never actually rendering.
test "$SZ" -gt "$MIN_SHOT_BYTES" || {
echo "::error::screenshot too small (${SZ}B) — likely a blank/all-zero framebuffer"
exit 1
}
echo "::group::gesture-tap"
# Mirror the screenshot retry above: the successful screenshot already proves the
# device is painted, so a tap race (#391) is unlikely here — but a tiny loop keeps
# the tap symmetric with the screenshot and avoids an unguarded hard-fail if the
# same transport race ever surfaces. A non-JSON / error response just retries
# instead of spewing a raw python traceback.
TAPPED=""
for attempt in 1 2 3; do
TAP_RESP=$(curl -sS -m 30 -X POST "${BASE_URL}/gesture-tap" \
-H 'Content-Type: application/json' \
-d "{\"udid\":\"${DEVICE_ID}\",\"x\":0.5,\"y\":0.5}" || true)
echo "gesture-tap attempt ${attempt}/3: $TAP_RESP"
if printf '%s' "$TAP_RESP" | python3 -c "import json,sys
try:
d = json.load(sys.stdin)
except Exception:
sys.exit(1)
sys.exit(0 if isinstance(d, dict) and d.get('data', {}).get('tapped') is True else 1)"; then
TAPPED=1
break
fi
sleep 2
done
echo "::endgroup::"
[ -n "$TAPPED" ] || {
echo "::error::gesture-tap did not report tapped:true after 3 attempts"
exit 1
}
echo "✅ smoke OK: booted + screenshot ${SZ}B + tap on ${DEVICE_ID}"