mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
05e619e55d
## What `wayland-e2e.yml` boots an Android AVD on Linux under headless Weston and asserts the full pipeline works without a display. It guards exactly one cell of the support matrix. This adds the same boot→screenshot→gesture-tap smoke test for every other (target × host) cell that can run on GitHub-hosted runners. | target | host | job | status | |---|---|---|---| | Android emulator | Linux (KVM) | `wayland-e2e.yml` (existing) | ✅ | | iOS simulator | macOS | `ios-sim-macos` | ✅ | | Chromium / Electron | Linux (Xvfb) | `chromium-linux` | ✅ | | Chromium / Electron | macOS | `chromium-macos` | ✅ | ## How - **`scripts/e2e/drive-device.sh`** — shared driver. Boots one device through the tool-server, then asserts `booted:true` → screenshot has real (non-blank) pixels → `gesture-tap` round-trips. Every tool takes the device id as `udid` and screenshots come back as the same `data.image.hostPath` envelope on all platforms, so the body is uniform; each job just supplies the cell-specific boot JSON + device id. - **`packages/tool-server/test/fixtures/electron-smoke-app/`** — minimal self-contained Electron app the Chromium jobs point `electronAppPath` at. It vendors its own electron via its own lockfile (kept out of the repo root lockfile); CI `npm ci`s it so `boot-electron` resolves `./node_modules/.bin/electron`. - **`.github/workflows/e2e-device-smoke.yml`** — three jobs, `workflow_dispatch` + path-filtered `pull_request`. iOS downloads the darwin native binaries (injection must succeed for `bootIos` to report `booted:true`); Chromium-Linux runs under Xvfb with `--no-sandbox`. The tool-server is started with `TS_NODE_TRANSPILE_ONLY=1` so cold ts-node startup doesn't flakily exceed the readiness poll on loaded macOS runners (types are still enforced by the typecheck job). ## Known gaps (supported on real machines, not runnable on hosted runners) - **Android emulator on macOS** — the arm64 emulator needs HVF (`-enable-hvf`), but hosted macOS runners are themselves VMs with no nested virtualization. Confirmed by running it: qemu dies with `HVF error: HV_UNSUPPORTED` ~18s into boot, regardless of GPU mode or RAM. The Android boot/screenshot/tap path is already regression-guarded on Linux via KVM (`wayland-e2e.yml`), so macOS adds no runnable coverage. - **Physical devices** — real iOS (CoreDevice HID) and real Android (adb-USB) have no attached hardware on hosted runners. Both would need self-hosted runners (a Mac exposing HVF / wired-up devices). Documented in the workflow header. ## Verification - All checks green: the 3 device jobs + ESLint + Prettier + Unit tests. - Locally (before CI): ran `drive-device.sh` end-to-end against a real tool-server → boot `booted:true`, screenshot **142 KB** (floor 20 KB), tap `tapped:true`, exit 0; and confirmed via the Argent stack that a tap at (0.5, 0.5) lands on the fixture's centered button and increments its counter.
186 lines
8.6 KiB
YAML
186 lines
8.6 KiB
YAML
name: Device E2E smoke
|
||
|
||
# Boot-and-drive smoke tests for every (target × host) cell of the support
|
||
# matrix that runs on GitHub-hosted runners. Each job boots ONE device through
|
||
# the tool-server and asserts the headless interaction pipeline end-to-end via
|
||
# scripts/e2e/drive-device.sh: booted:true → screenshot has real pixels →
|
||
# gesture-tap round-trips.
|
||
#
|
||
# ┌────────────────────┬───────────────┬──────────────────────────────────┐
|
||
# │ target │ host │ where │
|
||
# ├────────────────────┼───────────────┼──────────────────────────────────┤
|
||
# │ Android emulator │ Linux (KVM) │ wayland-e2e.yml (separate file: │
|
||
# │ │ │ also asserts gpu.mode=swiftshader)│
|
||
# │ iOS simulator │ macOS │ ios-sim-macos (this file) │
|
||
# │ Chromium / Electron│ Linux (Xvfb) │ chromium-linux (this file) │
|
||
# │ Chromium / Electron│ macOS │ chromium-macos (this file) │
|
||
# └────────────────────┴───────────────┴──────────────────────────────────┘
|
||
#
|
||
# KNOWN GAPS — cells Argent supports on real machines but that GitHub-hosted
|
||
# runners can't run, so they are intentionally not jobs here:
|
||
# - Android emulator on macOS: the arm64 emulator needs HVF (`-enable-hvf`),
|
||
# but hosted macOS runners are themselves VMs with no nested virtualization
|
||
# — qemu dies with "HVF error: HV_UNSUPPORTED" seconds into boot. The
|
||
# Android boot/screenshot/tap path is already regression-guarded on Linux
|
||
# via KVM (wayland-e2e.yml), so macOS adds no runnable coverage here.
|
||
# - Physical devices: real iOS (CoreDevice HID) and real Android (adb-USB)
|
||
# have no attached hardware on hosted runners.
|
||
# Both would need self-hosted runners (a Mac exposing HVF / wired-up devices).
|
||
#
|
||
# Manual + on-change trigger only — the macOS jobs are slow and billed.
|
||
|
||
on:
|
||
workflow_dispatch:
|
||
pull_request:
|
||
branches:
|
||
- main
|
||
paths:
|
||
- "packages/tool-server/src/tools/devices/**"
|
||
- "packages/tool-server/src/tools/gesture-tap/**"
|
||
- "packages/tool-server/src/tools/screenshot/**"
|
||
# Transport layer that drives screenshot/gesture-tap for each platform —
|
||
# where the #391 screenshot regression lived, so changes here must re-run
|
||
# these smoke jobs too.
|
||
- "packages/tool-server/src/blueprints/simulator-server.ts"
|
||
- "packages/tool-server/src/blueprints/chromium-cdp.ts"
|
||
- "packages/tool-server/src/utils/simulator-client.ts"
|
||
- "packages/tool-server/test/fixtures/electron-smoke-app/**"
|
||
- "scripts/e2e/drive-device.sh"
|
||
- ".github/workflows/e2e-device-smoke.yml"
|
||
# The shared composite action is part of the test harness too.
|
||
- ".github/actions/start-tool-server/action.yml"
|
||
|
||
# Cancel superseded runs on the same ref: rapid pushes to a PR would otherwise
|
||
# stack up the slow, billed macOS jobs (see header) with no cancellation.
|
||
concurrency:
|
||
group: ${{ github.workflow }}-${{ github.ref }}
|
||
cancel-in-progress: true
|
||
|
||
jobs:
|
||
# ── iOS simulator on macOS ────────────────────────────────────────────────
|
||
ios-sim-macos:
|
||
name: macOS + iOS E2E
|
||
runs-on: macos-latest
|
||
timeout-minutes: 30
|
||
steps:
|
||
- uses: actions/checkout@v6
|
||
|
||
- uses: ./.github/actions/start-tool-server
|
||
|
||
- name: Download signed native binaries (darwin)
|
||
# iOS needs the simulator-server (screenshot/gesture-tap backend) AND the
|
||
# dylibs + ax-service: bootIos() only returns booted:true when native-
|
||
# devtools injection does NOT give up (boot-device.ts), so missing dylibs
|
||
# would surface here as a hard failure rather than a silent skip. Only
|
||
# consumed at boot time, so it can run after the server starts — and after
|
||
# start-tool-server has set up Node (download-native-binaries.sh reads the
|
||
# Android manifest version with `node -p`).
|
||
run: |
|
||
bash scripts/download-simulator-server.sh
|
||
bash scripts/download-native-binaries.sh
|
||
env:
|
||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||
|
||
- name: Pick an available simulator UDID
|
||
run: |
|
||
UDID=$(xcrun simctl list devices available --json | python3 -c "import json,sys; d=json.load(sys.stdin)['devices']; print(next(dev['udid'] for rt in d for dev in d[rt] if dev.get('isAvailable') and 'iPhone' in dev['name']))")
|
||
echo "Chosen simulator: $UDID"
|
||
echo "UDID=$UDID" >> "$GITHUB_ENV"
|
||
|
||
- name: Boot + drive
|
||
run: bash scripts/e2e/drive-device.sh
|
||
env:
|
||
BOOT_JSON: '{"udid":"${{ env.UDID }}"}'
|
||
DEVICE_ID: ${{ env.UDID }}
|
||
|
||
- if: always()
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: e2e-ios-sim-macos
|
||
path: |
|
||
${{ runner.temp }}/smoke-shot.png
|
||
${{ runner.temp }}/smoke-shot.json
|
||
~/.argent/tool-server.log
|
||
if-no-files-found: warn
|
||
retention-days: 7
|
||
|
||
# ── Chromium / Electron on Linux (Xvfb) ───────────────────────────────────
|
||
chromium-linux:
|
||
name: Linux + Chromium E2E
|
||
runs-on: ubuntu-latest
|
||
timeout-minutes: 20
|
||
steps:
|
||
- uses: actions/checkout@v6
|
||
|
||
- name: Start Xvfb
|
||
# boot-electron spawns the Electron child inheriting the tool-server's
|
||
# env, so DISPLAY must be set (in GITHUB_ENV) BEFORE the tool-server
|
||
# starts in the start-tool-server action below.
|
||
run: |
|
||
sudo apt-get update -y
|
||
sudo apt-get install -y --no-install-recommends xvfb
|
||
Xvfb :99 -screen 0 1280x1024x24 >/tmp/xvfb.log 2>&1 &
|
||
echo "DISPLAY=:99" >> "$GITHUB_ENV"
|
||
|
||
- uses: ./.github/actions/start-tool-server
|
||
|
||
- name: Vendor the fixture's electron
|
||
# The fixture brings its own electron (kept out of the root lockfile);
|
||
# boot-electron resolves ./node_modules/.bin/electron from the app dir.
|
||
# Standalone project (not a packages/* workspace member), so this ci is
|
||
# independent of the root install. Electron is only needed at boot time,
|
||
# so vendoring it after the server starts is fine.
|
||
run: npm ci
|
||
working-directory: packages/tool-server/test/fixtures/electron-smoke-app
|
||
|
||
- name: Boot + drive
|
||
run: bash scripts/e2e/drive-device.sh
|
||
env:
|
||
BOOT_JSON: '{"electronAppPath":"${{ github.workspace }}/packages/tool-server/test/fixtures/electron-smoke-app","electronPort":19222,"electronArgs":["--no-sandbox"]}'
|
||
DEVICE_ID: chromium-cdp-19222
|
||
|
||
- if: always()
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: e2e-chromium-linux
|
||
path: |
|
||
${{ runner.temp }}/smoke-shot.png
|
||
${{ runner.temp }}/smoke-shot.json
|
||
/tmp/xvfb.log
|
||
~/.argent/tool-server.log
|
||
if-no-files-found: warn
|
||
retention-days: 7
|
||
|
||
# ── Chromium / Electron on macOS ──────────────────────────────────────────
|
||
chromium-macos:
|
||
name: macOS + Chromium E2E
|
||
runs-on: macos-latest
|
||
timeout-minutes: 20
|
||
steps:
|
||
- uses: actions/checkout@v6
|
||
|
||
- uses: ./.github/actions/start-tool-server
|
||
|
||
- name: Vendor the fixture's electron
|
||
run: npm ci
|
||
working-directory: packages/tool-server/test/fixtures/electron-smoke-app
|
||
|
||
- name: Boot + drive
|
||
# macOS runner has a window server and Electron's sandbox is fine there,
|
||
# so no Xvfb and no --no-sandbox.
|
||
run: bash scripts/e2e/drive-device.sh
|
||
env:
|
||
BOOT_JSON: '{"electronAppPath":"${{ github.workspace }}/packages/tool-server/test/fixtures/electron-smoke-app","electronPort":19222}'
|
||
DEVICE_ID: chromium-cdp-19222
|
||
|
||
- if: always()
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: e2e-chromium-macos
|
||
path: |
|
||
${{ runner.temp }}/smoke-shot.png
|
||
${{ runner.temp }}/smoke-shot.json
|
||
~/.argent/tool-server.log
|
||
if-no-files-found: warn
|
||
retention-days: 7
|