callstack__agent-device/examples/test-app

Agent Device Tester

Agent Device Tester is a minimal Expo Router fixture app for agent-device and skillgym experiments.

It is intentionally small, but each surface is dense with durable accessibility targets so a few screens cover a large share of the workflows we care about.

Why this app exists

• It gives agent-device a stable React Native target that we control.
• It makes skillgym prompts concrete: the agent can inspect real app files instead of answering against an imagined UI.
• It keeps the number of screens low while still covering roughly 50 practical interaction and verification cases.

Screens

• Home: visible-text checks, dismissible banner, modal open/close, async loading, status badge, switch state
• Catalog: search debounce, filter chips, long-list scroll, favorite toggles, cart updates, drill-in navigation
• Product detail: back navigation, quantity stepper, multiline notes, save action
• Checkout form: required-field validation, fill vs type, checkbox state, choice groups, keyboard dismiss, success summary
• Settings: switch rows, accordion content, loading and error states, retry flow, destructive-confirm modal

Navigation uses Expo Router native bottom tabs, so the tab bar itself is also part of the test surface.

Coverage map

These are the main case families this app can support without adding more screens:

• app open and close
• visible text verification with plain snapshot
• interactive discovery with snapshot -i
• press on stable buttons, pills, and rows
• fill on single-line and multiline fields
• type after focus for append flows
• get text on headings, badges, summaries, and accordion content
• is visible and is exists assertions
• wait for async loading and success states
• diff snapshot after dismissals and submits
• long-list scrolling and scrollintoview
• selector-based navigation across repeated cards
• modal open, cancel, and confirm flows
• switch and checkbox state changes
• validation-error and recovery loops
• retryable error banners
• cart counters and quantity changes
• screenshot and recording proof capture

Run locally

This fixture uses an Expo development build, not Expo Go. Expo's development-build workflow installs expo-dev-client, builds the native app with expo run:ios or expo run:android, and then serves JavaScript from Metro with expo start. The app declares @expo/dom-webview directly to keep Expo's development runtime on the SDK 56 native module; Android verification failed when the dev client resolved an older transitive copy.

From the repo root:

pnpm test-app:install
pnpm test-app:ios -- --device "iPhone 17 Pro"

expo run:* keeps Metro in the foreground after launching the app. Leave that terminal running, then use a separate terminal for agent-device or Maestro commands.

Or on Android:

pnpm test-app:install
pnpm test-app:android -- --device "$ANDROID_DEVICE"

If you prefer to work from inside the app folder:

cd examples/test-app
pnpm install --ignore-workspace
pnpm ios

Or on Android:

cd examples/test-app
pnpm install --ignore-workspace
pnpm android

After the first native build is installed, use pnpm test-app:start when you only need to restart Metro for JavaScript or TypeScript changes. Once the app is running, use agent-device against Agent Device Tester like any other target app.

Local Agent Device suites

The repo includes two local suites for iterating on the fixture app:

pnpm test-app:replay:ios
pnpm test-app:replay:android

These run the .ad replay suite in examples/test-app/replays.

gesture-lab.ad verifies gesture pan, gesture fling, gesture pinch, and gesture rotate against the gesture metrics rendered by the Home screen on iOS and Android. Android and iOS simulator sessions also support gesture transform for a combined pan/zoom/rotate gesture. On Android, treat combined transform assertions as qualitative because recognizers can report non-exact centroid, scale, and rotation values for one simultaneous two-finger gesture.

To target a specific iOS simulator or an installed Expo development build, run the underlying command directly so global flags stay before replay inputs:

node bin/agent-device.mjs test examples/test-app/replays \
  --platform ios \
  --device "iPhone 17 Pro" \
  --env APP_TARGET=com.callstack.agentdevicelab \
  --env APP_URL=<project-url> \
  --artifacts-dir .tmp/test-app-replay/ios

Omit APP_URL when the installed development build can discover the local Metro server from its launcher.

The Maestro prototype suite lives in examples/test-app/maestro and runs through agent-device replay --maestro:

pnpm test-app:maestro:ios
pnpm test-app:maestro:android

The Maestro flow includes launchApp, so the suite launches the app inside each test attempt. Start Metro first when the installed development build needs the local bundle.

The suite intentionally covers the compat layer syntax used by public Maestro suites: runFlow file/inline blocks, when.platform, config hooks, deterministic repeat.times, flow env, selectors, input, assertions, and swipe.