* fix(test): repair Android replay fixtures against live device reality
Three Android fixture defects from the #1482/#1484 full-tier suite, none of
which ever executed in CI (both nightlies since failed on adb infra before
the suite ran). All three verified live on a fresh API 36 emulator with a
pixel_7-geometry AVD and a Release fixture APK:
- 01-navigation-scroll.ad clicked label=Catalog, but the expo-router
NativeTabs cart badge leaks '0 new notifications' into the tab's content
description even while hidden, and unselected native tabs expose no child
text node - exact match can never hit. Target the composed label the
device actually exposes (deterministic at fixture start: cart is 0 after
--relaunch). The badge does not leak on iOS, so the iOS twin keeps
label="Catalog".
- checkout-form-android.ad opened by iOS display name 'Agent Device
Tester'; Android open resolves packages (the APK label is
'Agentdevicelab'), so APP_NOT_INSTALLED was guaranteed. Use the package
id, matching gesture-lab-android.ad.
- gesture-lab-android.ad aimed every gesture at y=700, above the gesture
card (its targets span y754-1329 on pixel_7 geometry; the home screen
gained content above the card since authoring). Re-aim pans inside the
exact-two-pointer zone, flings on the image clear of that zone, and
pinch/rotate/transform at the card center. Verified: full suite passes
2/2 via the public test command (20 + 32 steps replayed).
Refs #1478
* docs(test): pin the Android gesture fixture's validated emulator geometry
The re-aimed coordinates are validated on CI's profile (pixel_7 1080x2400
@420); any booted emulator can receive them via test-app:replay:android, so
the fixture and README now say which geometry the numbers mean and what a
mismatch failure looks like. The checkout twin is selector-driven and
unconstrained.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* chore: drop SkillGym and the repo-health aggregator (#1412 descope)
Remove the SkillGym harness (test/skillgym/), its check-affected lane,
package scripts, and devDependency — the help-conformance bench is now
the single non-gating small-model oracle. skills/ markdown classifies
as docs in the affected-check selector instead of failing open.
Remove scripts/repo-health: its only gating assertion duplicated the
Layering Guard job, its case-count metric imported the deleted SkillGym
suite, and its sole planned consumer (#1424 / PR #1477) was closed with
the Track C descope on #1412.
Verified: check-affected node --test suites, oxfmt, oxlint, tsc,
check:layering, fallow audit vs origin/main, and the full unit suite
(unit-core + subprocess-stub) all pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep
* fix(scripts): fold slow-test budgets into the reporter for production-exports
The Fallow production-exports gate flagged all three budget exports:
their in-file consumer (SLOW_TEST_RATCHET) and the repo-health entry
point that kept the module reachable were both removed in the descope,
leaving the config-loaded reporter as the only consumer — invisible to
--production analysis. The data-only module's second consumer is gone,
so per the boundaries-are-earned norm the constants move into the
reporter instead of gaining a suppression.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep
* docs: align skills/ format policy and purge last SkillGym mention
Address both P2 review findings on #1480: the testing-matrix row and
the selector's formatGate both still claimed oxfmt covers skills/,
while selectChecks classifies skills/*.md docs-only (oxfmt ignores
**/*.md, so the claim was a no-op even before). The matrix now states
the docs-only policy and formatGate drops the dead underSkills fact.
The merged examples/README.md index (from #1469) loses its skillgym
mention.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(examples): add runnable Node.js SDK examples under examples/sdk/
examples/test-app is a fixture and the repo's only prior examples/
content; the real SDK usage patterns lived only in
website/docs/docs/client-api.md with no runnable script anywhere.
Adds four standalone, typechecked examples covering the minimum surface
from #1463: root client session (create -> open -> snapshot/tap ->
close with typed error handling), agent-device/metro
(normalizeBaseUrl/resolveRuntimeTransport), agent-device/contracts
(centerOfRect on a snapshot node), and agent-device/batch (runBatch for
a custom transport). Each imports the published `agent-device/...`
subpaths rather than relative src/ paths.
examples/sdk/tsconfig.json path-maps those subpaths to src/sdk/ so
`pnpm typecheck` (now also run against this tsconfig) checks the
examples in CI without a prior build, workspace link, or publish step.
Running an example for real still resolves `agent-device` as a
self-referencing package after `pnpm build`.
src/__tests__/client-api-examples-drift.test.ts guards the examples
against drifting from client-api.md's subpath API manifest in both
directions, picked up automatically by the existing unit-core vitest
project (no new script or workflow needed).
examples/README.md indexes the new examples and notes that test-app/
remains a fixture, not an example; it is not renamed or moved.
Refs #1463
* fix: address Fallow findings on the new SDK examples
Fallow flagged the four examples/sdk/*.ts files as unused files (not
reachable from any entry point) and three functions as high complexity.
- Register the examples as manual entry points in .fallowrc.json,
matching how other standalone scripts (scripts/patch-xcuitest-runner-icon.ts,
scripts/runner-request-count/run.ts) are already declared.
- Reduce complexity in client-session.ts and contracts-result.ts by
extracting device-resolution/error-reporting and rect-assertion
helpers out of main().
- Reduce complexity in the drift guard's parseSubpathManifest by
splitting bullet-matching and backtick-name extraction into their
own functions.
Verified: pnpm check:fallow --base <PR base sha> now reports no issues,
and pnpm check:tooling / pnpm test:unit stay green.
Refs #1463
* fix: compile client-api.md's actual code snippets, not just its symbol manifest
Addresses review feedback on #1463's drift guard: the existing guard only
parsed the doc's "Public subpath API" bullet manifest and compared imported
symbol names, so a fenced ```ts snippet could drift or stop compiling
without the guard noticing.
Added test/integration/client-api-doc-snippets.test.ts, which extracts every
fenced ```ts block from client-api.md and typechecks it against the real
agent-device/* sources (reusing examples/sdk/tsconfig.json's existing paths
mapping, read via `tsc --showConfig` so there's one source of truth). Free
identifiers that continue a `client`/`snapshot` from an earlier snippet are
stubbed — typed against the real SDK return type, not `any`, so continuation
snippets still get real checking. Lives in the Node integration lane
(test/integration/*.test.ts), not vitest's unit-core: it spawns a real tsc
Program, well past the unit suite's 2.5s budget.
Running this check against the existing doc surfaced real, pre-existing
snippet bugs (unrelated to the new examples), fixed here:
- "sessions.artifacts": `result.cloudArtifacts` accessed without narrowing
the `CloudArtifactsResult | DaemonArtifactsResult` union first.
- "Device cloud sessions": `platform`/`device` were passed into the client
constructor config, which doesn't accept them; moved to the `apps.open()`
call where those fields actually belong.
- "Android ADB providers": the inline `exec` handler had no parameter types,
so it failed under strict/noImplicitAny; annotated with the real
`AndroidAdbExecutorOptions` type.
Two further gaps the check surfaced are pre-existing product/API-surface
questions out of scope for this PR (not the new examples), so they're
allowlisted in KNOWN_DOC_GAPS with comments rather than silently patched:
- "Remote Metro helpers" documents prepareRemoteMetro/reloadRemoteMetro/
stopMetroTunnel/resolveRemoteConfigProfile as public, but none of them are
exported from agent-device/metro or agent-device/remote-config today.
- "Web sessions"/audio probe pass `platform` to `observability.network()`/
`.audio()`, but NetworkOptions/AudioOptions have no `platform` field even
though the CLI's network/audio commands accept `--platform`.
Refs #1463
* fix: close the doc-snippet compiler's stubbing hole and the two suppressed gaps
Addresses the second round of review feedback on #1463's drift guard:
1. stubFreeNamesAndRecompile auto-stubbed every "Cannot find name" as `any`,
so a typo like `cliet.apps.open()` would silently pass on the second
compile. It now only stubs identifiers in an explicit allowlist
(KNOWN_FREE_NAME_STUB_TYPES) — the real SDK-derived continuations
(`client`, `androidClient`, `snapshot`) plus the doc's own invented
host-glue names, each typed precisely rather than loosely. Anything else
is left as a real compile failure. Added a regression test that feeds a
`cliet` typo through the guard and asserts it fails.
2. KNOWN_DOC_GAPS filtered six real compiler errors out of the final
assertion while the test claimed every snippet compiles. Investigated
both and fixed the actual contracts instead of suppressing them:
- `prepareMetroRuntime`/`reloadMetro` (src/metro/client-metro.ts) and
`stopMetroTunnel` (src/metro/metro.ts) already existed and matched the
doc's described workflow almost exactly (same result shape) but were
never re-exported from `agent-device/metro`; same for
`resolveRemoteConfigProfile` and `agent-device/remote-config`. Added
the four exports and fixed the doc's stale function names
(`prepareRemoteMetro`/`reloadRemoteMetro`) and one stale field name
(`profileKey` -> `companionProfileKey` on the prepare call) to match.
- `NetworkOptions`/`AudioOptions` (src/contracts/client-observability.ts)
had no `platform` field even though the CLI's `network`/`audio`
commands accept `--platform` for the same use case, and the client
methods already forward the options object to the daemon generically
(`executeCommand('network'|'audio', options)`) — so this was a type
gap, not a runtime one. Switched both from AgentDeviceRequestOverrides
to DeviceCommandBaseOptions (matching PerfOptions' existing pattern),
closing the gap for real instead of stripping `platform` from the doc.
- The "Android installFromSource()" snippet was missing its
`createAgentDeviceClient` import outright; added it.
KNOWN_DOC_GAPS is gone — every fenced snippet now compiles for real, and
the test's assertion matches what it claims.
3. Switched the raw `execFileSync` calls to `runCmdSync` from
src/utils/exec.ts, per AGENTS.md's process-execution invariant (this is
a .ts integration test, not a packaging fixture that needs to stay
dependency-free).
Refs #1463
* docs: fix stale reloadRemoteMetro() prose reference to reloadMetro()
The prose right after the Remote Metro helpers snippet still named the old
function; the compile guard only checks the fenced snippet, not surrounding
prose, so it didn't catch this leftover from the prior rename.
Refs #1463
---------
Co-authored-by: Claude <noreply@anthropic.com>
* chore(deps): add Renovate config and enforce packageManager pnpm version in CI
Refs #1422
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: bump pnpm to 11.17.0 and format the whole repo with oxfmt
format/format:check drop their hand-maintained path list: oxfmt already skips
node_modules and honors .gitignore, so the only exclusion list is
.oxfmtrc.json ignorePatterns.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mutation): accept either quote style in the affected-lane path filter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): keep fixture-app runtime deps as individual Renovate PRs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Pin transitive dependencies past their vulnerable ranges via pnpm
overrides, scoped to the affected major so unrelated majors elsewhere
in the tree stay untouched:
- undici 7.24.7 -> 7.28.0 (root/website): @limrun/api pins undici to
an exact version even in its latest release (0.44.0), so bumping the
direct dependency can't fix this; override the transitive resolution
instead.
- shell-quote 1.8.4 -> 1.10.0 (examples/test-app)
- js-yaml 4.1.1 -> 4.3.0 (examples/test-app)
- brace-expansion 5.0.6 -> 5.0.7 (examples/test-app)
- @babel/core 7.29.0 -> 7.29.7 (examples/test-app)
- ws 7.5.10 -> 7.5.13 (examples/test-app)
Resolves all 13 open Dependabot alerts (7 high, 3 moderate, 3 low).
Splits the test app's build caching by context instead of running one remote
cache for both.
Locally, `expo run:*` caches the native build on disk via the
expo-build-disk-cache provider, keyed by the Expo fingerprint. A second run with
no native change reuses the first build; a screen edit never rebuilds, because
Metro serves JS. This is the original ask — "next time we don't build unless
native changes" — and needs no token, no network, and no custom provider.
In CI, test-app-build-cache.yml builds a Release binary per platform when the
fingerprint has no artifact yet, and publishes it as a GitHub Actions artifact
named `fingerprint.<hash>.<platform>`. Release, not dev-client, so the JS bundle
is embedded and a consuming job needs no Metro. setup-fixture-app installs it by
downloading the artifact and refreshing the JS with @expo/repack-app, so keying
on the native-only fingerprint stays correct — a JS-only change reuses the same
native binary in seconds. It falls back to an inline build when no artifact
exists yet, so a caller is never left without an app.
Release removes the sharp edges the dev-client cache needed. Its simulator .app
is universal (x86_64+arm64) rather than the active-arch-only slice a debug build
emits, so no architecture tag. It links against the SDK but loading is gated by
the deployment target, which the fingerprint already covers, so no toolchain
tag. And the CLI only narrows *debug* builds to the device ABI, so a Release APK
spans every ABI without the undocumented --all-arch flag. The artifact name
collapses to fingerprint plus platform.
This deletes build-cache-provider.js entirely — with it goes the custom Expo
provider that had to reach GitHub from inside @expo/cli, and every workaround
that forced: the fetch-nodeshim User-Agent shim, the arch/Xcode identity, the
upload-intent handoff. CI now talks to the artifacts API with plain `gh api`
outside the patched fetch, and locally the disk cache never hits the network.
The fingerprint comes from @expo/fingerprint's own `fingerprint:generate` (no
--platform, matching what @expo/cli hashes). Gitignoring /ios and /android is
what makes it machine-independent: the library asks the VCS whether the platform
markers are ignored and, concluding CNG, skips hashing them — so a developer's
prebuild output and a fresh CI checkout agree.
conformance-differential consumes setup-fixture-app, so it gains
`permissions: actions: read` for the artifact lookup.
The artifact lookup is non-fatal: a query outage leaves the id empty and
falls through to an inline build like a miss does, rather than exiting the
composite under set -e and turning a cache blip into a caller failure.
test/scripts/setup-fixture-app-fallback-smoke.sh drives that step's real shell
against a failing gh and asserts source=build; ci.yml runs it.
* test: give the tap-retry differential a fixture control that forces the retry
tap-retry-if-no-change was parked in #1289 for being a coin flip: tapRetries
measured 0 in run 29504440599 and 1 in 29510020718 with no change to the flow or
commit. This re-adds it with a control that holds still, so the retry fires every
run.
The original diagnosis (a dynamic cart badge in the tapped title's subtree) had
the right shape but the wrong scope. maestroSnapshotSignature hashes EVERY node
on screen, not the tapped subtree, so no "static region" of the home screen could
have worked. The actual coin flip is the gesture lab's remote image
(reactnative.dev/img/logo-share.png): whether it lands before or after the tap
decides whether the engine sees "changed" and skips the retry.
So the fixture gets a dedicated inert surface — no state, effects, timers,
images, or pressables — presented as a full-screen modal so iOS detaches the
presenting screen and the tab bar's live badges leave the hierarchy too. On a
real run it is 9 nodes against Settings' 55.
Reached by a launcher on Settings via the Settings TAB, which is a deliberate
choice twice over. A deep link would have kept the launcher out of every other
screen's snapshot, but simctl openurl raises a SpringBoard "Open in app?"
confirmation on iOS 26 even cold, and Maestro's openLink goes through the same
path. And home's "Open settings" button sits below the fold, so reaching it needs
scrollUntilVisible — the engine bug already waived under #1299.
The flow carries no waitForAnimationToEnd: a navigating tap defers a stability
requirement that the next tap settles before resolving its target, so the
baseline signature is already captured on a settled screen. Adding one fails the
flow outright, which is a real engine divergence filed as #1326.
Verified on device (iPhone 17 Pro Max, iOS 26.2, Maestro 2.5.1): 10/10
consecutive differential runs ok, tapRetries [0,0,1] every run — the two
navigating taps correctly do not retry, the inert tap retries exactly once. A
single green run proves nothing here, which is the trap #1300 fell into.
The parking guard in invariants.test.ts is replaced by three guards: the scenario
stays active, carries its tapRetries invariant, and is never waived by a
knownDivergence — a flaky scenario must be fixed, not declared.
Fixes#1300
* chore: gitignore expo prebuild output in the test app
Building the fixture app locally (what .github/actions/setup-fixture-app does in
CI) runs expo prebuild and generates examples/test-app/ios/. It is generated and
untracked but not ignored, so it shows up in git status and a `git commit -a`
would sweep the whole native project in. Same for android/ when building there.
Scoped to examples/test-app, so the repo-root android/ — which holds real tracked
sources like android/ime-helper — is unaffected. Nothing is tracked under either
path today, and the CI fixture-app cache key hashes src/**, app/**, modules/**
and the config/lockfiles, so it does not reference these and is unaffected.
* feat: polish replay test progress reporter
* test: stabilize replay reporter cursor test in CI
* refactor: dedupe replay reporter live progress checks
* fix: make Expo build cache path configurable
* test: migrate test app to expo dev client
* docs: align test app device targeting
* docs: clarify dev client setup tradeoffs
* docs: remove stale sdk reference
* fix(daemon): timing-safe token comparison and daemon.json permission hardening
Use crypto.timingSafeEqual (via SHA-256 digests, length-independent) for the
three daemon token checks, and chmod daemon.json to 0600 after writes since
writeFileSync only applies mode on creation.
https://claude.ai/code/session_01LXZXzxi55sZ11DSyqWyBA2
* fix(deps): clear CVE-2026-9277 by overriding shell-quote to >=1.8.4 in test-app
Override added to examples/test-app/pnpm-workspace.yaml (package.json-level
overrides are silently ignored for this nested app, see the comment there).
Lockfile change is limited to shell-quote 1.8.3 -> 1.8.4; pnpm audit is clean.
https://claude.ai/code/session_01LXZXzxi55sZ11DSyqWyBA2
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: resolve test-app dependabot alerts
The postcss/uuid overrides added in #464 stopped applying once test-app
ended up nested under the repo-root pnpm-workspace.yaml: pnpm only honors
overrides from a workspace root, so test-app's package.json `pnpm.overrides`
were silently ignored and the lockfile drifted back to vulnerable versions.
Move the overrides into a dedicated examples/test-app/pnpm-workspace.yaml so
test-app is its own pnpm root and the overrides are honored, and add scoped
overrides for the two remaining alerts:
- postcss 8.4.49 -> 8.5.12 (XSS in CSS stringify)
- uuid 7.0.3 -> 14.0.0 (missing buffer bounds check)
- ws@8 8.20.0 -> 8.21.0 (uninitialized memory disclosure)
- brace-expansion@5 5.0.5 -> 5.0.6 (ReDoS / max bypass)
ws and brace-expansion overrides are scoped to the vulnerable majors so the
non-vulnerable ws@7 / brace-expansion@1 copies in the tree are left untouched.
* chore: drop dead lodash-es override, document test-app workspace
- Remove the no-op `lodash-es` override from the root package.json (leftover
from #368). lodash-es is no longer in the dependency tree, so the override
resolved to nothing; regenerating the root lockfile is a no-op.
- Add a comment to examples/test-app/pnpm-workspace.yaml explaining why the
file exists, so it isn't "tidied away" and the override drift reintroduced.
Michał Pierzchała