* fix(test): deterministic temp-dir cleanup across node --test lanes
node --test has no global setup/teardown hook, so unlike Vitest (#1593) every
node --test package.json script (maestro:conformance, mutation:test,
check:affected:test, check:coverage-changed:test, check:layering,
depgraph:test, check:tmpdir-leaks:test, check:contention-retry,
test:fixture-cache, test:smoke(:web), test:integration:node,
test:concurrency-torture) still created scratch directories against the
real, unredirected os.tmpdir(), with cleanup only as reliable as each call
site's own try/finally — which a crash, OOM, or timeout kill bypasses
entirely.
Add scripts/node-test-tmpdir.ts: it wraps the whole `node --test`
invocation as a child process, redirecting TMPDIR to one disposable,
pid-tagged directory (shared root/prefix with the Vitest lane) and removing
it from the process 'exit' event, which fires on normal completion, a
thrown error, or a forwarded SIGINT/SIGTERM alike. Every node --test script
now runs through it. check-tmpdir-leaks.ts already scans by root/prefix, so
it covers both mechanisms with no changes to its detection logic.
Verified: a node --test process that mkdtemp's then gets SIGKILL'd leaves a
directory behind unwrapped; wrapped and SIGTERM'd, TMPDIR is redirected and
the directory is gone with no orphaned processes. All 13 wrapped lanes and
the full Vitest suite (5,591 tests) pass with zero residual
agent-device-test-run-* directories after the run.
Fixes#1595
* test(tmpdir): ratchet every node --test script through the wrapper
The 13 lanes wrapped in package.json were a one-time hand sweep with
nothing enforcing the pattern going forward — a 14th node --test script
added later without scripts/node-test-tmpdir.ts would silently reopen
#1595 for that one lane.
Add a structural check to scripts/node-test-tmpdir.test.ts (now part of
check:tmpdir-leaks:test) that reads package.json and fails if any script
invokes `node ... --test` without routing through the wrapper. Dumb
string matching over the scripts map, no shell parsing, with an explicit
(currently empty) NODE_TEST_WRAPPER_BYPASS_ALLOWLIST for any lane that
must legitimately bypass it. Verified it both passes on the current
package.json and fails when a synthetic unwrapped `node --test` script is
added.
* fix(test): preserve the Swift cache and close the raw node --test bypasses
Review on #1661 found two gaps:
1. The wrapper only overrode TMPDIR, so it discarded and forced a
recompile of the durable Swift compiler cache every run instead of
mirroring vitest-tmpdir-global-setup.ts's carve-out for it. Read
os.tmpdir() before the child's TMPDIR redirect takes effect and set
AGENT_DEVICE_SWIFT_CACHE_DIR from that (only when unset), same as the
Vitest lane — the two now share one durable cache instead of each
discarding and recompiling their own. Added a probe assertion
(scripts/node-test-tmpdir.test.ts) that fails without the fix and
passes with it (verified both ways).
2. docs/agents/testing.md documented raw `node --test` commands for the
iOS smoke files, and the android/ios/conformance-regenerate/nightly
workflows invoked `node --test` directly outside package.json. Routed
all of them through scripts/node-test-tmpdir.ts so the documented
local commands and CI lanes get the same crash/timeout-safe cleanup
the package.json scripts already have.
* chore(deps): add Renovate config and enforce packageManager pnpm version in CI
Refs #1422
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: bump pnpm to 11.17.0 and format the whole repo with oxfmt
format/format:check drop their hand-maintained path list: oxfmt already skips
node_modules and honors .gitignore, so the only exclusion list is
.oxfmtrc.json ignorePatterns.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mutation): accept either quote style in the affected-lane path filter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): keep fixture-app runtime deps as individual Renovate PRs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(android): consolidate touch injection and gesture viewport into the persistent snapshot helper (#1275)
One Android automation helper now owns snapshot capture, gesture viewport
resolution, and canonical one-/two-pointer plan injection. A live persistent
helper session executes gesture/viewport commands over its socket protocol;
without a session the same APK runs one-shot via am instrument. The separate
one-shot multitouch helper APK is deleted (atomic replacement, no fallback).
Touch scheduling/injection is extracted into focused Java classes
(TouchPlan, TouchPlanInjector, PointerEventSchedule, GestureViewportReader)
instead of growing SnapshotInstrumentation. ADR 0013 amended.
* fix(android): stop a structurally-failed helper session before the one-shot viewport retry
A structured ok=false viewport response leaves the session process alive, and
Android permits only one instrumentation owner of UiAutomation - running the
one-shot fallback against a still-live helper contends with it and masks the
original structured failure. Stop the session first; regression pins that the
one-shot retry only executes once the session is gone.
* refactor(android): extract helper touch dispatch into focused classes; split session tests; document helper API v2 (PR #1281 review)
Addresses findings 2 and 3 from PR #1281 review (finding 1, viewport
session-stop ordering, was already fixed in 5961b9247).
- Extract SnapshotInstrumentation.java's one-shot/session touch dispatch
into TouchCommandHandler.java (viewport/gesture population, UiAutomation-
parameterized) and SessionResponseWriter.java (session response encoding),
with shared PROTOCOL/HELPER_API_VERSION/OUTPUT_FORMAT constants moved to
a tiny HelperProtocol.java. SnapshotInstrumentation.java shrinks from 908
to 803 lines; wire format (header keys/values, error shapes) is unchanged.
- Split touch-helper.test.ts (~720 lines) into touch-helper.test.ts
(normalize/parse/one-shot gesture+viewport+result envelope) and
touch-helper-session.test.ts (persistent-session transport + fake-session
harness), moving shared device/plan/install-probe fixtures used by both
files into touch-helper.fixtures.ts.
- Update android/snapshot-helper/README.md to document helper API v2: the
one-shot viewport/gesture modes, the android-touch-plan-v1 payload shape,
and the persistent session's socket command/response contract.
* fix(android): invalidate helper session after APK replacement; recycle viewport windows; align ADR 0002 (PR #1281 re-review)
- prepareAndroidTouchHelper now mirrors the snapshot path: when
ensureAndroidSnapshotHelper replaces the APK (install.installed), the
persistent session started against the previous binary is stopped before
any touch command, so gestures run one-shot against the fresh install
instead of a dead/stale session socket. Regression: 'an APK replacement
stops the stale session and the gesture runs one-shot' drives a live fake
session through an outdated-install probe (new outdatedVersionAdb fixture)
and asserts the session socket receives no gesture, the one-shot
instrumentation path executes, and the session is gone.
- GestureViewportReader.read no longer leaks AccessibilityWindowInfo: a
single pass copies the active/focused and first-application bounds into
locals, every window is recycled in a finally, and the existing precedence
(active/focused app bounds, root-in-active-window, fallback app bounds,
IllegalStateException) is applied afterwards, unchanged.
- ADR 0002's touch-synthesis paragraph is amended (2026-07, issue #1275) to
the shared-helper model, consistent with ADR 0013: a live persistent
helper session executes touch commands directly, one-shot otherwise; the
old stop-before-gesture requirement is kept as historical context.
* fix(android): resolve touch helper artifact from the ADB provider like snapshots do (PR #1281 re-review)
- prepareAndroidTouchHelper now uses the same artifact precedence as
snapshot capture: the scoped adbProvider's snapshotHelperArtifact when
present, otherwise the bundled resolver (whose strict unavailable error
is preserved). The provider artifact drives both the install decision
and the instrumentationRunner used for one-shot commands, so an
ADB-backed provider that supplies a helper artifact but no native touch
override runs snapshots and gestures against the same single helper
(issue #1275). Regression: 'a provider-supplied snapshotHelperArtifact
overrides the bundled artifact for touch' pins the provider packageName
on the install probe, the provider apkPath on the install call, the
provider instrumentationRunner on the am instrument args, and that the
bundled resolver is never invoked.
- ADR 0002 now states explicitly that one-shot retry applies only to
idempotent reads (viewport) after the failed session is stopped;
non-idempotent gesture failures surface directly.
- Helper README session transport corrected: a persistent process serving
one short-lived socket connection per request (the server closes each
accepted connection), not a single long-lived connection.
* fix(android): guard touch session reuse on helper identity, stop mismatched sessions (PR #1281 re-review)
Persistent helper sessions are keyed by device, so touch reuse must also
prove the live session runs the helper binary the command selected. The
session record now stores its helper identity (packageName, runner,
helperVersion, helperVersionCode — the same values that feed the snapshot
session identity), and runAndroidSnapshotHelperSessionTouchCommand takes
the requesting helper identity: on mismatch (packageName/runner always;
version/versionCode when both sides define them) it stops the session and
returns undefined, so the touch command runs one-shot against the selected
artifact — gestures never start sessions; the next snapshot restarts one
with the right artifact. Matching identity reuses the session as before.
Snapshot capture identity and behavior are unchanged.
Regression: 'a provider artifact that mismatches the live session helper
stops it and runs one-shot' — a live fake session from the bundled fixture
artifact, then a gesture through an ADB provider supplying an
already-current artifact with a distinct packageName/runner (no install):
the old session socket receives zero gesture commands, the session is
stopped, the one-shot am instrument args end with the provider runner, and
helperTransport is 'instrumentation'.
* fix(android): include artifact sha in helper session identity; evict stale install memo entries (PR #1281 re-review)
Same-version binary replacement changes only the APK sha, so identity
guards keyed on package/runner/version/versionCode could not detect a
crossover between two artifacts that differ only in bytes:
- The artifact sha256 now joins the helper identity end-to-end:
AndroidSnapshotHelperCaptureOptions gains helperSha256 (snapshot.ts
passes artifact.manifest.sha256 alongside version/versionCode), the
session record stores it, createSessionIdentity includes it (making
snapshot session reuse sha-aware, consistent with the install path's
existing sha check), and the touch identity guard compares it via the
same both-defined rule.
- ensureAndroidSnapshotHelper's install memo now evicts every other
cached decision for the same device+package when it records an
install/current decision, so installing B invalidates A's stale
'current' memo and a later command selecting A re-inspects the device
instead of skipping the sha check.
Regressions: 'a same-version artifact with a different sha stops the live
session and runs one-shot' (touch-helper-session.test.ts — B owns the live
session, a gesture selecting same-version different-sha A sends zero
commands to B's socket, stops it, and completes one-shot) and 'installing
a same-version different-sha helper evicts the stale install memo'
(snapshot-helper.test.ts — A:current cached, B installed, selecting A
re-inspects and reinstalls instead of serving the stale memo). Both
verified to fail without their fix.
* feat: add Android test IME helper for deterministic text entry (#1198)
Ships a headless InputMethodService (android-ime-helper) as a third Android
helper APK, replacing the visible system keyboard during automated sessions.
Renders zero accessibility nodes and accepts Unicode/CJK/emoji text over a
base64-encoded broadcast channel, fixing both the settle-diff IME-chrome
flood and the ASCII-only adb-shell text entry limit in one structural fix.
- android-ime-helper/: InputMethodService + build/package scripts on the
existing helper-APK toolchain (javac+d8+aapt2+zipalign+apksigner).
- src/platforms/android/ime-helper.ts, ime-lifecycle.ts: install/version
lifecycle (shared with the other two helpers via the new
helper-package-install.ts), activation on session open, and on-device
restore-hygiene (previous IME persisted to a device settings key so any
daemon/state-dir can recover it; restored on close, daemon teardown, and
daemon startup for orphans left by a crashed run).
- input-actions.ts: fill/type route through the helper's broadcast channel
when active, unicode-safe; unchanged ASCII-shell fallback otherwise.
- doctor: new android-test-ime check flags a stuck helper IME with a
copy-pasteable `adb shell ime set` remediation command.
- Gating: default-on for emulators, opt-in via `open --test-ime` on real
devices.
- Dead-weight: rewrote the manual ADBKeyBoard workaround doc, dropped the
now-provably-live skillgym non-ASCII eval case, updated the ASCII
fallback's error message to point at the helper instead of dead-ending.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(#1201 review): permission-gate the IME receiver, fix CI, add opt-out
Addresses the independent review's blockers and should-fixes.
SECURITY (blocker 1): the text-injection receiver was RECEIVER_EXPORTED with
no gate — any co-installed app could inject text into the focused field while
the test IME was active. Fixed by requiring the WRITE_SECURE_SETTINGS sender
permission on the (in-process, dynamically-registered) receiver: adb shell
holds it, third-party apps cannot. The reviewer's suggested exported=false +
explicit-component approach was tried first but empirically breaks delivery on
API 36 (adb shell cannot reach a non-exported receiver there) — documented in
the helper README. Live-verified: a purpose-built rogue APK's broadcasts
(implicit and package-scoped, no permission) are silently dropped, field
unchanged; adb shell's bare broadcast still injects. Added
ime-helper-security.test.ts asserting the permission gate and that no
permissionless exported registration returns.
CI (blocker 2): (a) added `testIme` to integration-progress-model flag buckets
(Integration Tests was red on the unclassified flag). (b) mocked
resolveAndroidImeHelperArtifact in session-doctor-android / ime-lifecycle /
input-actions-test-ime tests so they no longer depend on android-ime-helper/dist
existing on disk (Coverage was red on a fresh checkout); verified by running
them with dist removed.
Should-fixes: added `--no-test-ime` to opt out on emulators (tri-state gating,
parser-tested); PR body's "byte-identical" claim corrected to size/CRC-match.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(#1201): pin the API-36 exported-receiver constraint in a comment
The RECEIVER_EXPORTED flag cannot express why it must stay exported. Add a
one-line note so a future hardening pass doesn't switch to RECEIVER_NOT_EXPORTED
and silently break the CLI (adb shell can't deliver explicit broadcasts to
non-exported components on API 36+; WRITE_SECURE_SETTINGS is the actual gate).
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(#1201 review): harden IME restore lifecycle (blockers 1 & 2)
Blocker 1 — a failed restore no longer deletes the recovery value. restore now
reads back default_input_method after `ime set` and only clears the persisted
previous-IME record on a confirmed-successful restore; a failed set keeps the
value so a later retry / startup recovery / doctor remediation can still
un-strand the user off the helper IME.
Blocker 2 — startup orphan-recovery no longer overwrites/races user state.
It only restores when the device's CURRENT default IME is still our helper
(so a user who legitimately switched away is left alone), and skips any device
a live session in this process owns (the fire-and-forget startup vs. concurrent
`open` race — activate now marks the device active BEFORE the `ime set`, so any
recovery pass that could observe the helper active also observes the flag and
skips). Never persists the helper itself as the previous IME. activate also
verifies its own switch via read-back.
Exported ANDROID_IME_HELPER_SERVICE_COMPONENT so restore compares the active IME
without reading the packaged artifact from disk. Tests: failed-restore keeps the
value (+ later recovery succeeds), startup no-op when current != helper, startup
skips a live-owned device.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(#1201): delete unused ACTION_ENTER path, baseline test-only export seams
Rebased onto main (#1202 production-unused-exports gate). Two follow-ups:
- Deleted the unused ACTION_ENTER broadcast end-to-end (TS sendAndroidImeHelperEnter
+ its test, Java handler, README): nothing routes through it — `keyboard enter`
uses the keyevent ENTER path — so the new production-exports gate flagged it as
dead production code. Removed rather than grandfathered.
- Added the three legitimate test-only seams (resetAndroidImeHelperInstallCache,
resetAndroidTestImeActivationCacheForTests, setAndroidTestImeActiveForTests) to
fallow-baselines/production-unused-exports.json, matching how the sibling helper
reset functions (resetAndroidMultiTouchHelperInstallCache, ...) are already
grandfathered there.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(#1201): stop daemon-startup adb spawn on non-Android hosts (macOS Smoke)
Root cause of the red macOS Smoke shard (proven, not hand-waved): the
fire-and-forget restoreOrphanedAndroidTestImeOnDaemonStartup ran `adb devices`
at EVERY daemon startup, on every platform. GitHub macOS runners ship the
Android SDK, so this cold-started the adb server mid-replay and destabilized the
macOS System Settings replay timing — the failed job's cleanup shows
"Terminate orphan process: pid (N) (adb)"; main's green runs spawn no adb.
Fix: gate the startup orphan scan behind a host-side marker written in the
daemon state dir when a session activates the test IME (mirrors the
managed-web-browser orphan-cleanup `installed` gate). A host that never uses the
Android test IME — the macOS CI runner included — never writes the marker and so
never spawns adb at startup. The marker is cleared once nothing is left stuck.
Adds SessionStore.resolveStateDir(); tests: startup recovery does not scan adb
when no marker exists (+ marker cleared after a clean scan).
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* chore(#1201): suppress fallow class-member false-positive on state-dir accessor
CI's Fallow audit flags SessionStore.resolveDaemonStateDir as an unused class
member, but it is called via sessionStore.resolveDaemonStateDir() in
session-open.ts — fallow's class-member tracer just doesn't resolve a method
call sited inside a call argument. Renamed for clarity (avoids the collision
with config.ts's free resolveStateDir) and added the localized
fallow-ignore-next-line unused-class-member suppression.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(#1201 review): durable persist before switch + device-scoped recovery markers
Addresses devin-ai-integration's two P1 restore-safety blockers on 19cbce79d.
P1.1 — durably persist the restore target BEFORE the global IME switch.
writePersistedPreviousIme now checks the `settings put` exit code AND reads the
value back, returning a boolean. activate persists first and, if it cannot be
persisted, fails open to the existing input path WITHOUT switching — a rejected
`settings put` can no longer strand the user on the helper with no restore
target. Regression test added.
P1.2 — close the marker crash/offline blind spot. Recovery intent is now
recorded per device, BEFORE the switch (ordering: durable record -> marker ->
ime set), eliminating the post-switch/pre-marker crash window. Markers are
device-scoped and each is retained until that device is actually observed clean:
an offline/disconnected-but-stuck device keeps its marker and is recovered on
reconnect instead of being cleared because the current `adb devices` scan saw no
set-failed. Close-time restore clears only that device's marker (stateDir plumbed
through teardown/close). Tests cover the persist-failure, post-switch/pre-marker
crash, offline-then-reconnect, live-session-owned, and user-switched-away cases.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* refactor: remove JS test wrappers, run .ad replay suites directly on CI
The platform JS test files (ios.test.ts, android.test.ts, macos.test.ts)
were thin wrappers that shelled out to `agent-device test` and asserted
on JSON counts. Since the CLI already exits non-zero on failure, the
wrappers added no value. CI now invokes the replay suites directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add local replay entrypoints and restore physical device test
- Add pnpm scripts for each replay suite (test:replay:ios,
test:replay:ios-device, test:replay:android, test:replay:macos)
so local runs still exercise platform replays
- Restore physical-device iOS replay step in CI workflow
(conditional on IOS_UDID variable)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Michał Pierzchała