* fix: preserve the runner's non-hittable fallback marker through direct selector press
The direct iOS selector press handler spread successText('Tapped <selector>')
after the runner payload, clobbering the 'tapped via non-hittable coordinate
fallback' message that directIosSelectorFallbackDetails keys on — so
maestroNonHittableCoordinateFallbackUsed could never be true end-to-end (the
existing unit test passes because it mocks dispatchCommand above this layer).
Found by the ADR 0011 Layer-3 maestro-fallback contract scenario.
Share the marker string as MAESTRO_NON_HITTABLE_FALLBACK_MESSAGE and keep it
as the success message when the runner reports fallback usage.
* test: interaction contract suite with registry-driven coverage gate (ADR 0011 Layer 3)
test/integration/interaction-contract/ holds one scenario file per dispatch
path, each with a sibling .coverage.ts manifest declaring which guarantee
matrix cells it proves (scenario strings double as the vitest test titles).
index.ts aggregates the manifests statically, and a new Layer-3 gate in
src/contracts/__tests__/interaction-contract-coverage.test.ts fails when any
enforced (runtime/runner/delegated) cell lacks a scenario or when a scenario
claims a waived/inapplicable cell — coverage of the matrix is by
construction in both directions.
Path forcing is natural (no test-only env switch needed): selector/ref
targets take the runtime path, a tapTarget backend takes the native-ref fast
path, x/y takes the coordinate path, and simple-selector clicks on an iOS
provider transcript take the direct runner path (the transcript itself
proves which path ran via assertComplete).
Fixtures are the permanent Bluesky shapes: closed drawer, drawer + visible
twin, edge-grazing container, covered button, non-hittable cell.
Refs #1081
A timed-out adb invocation has no stderr for the failure classifier to
match, so it fell through to the generic retry hint — retrying a wedged
adb server times out identically. Classify exec-layer timeouts on the
structured timeoutMs detail (the signal createTimeoutError sets) as a
'timeout' family with a kill-server/start-server hint, winning over
untrustworthy partial stderr and losing to site hints as before.
Doctor's android inventory sweep already routes adb failures through
attachAdbFailureHint via listAndroidDeviceEntries, so the device-android
check now surfaces the wedged-server hint; a regression test locks that
path.
Closes#1079
- Remove the runner .tap text arm: the daemon only ever sends
selectorKey/selectorValue or x/y taps (src/platforms/apple/interactions.ts),
and daemon+runner ship in lockstep. findElement(app:text:) stays for the
findText probe.
- Drop settledAfterMs from the wait stable result: it always equaled waitedMs
and never shipped in a release (v0.18.3 predates #1059).
- Add a loading hint when wait stable settles on a tree with fewer than 5
nodes (#1078): stability on a nearly-empty tree is a weak readiness signal.
Refs #1078
* build: migrate the library build from rslib to tsdown (Rolldown)
Replace the Rspack-based rslib build with tsdown, the Rolldown-based
library bundler from the Vite toolchain family, so bundling, testing
(Vitest/Vite), linting (oxlint), and formatting (oxfmt) all run on the
same OXC/Rolldown stack.
Outcome vs the rslib baseline (size-report):
- build time: ~53s -> ~2s
- JS raw +16.2 kB (+1.1%), JS gzip +2.7 kB (+0.6%) - the residual gap
is OXC vs SWC minifier tightness, not chunking
- npm tarball -3.0 kB
- CLI --version startup ~3 ms faster; --help within the +/-5 ms
measurement noise of interleaved A/B runs
Chunk-merging experiments (single shared group, entries-aware groups,
small-module groups) all regressed either total size or --help startup
(a merged shared chunk adds +140 ms), so the default Rolldown split
graph is kept. Custom codeSplitting groups also currently trip a
rolldown-plugin-dts bug that re-emits type-only imports as runtime
imports.
Declarations still bundle per entry via tsgo; dist layout, entry names,
and the internal/ worker/daemon entry resolution contract are unchanged.
@microsoft/api-extractor was only consumed by rslib dts bundling and is
removed together with @rslib/core.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS
* ci: only cache the pnpm store when setup installs dependencies
The layering-guard job uses setup-node-pnpm with install-deps: false, so
it never creates a pnpm store. setup-node's post-job cache save then
fails with a path validation error whenever the lockfile hash misses the
cache - which any lockfile-changing PR does. Gate the cache on
install-deps so no-install jobs skip pnpm store caching entirely.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS
---------
Co-authored-by: Claude <noreply@anthropic.com>
ADR 0011 Layer 2: the offscreen tap-point rule now has exactly one home per
side of the wire, proven equivalent by a shared golden fixture table.
- contracts/fixtures/tap-point-policy.json: 12 cases including this week's
real bug shapes (closed drawer fully left, 0.07pt edge-grazing container)
plus edge-inclusive and empty-frame fail-open semantics.
- Swift: pure TapPointPolicy.isAllowed(elementFrame:windowFrame:) extracted;
the ELEMENT_OFFSCREEN guard (onScreenWindowFrame stays the frame getter)
and hasTappableFrame (Maestro fallback) both delegate the decision to it.
Gated XCTest asserts every table case (runs in the existing
AGENT_DEVICE_RUNNER_UNIT_TESTS surface CI already compiles).
- TS: isTapPointInsideViewport extracted from isNodeVisibleOnScreen (no
duplicated math); vitest parity test asserts the same table.
- Registry: parityTable wired on direct-ios-selector/offscreen and
maestro-non-hittable-fallback/offscreen.
* feat: warn on out-of-viewport coordinate taps (ADR 0011)
Implements the coordinate/offscreen guarantee: when a point tap target
falls outside the last-known viewport, attach a warning to the result
instead of silently forwarding the tap. This is a WARNING not an error,
maintaining the escape-hatch semantics of coordinate taps.
The warning is generated at the runtime layer in resolvePointInteractionTarget
using the session's last snapshot to infer viewport bounds via resolveViewportRect.
No snapshot is captured (by design), so the implementation reuses the session's
existing snapshot state. When no snapshot is available, no warning is emitted.
Changes:
- Add `warning?: string` field to PressCommandResult and LongPressCommandResult
types (FillCommandResult already had it for --verify evidence)
- Implement tryResolveOutOfBoundsPointWarning in resolution.ts and export it
for registry verification
- Update interaction-guarantees.ts to mark coordinate/offscreen as runtime-enforced
- Remove coordinate/offscreen from the pinned gap list in the test registry
- Add tests: out-of-bounds point with snapshot → warning; in-bounds → no warning;
no snapshot → no warning
- Carry the warning field through daemon handlers for press/click/longpress
Closes#1081 (coordinate/offscreen gap).
* refactor: drop suppressed re-export; registry points at the enforcement carrier
The registry via now references resolveInteractionTarget (the exported
carrier, same precedent as the verifyEvidence cell) instead of forcing a
suppressed unused-export of the internal helper.
The wait timeout bug (#1075) happened because request-envelope budgets and
on-timeout daemon policy lived in two hand-maintained lists in the daemon
client: isExplicitTimeoutCommand (daemon-client.ts) and
DAEMON_PRESERVING_TIMEOUT_COMMANDS / shouldResetDaemonAfterRequestTimeout
(daemon-client-timeout.ts). A command could fall through both without anyone
noticing.
Both lists are deleted. Each command descriptor (ADR 0008 registry) now
declares a required timeoutPolicy:
timeoutPolicy: {
budget: { source: 'none' | 'flag' | 'positional-parser'; parser? };
envelopeMs: number | 'unbounded';
onTimeout: 'preserve-daemon' | 'reset-daemon';
}
The client derives the request envelope and the on-timeout daemon policy from
the declaration; the +30s margin / never-shrink rule for positional budgets is
preserved generically. Envelope constants move from
src/daemon/request-timeouts.ts to src/core/command-descriptor/timeout-policy.ts
next to the policies they parameterize.
A completeness gate (timeout-policy.test.ts) asserts every public command
declares a policy and pins the deviating sets (preserve-daemon = snapshot/
wait/find; flag budget = prepare/replay/snapshot; positional = wait; envelopes
= prepare 240s, install-like 180s, test unbounded) as bounded diffable lists.
The pre-existing oracle tests in src/utils/__tests__/daemon-client.test.ts
pass byte-for-byte unchanged, proving the migration is behaviorally exact.
Every press/click/fill/longpress dispatch branch now builds its result/
responseData payloads through buildInteractionResponseData
(interaction-touch-response.ts), so identity extras (ref/refLabel/
selectorChain/targetHittable/hint/evidence) are composed in exactly one
place. Hand-rolled per-branch assembly is the class of bug that dropped
fill @ref evidence (#1064 review); a guard test now fails any touch
handler that assembles a responseData literal outside the builder.
Wire shapes are byte-identical: the fill @ref backendResult+identity
shape is preserved behind refBackendWireShape, and the daemon +
integration suites pass unchanged.
Registry: all six responseConstruction cells flip from gap waivers to
the shared runtime cell; the pinned gap list shrinks accordingly.
Refs #1081, ADR 0011.
click @ref / fill @ref dispatch straight to backend.tapTarget/fillTarget,
and a backend fast path can silently "succeed" — delegation-on-error never
triggers there. The ref came from the stored session snapshot, so the node
is already in hand: preflightNativeRefInteraction runs the SAME shared
guards the runtime path uses against that node before the backend call.
- occlusion: isSnapshotNodeInteractionBlocked via assertInteractionNotBlocked
(same covered-element error shape as the runtime path)
- offscreen: isNodeVisibleOnScreen via assertVisibleRefTarget (same
offscreen_ref error shape)
- nonHittable: annotation only — targetHittable/hint attach to the result;
promotion stays a runtime-path behavior and the tapped element never changes
No snapshot in the session, an unresolvable ref, or a missing/unusable rect
make the preflight a no-op: it never adds a capture round trip.
Registry: the three native-ref gap cells flip to kind 'runtime' and leave
the pinned gap list.
Refs #1081
* docs+feat: ADR 0011 interaction guarantee contract, Layer-1 registry and gate
Design for making interaction guarantees hold across every dispatch path
(runtime selector/ref, direct iOS selector, native ref, coordinate,
maestro fallback) instead of eroding at path boundaries one incident at
a time — every interaction bug this week was a (path, guarantee) cell
nobody was watching.
Three layers (ADR 0011): declare the path x guarantee matrix as a typed
registry whose completeness is a compile error; share one implementation
per rule on both sides of the wire with golden fixture tables proving
TS/Swift parity; prove every non-waived cell with contract scenarios
generated from the registry.
This lands Layer 1: the registry with an HONEST initial classification —
ten cells are acknowledged gap waivers (direct-path disambiguation/
occlusion/nonHittable/responseFields/errorTaxonomy, native-ref guards,
coordinate bounds) — plus the gate test that keeps entries truthful:
referenced TS symbols must be exported, runner symbols must exist in the
Swift sources, delegations must land on paths that actually enforce the
guarantee, and the gap list is pinned so it can only change explicitly
in a reviewed diff.
* refactor: apply ADR 0011 design review
- Frame Layer 1 as an honesty/completeness gate, not a truth gate:
it proves every path declared a stance and referenced symbols exist;
behavioral parity starts with the Layer-2/3 fixture and scenario work.
- Split responseFields into responseConstruction (one shared response
construction site — a single Layer-2 refactor) and responseIdentity
(which identity fields a path can provide — per-path capability work);
note the anticipated errorTaxonomy split (codes vs diagnostics).
- Encode the hybrid gap-closure strategy: runner-side parity for
geometry-local rules, delegation-on-error for semantic failures (with
the explicit caveat that delegation-on-error is NOT success-path
parity), and a shared runtime preflight for native-ref where a silent
backend success means delegation never triggers.
- Gap waivers now require a trackingIssue (gate-enforced URL); all 16
pinned gaps link the umbrella issue #1081. The honest reclassification
grew the pin list from 10 to 16 — responseConstruction is a gap on
every path including runtime ones, which is exactly the partial
progress the coarser guarantee was hiding.
- Align ADR wording with the code: parityTable is optional until
Layer 3, required once a runner cell claims parity.
* fix: address registry review — maestro disambiguation honesty, command-scoped verify
1. maestro-non-hittable-fallback/disambiguation was overclaimed: the
guarantee is defined as visible-first/deepest/smallest ranking, but
findElement only implements unique-or-ambiguous scanning. Reclassified
as an intentional waiver (deliberate Maestro-semantics divergence),
mirroring how the direct path keeps its success-path parity gap.
2. verifyEvidence was claimed path-wide on paths that dispatch longpress,
which has no --verify. Cells can now be command-scoped via appliesTo
(non-empty strict subset of the path's commands, gate-enforced), and
the three affected cells scope to press/click/fill.
* fix: agent-UX fixes from Bluesky dogfooding
Four fixes found by driving the Bluesky dev build end to end as an agent:
1. wait honors its user-supplied budget in the daemon request envelope.
The wait timeout travels as a positional, so the client never extended
the request timeout: any 'wait ... 180000' died at the 90s default.
resolveDaemonRequestTimeoutMs now parses wait positionals (shared
src/core parser) and extends the envelope to budget + 30s margin.
2. wait/find timeouts no longer reset the daemon. They are repeated
snapshot captures, sharing snapshot's stalled-bridge failure mode, but
sat on the daemon-reset path — one timed-out wait destroyed every
session the daemon owned (observed live: wait timeout -> runner
killed -> SESSION_NOT_FOUND on the next command).
3. Off-screen selector targets are refused instead of silently tapped.
'tap label=Explore' against Bluesky's closed drawer reported success
while tapping (-161, 265) — coordinates that cannot land:
- selector disambiguation now prefers candidates whose CENTER lies in
the root viewport (isNodeVisibleOnScreen; edge-grazing containers
poke fractions of a pixel into the viewport and must not count),
so 'press label=Profile' picks the visible tab over the drawer item;
- selector-resolved interactions get the same off-screen guard refs
already had (reason: offscreen_selector, with a scroll/open hint);
- the runner's direct selector tap refuses matches whose center is
outside the main window frame (app.frame unions transformed
subtrees, so a closed drawer inflates it) with ELEMENT_OFFSCREEN,
which falls back to the tree-based path.
4. The non-hittable 'may have had no visible effect' hint is dropped when
--verify evidence proves the interactive tree changed — the warning
sat directly next to data contradicting it.
Live-verified against Bluesky on the iPhone 17 Pro simulator: offscreen
tap -> offscreen_selector error; ambiguous Profile -> on-screen tab
(355, 817); 100s wait survives the old 90s envelope with the session
intact.
* refactor: dedupe resolved-target tails and off-screen guards, decompose disambiguation
Fallow follow-up on the dogfood fixes: extract the shared ref/selector
resolved-node tail (describeResolvedInteractionNode), the shared
off-screen check-and-throw skeleton (throwIfOffscreenInteractionTarget —
per-caller messages/hints preserved), and the candidate-accumulation
step out of analyzeSelectorMatches. No behavior change; suites and the
fallow gate are green.
* refactor: harden exec failure wrapping end to end
Follow-up to #1072, closing the structural gaps that let the
missing-processExitError bug class exist:
- requireExecSuccess(result, message, extra?) in utils/exec.ts: guards an
allowFailure result and throws the curated COMMAND_FAILED (flag set via
execFailureDetails) itself. Result-side by design — tool providers and
executor overrides return results without throwing, so an ExecOptions
knob interpreted at spawn time would silently not fire on those paths.
~30 pure guard-and-throw sites across apple/android/web converted; sites
with tolerance branches, cleanup, or exit-0 reachability keep their
explicit shape.
- Source-scan guard (src/__tests__/exec-wrap-guard.test.ts): fails when an
AppError('COMMAND_FAILED', ...) details literal rebuilds the
stdout/stderr/exitCode trio inline without the helper; intentional
holdouts opt out with a documented exec-guard-allow comment. The guard
immediately found three wrap sites the July audit missed (runtime-hints
run-as probe, device-ready not_ready branch, perf export table) — now
converted.
- coerceExecResult at the provider boundaries (executor overrides, apple
tool provider scope, android adb provider scope): SDK-supplied callbacks
cross an unchecked boundary; coercing once there replaces the per-site
String(result.stdout ?? '') defensiveness, which is removed.
- normalizeError stderr excerpts now strip noise prefixes (adb:/xcrun:/
simctl:/error:) before rendering; skip/strip lists stay in the kernel
deliberately (platforms cannot hook normalizeError) with a comment
marking the registry escape hatch if they grow.
- AppErrorDetails exported and documented in kernel/errors.ts: the magic
keys (hint, processExitError, retriable, reason, diagnosticId, logPath,
stdout/stderr/exitCode) now carry types and doc comments.
- runIosDevicectl gains tolerateOutput; the devicectl uninstall path in
app-install.ts reuses it instead of duplicating the wrap + hint logic
(its failure hint now falls back to the devicectl default hint).
* fix: reconcile exec hardening with the adb failure classifier
Main landed the central adb failure classifier (androidAdbResultError +
withAdbFailureHintProvider) while this branch was in flight. Resolution:
- Android call sites keep main's androidAdbResultError form — it composes
execFailureDetails with the classified adb hint, which is strictly
richer than a plain requireExecSuccess conversion for adb invocations.
requireExecSuccess remains the shape for non-adb tools and apple/web.
- Provider result coercion folds into withAdbFailureHintProvider's
enrichment pass (exec/pull/install), so the existing WeakSet memo also
prevents coercer stacking.
- The reverse-remove wrap in adb-executor now uses androidAdbResultError
instead of hand-rolling the trio (flagged by the exec-wrap guard).
- Guard-test scan split into helpers (fallow cyclomatic threshold) and
the perf artifact-tail clone carries fallow-ignore markers on both
platforms, matching the pre-existing marker on the apple side.
- Two expectations updated for the stderr noise-prefix strip: classifier
compose test and perf sampling reason ('device offline', no 'error:').
* fix(errors): close call-site and consumer gaps around the central error system
Audit + iOS/Android dogfood findings (see docs/adr/0010-error-system.md):
- press/click/fill targets that parse as neither @ref, selector, nor point
now fail with INVALID_ARGS grammar guidance (incl. unquoted multi-word
selector values) instead of UNKNOWN 'Expected x to be a finite number'
- daemon command-input validation throws AppError INVALID_ARGS instead of
bare Error surfacing as UNKNOWN
- selector-no-match and stale-ref failures carry targeted hints
(selectorFailureHint / STALE_REF_HINT)
- retriable/supportedOn survive wire rehydration to CLI --json and SDK
(previously dropped at throwDaemonError / toDaemonHttpRpcError)
- MCP tool errors carry code + hint instead of message-only text
- lease busy/capacity use DEVICE_IN_USE (the retriable code)
- asAppError(err, fallbackCode) replaces cause-dropping coercions in the
Apple runner; new default hints for AMBIGUOUS_MATCH, DEVICE_IN_USE,
UNSUPPORTED_PLATFORM, and a distinct UNKNOWN hint
- ADR 0010 documents the error-system conventions
* fix: format touched files and clear fallow audit gate
- privatize SELECTOR_NO_MATCH_HINT / SELECTOR_NOT_UNIQUE_HINT (consumed
only via selectorFailureHint in the same module) and integerSchema
(only used inside command-input.ts)
- dedupe the resolved-node return tail in interaction resolution into
describeResolvedNode
- extract stringDetail/booleanDetail readers so normalizeError stays
under the complexity threshold
* fix: reject unquoted trailing text after interaction selectors
press/click/longpress positionals like 'press text=Gesture lab' used to
silently drop the leftover tokens and act on the truncated selector
(text=Gesture), potentially hitting the wrong element. Reject non-empty
splitSelectorFromArgs rest with INVALID_ARGS guidance that suggests the
merged quoted form (text="Gesture lab"). Fill keeps consuming rest as
its text payload; wait/is/replay-heal already handle rest explicitly.
* feat: classify adb failures with actionable hints at the executor level
Android COMMAND_FAILED errors surfaced raw adb stderr with only the
generic retry-with-debug hint; the sole stderr classification lived in
snapshot.ts and was retry-only. Add a central classifier in
adb-executor.ts that recognizes the common adb failure families (device
offline/unauthorized/not found, more than one device, no devices,
server version mismatch, connection drops, INSTALL_FAILED_* variants)
and attaches the resolved hint — plus retriable for clearly transient
families — to the AppError details in both executor funnels (local
serial exec and provider-scoped exec), so every adb call site benefits
without per-site changes. The adb devices -l discovery call is wrapped
too, where server-mismatch/no-devices failures actually surface.
snapshot.ts now consumes the shared classifier for its retry decision,
keeping only its snapshot-specific dump-race patterns. normalizeError
lifts details.retriable to the top-level error (mirroring the hint
lift) and the daemon typed-error graft lets a throw-site classification
win over the code-level policy. Also fix the port-reverse removal path
throwing a bare Error that surfaced as UNKNOWN.
Update session-appstate-input-perf.test.ts: its Android sampling-failure
fixture uses stderr "error: device offline", which the classifier now
recognizes, so error.hint carries the actionable adb-reconnect hint
instead of the generic "retry with --debug". The result shape (hint
string, details.metric/package, code) is unchanged — only the hint text
improved, which is this change's intent — so the test expectation moves
to the new hint rather than reverting production.
* feat: classify tolerated adb failures and semantic provider methods
Review follow-up closing two classifier coverage gaps:
1. allowFailure paths bypassed the classifier: the executor wrapper only
enriches THROWN errors, but many Android flows run adb with allowFailure,
inspect the nonzero result, and throw a fresh AppError built from its
stdout/stderr — so e.g. "device offline" during uninstall still surfaced
the generic hint. There was no common construction point for these
errors, so androidAdbResultError in adb-executor.ts now IS that point:
it builds the COMMAND_FAILED error from a tolerated result and runs it
through the classifier, keeping hint strings central. The
result-inspecting throw sites (app launch/uninstall, appearance read,
keyboard/clipboard queries, package listing, logcat capture, helper APK
installs, heap-dump pull, uiautomator dump) route through it; a site
hint (heap-dump pull) still wins because attachAdbFailureHint never
overwrites.
Composes with #1072 rather than replacing it: androidAdbResultError
builds its details via execFailureDetails for nonzero exits, so
normalizeError still suffixes the curated message with the first stderr
line while the classified hint/retriable/adbFailure ride along. Semantic
failures thrown at exit 0 (the am start error-on-stdout path) stay
unflagged, matching #1072's deliberate exit-0 exclusion.
2. Provider-scoped enrichment only wrapped exec, but the transfer
helpers prefer the semantic provider methods, so a provider whose
install rejects with INSTALL_FAILED output kept the generic hint.
Enrichment now lives in normalizeAndroidAdbProvider — the single funnel
every provider passes through (scope installation and explicitly passed
providers alike) — wrapping exec plus pull/install/installBundle; a
WeakSet keeps repeated normalization from stacking wrappers. The local
provider needs no wrap since its methods delegate to the already
enriched serial executor.
Tests: androidAdbResultError classification, hint-plus-excerpt
composition through normalizeError, exit-0 no-excerpt guard, site-hint
precedence, and provider install/pull classification in
adb-executor.test.ts; an allowFailure regression on a real production
path (keyboard state query with a nonzero device-offline result) in
device-input-state.test.ts.
* feat: tag daemon artifacts with semantic types
* Normalize artifact type threading through daemon tracking
* Add timeout error to http server artifact test helper
* test: restore artifact wait helper behavior
* test: fail artifact wait helper on timeout
* refactor: model artifactType as optional on wire shapes
Applies the design-review tweak: producer-owned APIs (reserveOutput,
trackDownloadableArtifact, finalization callbacks) keep the required
'DaemonArtifactType | undefined' form so artifact owners must explicitly
decide, while public/wire/result shapes (DaemonArtifact, both artifact
inventory entry types) become 'artifactType?:' — missing metadata is
valid, JSON drops undefined, and remote/older daemons may omit the
field. Construction sites now omit the key for untyped artifacts, and
the finalization test asserts key absence (toEqual cannot distinguish
absent from explicitly-undefined).
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
* feat(interaction): opt-in --verify evidence for press/click/fill (#1047)
Adds an opt-in --verify flag that returns cheap post-action evidence
(foregroundApp, nodeCount, interactiveNodeCount, digest,
changedFromBefore) instead of requiring a full follow-up snapshot to
confirm a mutating command had an effect. The digest hashes the
(type, label, identifier) multiset of an interactive-only capture,
order-independent so it doesn't flip on harmless re-ordering; the
node tree is never serialized back to the client, only the digest and
counts. Default behavior (no --verify) is byte-identical to today.
Implements the approved design from the #1047 issue comment:
- src/utils/ax-digest.ts: new standalone digest module.
- Pre-action digest reuses the snapshot the resolution path already
captures for ref/selector targets (zero extra cost); point targets
opt into one extra baseline capture only when --verify is set.
- Post-action: one interactive-only capture through the same capture
helper, digested and discarded.
- --verify threaded through the CLI flag schema, MCP input schema,
interactionResultExtra allowlist, and MCP output schemas, following
the same plumbing as --double-tap and the #1040 targetHittable
precedent.
- The native-ref/direct-iOS-selector fast paths are skipped when
--verify is set, since they bypass the resolution/capture path
evidence depends on.
* test: cover press --verify with a provider scenario; classify the flag
CI's architecture-progress gate failed on 1 unclassified public flag.
--verify drives real device captures, so it belongs in the
device-observable list backed by an actual provider scenario rather than
the intentionally-outside bucket: the new scenario asserts evidence
(changedFromBefore, digest, nodeCount) on press @ref --verify, that the
verify capture's tree is never serialized into the response, and that
the transcript completes (snapshot -> tap -> verify snapshot; the @ref
path reuses the session snapshot as its baseline, so no extra
resolution capture entry exists).
* fix: type the verify scenario transcript entries
* fix: include interaction extras in the fill @ref response branch
The ref branch of dispatchFillViaRuntime rebuilt responseData from
backendResult/coordinates, dropping interactionResultExtra(result) — so
fill @ref --verify returned no evidence even though the post-action
capture ran (live E2E gap found in PR #1064 review). Spreading the
extras also gives fill @ref the same ref/refLabel/selectorChain (and
conditional targetHittable/hint) fields press @ref already returns.
Adds daemon tests for fill @ref --verify evidence and for the no-verify
path staying evidence-free with no post-action capture.
* fix: report expired daemon resources as expiry, not INVALID_ARGS
TTL/GC'd registry entries (downloadable/uploaded artifacts, resumable
uploads, materialized paths) surfaced as INVALID_ARGS, so agents got
"Check command arguments and run --help" when the resource had simply
expired. Add a shared requireTenantOwnedEntry helper that reports these
as COMMAND_FAILED with reason RESOURCE_EXPIRED and a per-resource
recovery hint, and dedupes the paired wrong-tenant UNAUTHORIZED check.
Also: give request cancellation an explicit "canceled intentionally"
hint instead of the misleading retry default; replace the ad-hoc
INTERNAL_ERROR code (not in the KnownAppErrorCode union) with UNKNOWN
plus a report-a-bug hint; and fix materialized-path TTL expiry of
tenant-owned entries, which threw UNAUTHORIZED in a void-ed promise
(unhandled rejection) and leaked the entry and its temp dir.
* fix: report expired resumable upload tickets as expiry, not INVALID_ARGS
Route requireResumableUpload() through the shared requireTenantOwnedEntry
helper so a missing/expired upload ticket on PUT /upload/direct/:id and
POST /upload/finalize returns the COMMAND_FAILED + RESOURCE_EXPIRED shape
(with a recovery hint) instead of the misleading INVALID_ARGS/400.
Genuinely invalid input (malformed preflight args, bad content-range,
incomplete upload, hash mismatch) stays INVALID_ARGS.
* feat: add wait stable settle primitive
Add `agent-device wait stable [quietMs] [timeoutMs]` (defaults 500/10000)
as a new target kind on the existing wait command/plumbing: it polls the
interactive-only tree via the existing selector-snapshot backend at the
existing wait poll cadence, and resolves once two or more consecutive
captures are unchanged for quietMs, comparing a small local per-node
digest (type, label, identifier, rounded rect). Timeout throws the
standard wait COMMAND_FAILED shape with capture stats (captures,
nodeCount) attached, and the wait-current-surface enrichment now also
recognizes the stable timeout message. Successful results add
settledAfterMs/captures/nodeCount alongside waitedMs.
This rides the same CLI grammar -> command catalog -> daemon handler ->
runtime poll loop as `wait text`/`wait selector`; no new runner protocol.
Polling snapshots intentionally do not update the session cache, since
the interactive-only tree is a settle signal and would otherwise degrade
subsequent ref/get/find lookups in the same session.
Documents wait stable in `help workflow` as the replacement for guessed
sleeps between actions and snapshots in the core loop.
Closes#1049
* fix: bound wait stable by the user timeout when a capture stalls
The stable loop only checked the deadline between captures, so a stalled
backend capture (observed with macOS AX captures) rode the wait into the
90s daemon request timeout instead of the user-supplied timeoutMs.
- Race each settle capture against the remaining wait budget with a real
timer (test clocks advance synthetic time and cannot represent a hung
backend call) and abandon the capture when the deadline wins, throwing
wait_stable_timeout with captureStalled: true and a hint.
- Skip the current-surface enrichment on the stalled path: it fires
another capture, which is just as slow or hung and pushed the response
~8s further past the timeout on macOS.
Verified live against macOS System Settings: cold-runner stall now exits
in 10.1s for a 10s timeout (previously ~90s); warm settle unchanged.
* fix: accept selector-first is form and confirm find actions
is <selector> <predicate> failed with a misleading predicate error because
every entry point read positionals[0] as the predicate, and the trailing
predicate token could not be recovered later: visible/hidden/editable/selected
double as selector boolean keys, so greedy selector parsing swallowed it.
Normalize both argument orders into the canonical predicate-first shape at the
CLI reader and splitIsSelectorArgs, and explain the key/predicate collision in
the error hint when nothing parses.
find click succeeded silently: the delegated click response (with its
'Tapped @ref (x, y)' message) was discarded, and the find formatter printed
fill's raw text field instead of its success message. Carry the success
message through find click and print message-first, so find click/fill/focus/
type confirm exactly like their direct counterparts.
* style: format cli-grammar test
Bare `throw new Error(...)` surfaces to users as code UNKNOWN with a
generic hint. Convert the remaining user-reachable clusters from the
July 2026 error audit to AppError with the right code, preserving
messages:
- metro/client-metro.ts: timeouts, bridge failures, and Metro start /
not-ready errors are now COMMAND_FAILED (the not-ready error also
carries logPath in details); createMetroBridgeRequestError builds on
AppError so non-retryable bridge errors that escape via rethrow get
the right code too
- mcp/command-tools.ts + mcp/router.ts: MCP input validation is now
INVALID_ARGS (JSON-RPC responses unchanged; the router only reads
.message)
- commands/batch/metadata.ts: step validation is now INVALID_ARGS;
with the plain-Error opt-out gone, the now-unused
BatchStepErrorFactory injection seam is removed from
batch-contract.ts
- cloud-webdriver/webdriver-source.ts: xml.ts parser failures are
wrapped at the consumer boundary into a single COMMAND_FAILED naming
what was being parsed
Artifact downloads consumed the inventory entry only from the source
stream's 'end' / response 'finish' handlers. But the response carries an
explicit content-length, so a client's fetch completes as soon as that
many body bytes arrive - which happens right after the final chunk is
written to the socket, while the server's read stream still needs one
more threadpool read to detect EOF and emit 'end'. Under CPU contention
(e.g. the instrumented Coverage CI job) that threadpool completion can
land after the client's follow-up GET /artifacts, which then still lists
the consumed artifact.
Consume the entry as soon as the stream has emitted content-length
bytes, before pipe() writes the final chunk to the client. Any client
that observed a complete download is now guaranteed to observe the
entry as consumed. The 'end'/'finish'/'close' handlers stay as backstops
(idempotent via didCleanupArtifact).
* fix: surface stderr excerpts from hand-rolled exec failure wraps
normalizeError only replaces a generic COMMAND_FAILED message with the
first meaningful stderr line when details.processExitError is true, but
only createExitError in utils/exec.ts set that flag. Hand-rolled wrap
sites (throw new AppError('COMMAND_FAILED', msg, {stdout, stderr,
exitCode}) after an allowFailure run) missed it, so users saw messages
like "xcrun exited with code 22" with a generic hint instead of the
actual stderr excerpt.
Add execFailureDetails(result, extra?) to utils/exec.ts returning the
stdout/stderr/exitCode spread plus processExitError: true (extras spread
after the base keys so call sites can keep truncation or String()
normalization), and convert ~42 exit-guarded wrap sites across the apple
platform (apps launch/terminate paths, devicectl, simulator boot, perf,
perf-xctrace, runner modules, dsym/symbolication, macOS host provider)
and android/web (app-helpers, logcat, devices, perf, settings,
snapshot helpers, app-lifecycle, device-input-state, multitouch-helper,
agent-browser provider).
Sites reachable at exit 0 (parse/stdout checks), failure aggregations,
timeout/forced-kill errors, and sites that already build richer messages
(macos helper JSON, runner-contract early-exit, maestro run-script) are
deliberately left unflagged so stderr enrichment never replaces a more
specific message.
* fix: append stderr excerpts to curated wrap messages instead of replacing them
Spreading processExitError to curated call sites made normalizeError
replace specific messages like 'uiautomator dump did not return XML'
with the bare stderr line, losing the operation context (caught by the
provider-failures integration test). Keep wholesale replacement only for
the information-free generic '<tool> exited with code N' wraps; curated
messages now keep their specific reason and gain the stderr excerpt as a
': <excerpt>' suffix (skipped when already present).
Separate commit rather than amend so the semantics change to
maybeEnrichCommandFailedMessage stays individually reviewable on the PR.
* feat(snapshot): dedup inherited labels/identifiers in client output; promote diff snapshot
Deep RN/AX trees repeat the same accessibility label across a whole parent
chain (ScrollView -> Other -> Button -> inner Button), so agents pay for the
same string 3-4x in one snapshot. When a node's label (or, separately,
identifier) string-equals its nearest ancestor's, the CLI/JSON client output
now omits it and sets an `inheritsLabel`/`inheritsIdentifier: true` marker
instead; the human text renderer shows "[same label as parent]" in its place.
This is output-only: it runs in snapshotCliOutput (src/commands/capture/output.ts),
which feeds both the default text and --json presentations, right before they
are handed to the caller. The in-daemon session tree that selector building
(buildSelectorChainForNode), wait/is/get matching (selectors-match.ts), and
replay/replay-heal read from is never touched, since none of those consumers
go through this serializer -- they all read session.snapshot.nodes directly.
--raw and non-default MCP responseLevels (e.g. digest, which can omit `nodes`
entirely) bypass dedup and stay byte-for-byte.
Also promotes `diff snapshot` as the named post-action verification step in
the core-loop help text, replacing the generic "verify" step: it diffs the
rendered snapshot lines against the previous snapshot in the session and
prints only what changed, which is the actual token-cheap way to confirm a
mutation today (discovery, not new behavior).
* fix: dedup only labels long enough to pay for the marker
Review finding: dedup fired on any string-equal duplicate, but the
replacement markers cost ~21 chars in JSON and ~23 in text — short
duplicated labels (tab bars: Home, Settings) made output BIGGER, not
smaller. Only values of 24+ chars dedup now, which is net-positive on
both surfaces and keeps short labels readable inline. Fixtures updated
to threshold length; added a short-label regression test.
* build: run SWC minifier compress with 3 passes
Measured -9.7 kB raw / -7.0 kB npm tarball on the emitted JS at no
runtime cost. splitChunks minSize tuning was also evaluated for the
tiny-chunk overhead but measured as a byte-identical no-op (the sub-kB
files are dynamic-import boundaries, not split products), so it is
intentionally not included.
* perf(cli): keep help text, replay reporting, and diff runtime off the eager command path
Every CLI invocation eagerly parsed 599 kB of JS. Three static imports
dragged in code that most commands never run:
- cli.js -> cli-help.js (75 kB of help text) via the usage()/
usageForCommand() builders in parser/args.ts, which are only needed
on help and usage-error paths. They now lazy-import cli-help.ts.
- cli.ts/generic.ts -> replay/test/reporting.ts (~17 kB), only needed
by the test command. Now imported at the call sites.
- cli/commands/screenshot.ts -> createAgentDevice (68 kB client-side
command runtime chunk incl. screenshot pixel diffing), only needed
by the diff screenshot branch. Now imported inside diffCommand.
Eager closure of cli.js drops 599 kB -> 431 kB (-28%), ~2.5 ms median
module-load per command invocation. --help/--version fast paths in
bin.ts are unchanged. Package size is unchanged by design (the code
moves to lazy chunks; it does not disappear).
Add a "Guarantees" section to `agent-device help workflow`: statements of
fact for agents to reason from instead of probing behavior with trial
commands. Covers selector ambiguity (folds/extends the existing #1040
paragraph instead of duplicating it), hittability/targetHittable, open
idempotent-foreground vs --relaunch's single simctl launch
--terminate-running-process call, close's runner retention policy plus the
AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS idle-stop window, ref lifetime
(cleared by open/relaunch), diff snapshot's comparison baseline, and wait's
polling model. Each statement is traceable to source, noted in the PR body.
Adds three doc-assertion tests in cli-help.test.ts mirroring the #1040
pattern.
* refactor: centralize known cli command checks
* feat: emit drift diagnostic for registered-but-unhandled commands
Folds #1055's telemetry into this branch: known-command fall-through now
emits cli_known_command_unhandled at error level alongside the distinct
user-facing message, so catalog/dispatch drift is visible in diagnostics
as well as to the user who hits it.
* feat: accept tap as an alias of press
* test: restore #1036 ordering regression coverage, assert tap dispatches press
Making tap a real alias deleted the only tests covering
unknown-command-before-flag-validation ordering (they used tap as the
unknown command). Restore them with a genuinely unknown command, and
strengthen the alias test to assert the dispatched call records press
with positionals preserved instead of merely not-erroring.
* test: cover tap alias composing with the bare-ref hint
Adopted from #1052 (credit: @vku2018): tap e3 must normalize to press
and then surface the @e3 suggestion, proving the alias and the bare-ref
hint compose instead of masking each other.
* fix: report unknown command before flag-support validation
When an unknown command is provided with flags, the parser was
validating flag support before checking if the command exists,
resulting in misleading "Flags are not supported for command X"
errors instead of "Unknown command: X".
This fix reorders the validation to check for unknown commands
first, reporting more helpful error messages. Additionally, when
the unknown command is 'tap', the error now includes a hint
suggesting 'press or click' as alternatives.
Closes#1036
* refactor: drop as-any casts and note the cli.ts sync requirement
Review follow-ups: widen the command lists to readonly string[] instead
of narrowing the value, and document that isCommandKnown must stay in
sync with the cli.ts dispatch fall-through.
When a resolved press/fill/click selector or ref target lands on a node
that reports hittable: false (after any hittable-ancestor promotion), the
success response now carries targetHittable: false and a hint suggesting
the interaction may have had no visible effect. Resolution semantics are
unchanged: no failing, no filtering, no strict-unique matching — iOS AX
hittable flags are unreliable on deep RN trees, and react-navigation's
maestro suite depends on today's auto-disambiguation behavior (#1037).
Also documents the selector disambiguation policy (deepest node, then
smallest area, exact ties fail) in `help workflow`, including that
replay/replay-heal apply the same policy for touch/fill/get-text.
When a user forgets the @ prefix on a snapshot ref (e.g., 'click e3' instead
of 'click @e3'), the positional parser would attempt to parse it as
coordinates, resulting in a confusing error message "Expected x to be a
finite number."
Add a check in readInteractionTargetFromPositionals and readFillTargetFromPositionals
to detect bare snapshot refs matching /^e\d+$/ and throw INVALID_ARGS with
a helpful hint message suggesting the correct @-prefixed form.
Closes#1035
* feat: warm the iOS runner build cache from doctor and boot
Cold-cold startup (first run ever on a machine) pays a ~10s xcodebuild
build-without-testing before the first open can even start the runner.
The background cache prewarm existed but only fired on simulator cold
boot during open — too late for the fresh-machine flow.
- doctor now resolves an iOS simulator from the session or device
inventory and, when the runner artifact is missing, kicks the
best-effort background build and reports it as an ios-runner-cache
check ('build started in the background' / 'artifact cached'). The
artifact builds against a generic simulator destination, so any
simulator record works and the result is shared across runtimes.
- boot fires the same warmup for already-booted iOS simulators (cold
boots already warmed via the onIosSimulatorColdBootStart hook).
- help workflow, help debugging, doctor --help, and the agent-device
skill now say to run doctor first on fresh machines, so agents
(including small models) discover the warmup instead of paying the
build inside their first open. Doctor's help no longer claims to be
strictly read-only.
Verified live: wiped artifact cache -> doctor returns immediately with
the warming check while the build runs detached -> artifact present
~15s later -> second doctor reports cached; first open then skips the
build entirely (~10s off cold-cold).
* feat: show doctor runner warmup progress and guard remote/non-mac hosts
The warmup now emits request-progress lines ('Checking iOS runner build
cache (device)...' / 'Warming ... in the background') so doctor
progressively shows what it is doing, and it is guarded to environments
where a local xcodebuild makes sense: non-macOS hosts and provider-backed
devices skip it entirely (--remote already returned before device
checks). Tests cover both guards with a mocked platform.
* fix: validate the runner cache probe like the ensure path
Review finding (P2): hasCachedAppleRunnerArtifact treated any .xctestrun
under the expected derived dir as cached, while the open path rejects
partial/stale caches via cache-metadata and product-path validation - so
doctor could report 'artifact cached', skip the warmup, and leave the
first open paying the build anyway.
The probe now runs the same evaluateExistingXctestrun no-build evaluation
the ensure path uses (extracted into a shared device-scoped helper) and
reports cached only on reuse_ready. Regression tests cover the empty and
stray-xctestrun-only derived dirs; live-verified that a cache with
deleted metadata now triggers the background warmup instead of reporting
cached, and that a genuinely valid cache still probes true.
The AWS Device Farm runtime is registered with a hardcoded platform of 'android', so baseSessionForLease builds the createSession capabilities with platformName 'Android' before prepareSession resolves the real platform. On an iOS device Device Farm rejects this with "reserved ... invalid values: ['platformName']". Rebuild webdriverCapabilities in the AWS prepareSession using the resolved platform and device name (mirroring the BrowserStack provider).
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf: collapse iOS simulator relaunch into one simctl launch call
open --relaunch on a simulator dispatched close (simctl terminate,
~0.7s), settled 300ms, then dispatched open (simctl launch, ~0.8s).
simctl launch --terminate-running-process does the whole relaunch in one
call (~0.94s), so the fast path skips the close dispatch and the
post-close settle: steady relaunch ~2.9s -> ~2.0s.
The terminateRunningApp option threads from the open context through the
interactor to buildIosSimulatorLaunchArgs; only the simulator bundle-id
launch path consumes it. Runtime hints keep working - they are
user-defaults writes read at the next launch, which the collapsed call
still performs. --clear-app-state keeps the close-first ordering so it
never mutates a running app's container; real devices and Android keep
the two-step relaunch.
* fix: keep close-first ordering for URL relaunches on simulators
Review finding: open <app> <url> --relaunch reached the collapsed fast
path, but the URL dispatch branch never consumes terminateRunningApp -
a deep-link open never launches the app, so there is nothing to attach
the terminate to. The session layer skipped the close while the platform
layer did not terminate either, silently losing relaunch semantics.
The collapse now applies only to the single app-launch form
(openPositionals.length === 1); URL relaunches keep the explicit
close-then-open sequence, with a regression test.
* fix: bound retained iOS runner lifetime with an idle stop
A runner retained after session close (#1021) holds the device's runner
lease indefinitely, which blocks every other daemon on the machine from
using the device - observed in the wild when a leftover verification
daemon squatted a simulator through its retained runner.
Session close now schedules an idle stop when it retains the runner:
if nothing touches the runner within 5 minutes (any ensureRunnerSession
call cancels the timer), it is stopped and the lease released.
AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS overrides the window; 0 disables
idle stops entirely (the previous retain-until-daemon-exit behavior).
The timer is unref'd and cleared on explicit stop and shutdown detach.
Runners warmed by explicit prepare ios-runner are not timed - prepare
expresses intent to keep a warm runner, and its ensureRunnerSession call
cancels any pending stop.
* test: reset idle-stop scheduler mock between session tests
Review finding on #1025: every other mock in the file resets in
beforeEach; without it the retention assertion could pass on call
history leaked from an earlier close test.
Michał Pierzchała