mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
v0.19.0
28 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
8ef4e73408 | refactor: derive command exposure lists from descriptors (#1137) | ||
|
|
5c5fa012f7 |
feat: --settle returns the settled diff in the interaction response (#1101) (#1106)
* feat: --settle returns the settled diff in the interaction response (#1101) press/click/fill/longpress --settle executes the action, waits for the UI to go quiet (wait stable's loop, shared via stable-capture.ts), and returns the settled diff vs the pre-action tree in the same response — one round trip instead of the interact -> observe pair. - payload: changed lines only (bounded), summary counts, added-line refs, refsGeneration; best-effort (settled:false + hint on never-quiet content, never an action failure); --verify shares the settle captures - ref issuance: the settled tree becomes the session snapshot; a diff-carrying settle response clears snapshotRefsStale and the MCP layer merge-only re-pins added-line refs at the settle generation - grammar: --settle + --settle-quiet <ms> + --timeout <ms> (flag-sourced descriptor budget with new envelope:'widen' semantics mirroring wait) - ADR 0011: new settleObservation guarantee classified on every path with contract scenarios per enforced/delegated cell * test: give the two contention-flaky doctor scenarios explicit budgets The doctor provider scenarios sit at ~5s of real daemon-harness work on a loaded host and flake at vitest's 5s default during full-suite runs (the known contention flake AGENTS.md documents). Same in-file precedent as the Metro-probe scenario's 10s budget. * fix: move SettleParams to contracts to satisfy the layering DAG daemon/handlers/interaction-flags.ts imported the type across the daemon -> commands boundary (R2 commands-floor). The tuning params are part of the interaction contract like SettleObservation, so they live in contracts/interaction.ts and both layers import from there. * feat: keep settle diffs content-first — drop Key nodes, added lines win the cap Bluesky dogfood: a fill that summons the iOS keyboard spent 49 of the 80 capped diff lines spelling out QWERTY keys, and a screen transition with 269 removals could starve out the added lines entirely. Key-type nodes are now filtered from both diff sides (the [keyboard] container line still signals presence), and under truncation added lines — the ones carrying fresh refs — win slots over removals. * docs: state the core loop in the top-level help starting point Benchmarked with headless haiku/sonnet agents given only --help: both models skipped the help-workflow pointer and started with plain snapshot (38KB payloads they then had to re-read from files). One core-loop line at the starting point is what teaches snapshot -i and --settle to models that never read a second help page. * fix: preserve settle digest refs for mcp * fix: reduce settle fallow complexity * fix: surface settle output in CLI text * fix: complete settle handling for longpress * refactor: localize daemon timeout envelopes * refactor: deepen post-action observation * refactor: centralize post-action observation planning * refactor: derive settle capability from descriptors * refactor: trim settle descriptor helpers |
||
|
|
83d54614d8 |
fix: bound iOS capture stalls and make runner recovery session-preserving (#1105) (#1107)
* fix: bound iOS capture stalls and make runner recovery session-preserving (#1105) Runner (Swift): - Coalesce duplicate transport sends of one commandId onto the in-flight execution instead of enqueueing them again behind it (capture pileup). - Fail fast with RUNNER_BUSY while watchdog-abandoned main-thread work is draining; escalate to RUNNER_WEDGED past 120s so the daemon recycles. - Carry the capture-plan deadline into the query-sweep and private-AX ladder tiers so chained recovery cannot stack past the watchdog. - Penalize the tree backend after a slow (>5s) or abandoned capture and lead subsequent regular plans with private-AX for that bundle (sticky, 120s), stamped recovered/budget so the deferral stays observable. Daemon (TS): - Per-request runner recycle budget: at most one invalidate+reboot per request, then fail fast with an actionable, session-preserving hint. - RUNNER_WEDGED joins the runner-fatal invalidation reasons. - Interaction commands (click/fill/longpress/press/type/get/is) preserve the daemon on request timeout like snapshot/wait/find: resetting it destroyed every healthy app session the daemon owned. * fix: suppress AX-broken-screen snapshot issues so the runner survives capture XCTest records 'Failed to get matching snapshot: kAXErrorIllegalArgument' issues for every XCUIApplication query on AX-broken screens; after a few of them the test case tears down the moment the in-flight command completes, killing the long-lived runner after every capture of the screen (the restart loop behind #1105). The capture plan already classifies and recovers from AX failures, so this issue class is noise: swallow exactly it in record(_:); everything else still records and still drives XCTEST_RECORDED_FAILURE. * feat: time-slice the XCTest tree capture on a worker thread The tree snapshot XPC is a single blocking call whose duration moves with live content (4s to minutes on Bluesky profile screens); no in-process budget could bound it on the main thread. Run it on a worker bounded to an 8s slice: on timeout the plan penalizes the tree backend, skips the XCTest-backed tiers while the abandoned XPC drains (they would block behind it inside testmanagerd), and recovers through the private AX backend, which does not use testmanagerd. * tune: lower the tree-backend penalty threshold to 3s The Bluesky profile tree grind measures ~4.5s before kAXErrorIllegalArgument, just under the old 5s threshold, so every capture re-paid the doomed grind (9s each). At 3s the second capture onward defers to private AX (2.4s snapshot, 4.9s press on the live repro). * fix: harden the AX-issue suppression per review - Require the kAXError token: 'Failed to get matching snapshot: Timed out while evaluating UI query.' is a genuinely-hung-query signal and must keep recording (and keep driving XCTEST_RECORDED_FAILURE). Sibling AX server codes (kAXErrorCannotComplete, ...) are deliberately included: any AX-server rejection inside a matching-snapshot fetch is the same capture-plan noise. - State honestly that the override is suite-global and why (tap-triggered queries record the same noise; command outcomes stay honest via their own error paths). - Lock-guarded suppressed-issue counter following the file's existing abandoned-work counter pattern, logged with each suppression. - Unit-test the pure classifier (record(_:) itself is not invoked: the must-record variants would record real failures in the test run). |
||
|
|
cccd34fb27 |
docs: refocus AGENTS.md on principles and enforcement gates (#1097)
* docs: refocus AGENTS.md on principles and gates; index ADRs; extend CONTEXT.md vocabulary AGENTS.md: replace the routing/command-family prose maps (already drifting from the code) with pointers to the self-describing, parity-tested registries; add the two sections agents actually cannot rediscover cheaply — Principles (one line per incident-backed lesson) and Enforcement gates (the classify-don't-suppress index); extend the module-size guidance from raw LOC caps to answer-one-question files, 1:1 test topology mirroring (removing the integration-aggregation exemption that produced 3,400-line test files), sibling fixture modules, claim collocation, and boundary-only barrels; record the dev-loop staleness triple (dist/daemon/adopted-runner), the tsgo typecheck, the Gatekeeper first-node-exec stall, the DEVICE_IN_USE signature, and the contention-flake protocol; append the two gate steps to the new-flag checklist. CONTEXT.md: vocabulary for the ADR 0011 domain (dispatch path, guarantee cell, owned waiver, parity table, coverage manifest, delegation-on-error, ref generation pin) and an architecture paragraph positioning ADR 0011 as ADR 0008's interaction-semantics counterpart. docs/adr: flip 0011 to Accepted (implemented through Layer 3) and add a read-this-when index that names the registries as the living source of truth over ADR prose. * docs: defer versioned-ref references to the implementing PR Review sequencing note on #1097: these lines described #1096 behavior not yet on main. They move to #1096's branch so docs land with the implementation and the two PRs merge in any order. |
||
|
|
c506ddf3e7 |
RFC: ADR 0011 — interaction guarantee contract (path × guarantee matrix) (#1080)
* docs+feat: ADR 0011 interaction guarantee contract, Layer-1 registry and gate Design for making interaction guarantees hold across every dispatch path (runtime selector/ref, direct iOS selector, native ref, coordinate, maestro fallback) instead of eroding at path boundaries one incident at a time — every interaction bug this week was a (path, guarantee) cell nobody was watching. Three layers (ADR 0011): declare the path x guarantee matrix as a typed registry whose completeness is a compile error; share one implementation per rule on both sides of the wire with golden fixture tables proving TS/Swift parity; prove every non-waived cell with contract scenarios generated from the registry. This lands Layer 1: the registry with an HONEST initial classification — ten cells are acknowledged gap waivers (direct-path disambiguation/ occlusion/nonHittable/responseFields/errorTaxonomy, native-ref guards, coordinate bounds) — plus the gate test that keeps entries truthful: referenced TS symbols must be exported, runner symbols must exist in the Swift sources, delegations must land on paths that actually enforce the guarantee, and the gap list is pinned so it can only change explicitly in a reviewed diff. * refactor: apply ADR 0011 design review - Frame Layer 1 as an honesty/completeness gate, not a truth gate: it proves every path declared a stance and referenced symbols exist; behavioral parity starts with the Layer-2/3 fixture and scenario work. - Split responseFields into responseConstruction (one shared response construction site — a single Layer-2 refactor) and responseIdentity (which identity fields a path can provide — per-path capability work); note the anticipated errorTaxonomy split (codes vs diagnostics). - Encode the hybrid gap-closure strategy: runner-side parity for geometry-local rules, delegation-on-error for semantic failures (with the explicit caveat that delegation-on-error is NOT success-path parity), and a shared runtime preflight for native-ref where a silent backend success means delegation never triggers. - Gap waivers now require a trackingIssue (gate-enforced URL); all 16 pinned gaps link the umbrella issue #1081. The honest reclassification grew the pin list from 10 to 16 — responseConstruction is a gap on every path including runtime ones, which is exactly the partial progress the coarser guarantee was hiding. - Align ADR wording with the code: parityTable is optional until Layer 3, required once a runner cell claims parity. * fix: address registry review — maestro disambiguation honesty, command-scoped verify 1. maestro-non-hittable-fallback/disambiguation was overclaimed: the guarantee is defined as visible-first/deepest/smallest ranking, but findElement only implements unique-or-ambiguous scanning. Reclassified as an intentional waiver (deliberate Maestro-semantics divergence), mirroring how the direct path keeps its success-path parity gap. 2. verifyEvidence was claimed path-wide on paths that dispatch longpress, which has no --verify. Cells can now be command-scoped via appliesTo (non-empty strict subset of the path's commands, gate-enforced), and the three affected cells scope to press/click/fill. |
||
|
|
9aae457533 |
fix(errors): close call-site and consumer gaps around the central error system (#1071)
* fix(errors): close call-site and consumer gaps around the central error system Audit + iOS/Android dogfood findings (see docs/adr/0010-error-system.md): - press/click/fill targets that parse as neither @ref, selector, nor point now fail with INVALID_ARGS grammar guidance (incl. unquoted multi-word selector values) instead of UNKNOWN 'Expected x to be a finite number' - daemon command-input validation throws AppError INVALID_ARGS instead of bare Error surfacing as UNKNOWN - selector-no-match and stale-ref failures carry targeted hints (selectorFailureHint / STALE_REF_HINT) - retriable/supportedOn survive wire rehydration to CLI --json and SDK (previously dropped at throwDaemonError / toDaemonHttpRpcError) - MCP tool errors carry code + hint instead of message-only text - lease busy/capacity use DEVICE_IN_USE (the retriable code) - asAppError(err, fallbackCode) replaces cause-dropping coercions in the Apple runner; new default hints for AMBIGUOUS_MATCH, DEVICE_IN_USE, UNSUPPORTED_PLATFORM, and a distinct UNKNOWN hint - ADR 0010 documents the error-system conventions * fix: format touched files and clear fallow audit gate - privatize SELECTOR_NO_MATCH_HINT / SELECTOR_NOT_UNIQUE_HINT (consumed only via selectorFailureHint in the same module) and integerSchema (only used inside command-input.ts) - dedupe the resolved-node return tail in interaction resolution into describeResolvedNode - extract stringDetail/booleanDetail readers so normalizeError stays under the complexity threshold * fix: reject unquoted trailing text after interaction selectors press/click/longpress positionals like 'press text=Gesture lab' used to silently drop the leftover tokens and act on the truncated selector (text=Gesture), potentially hitting the wrong element. Reject non-empty splitSelectorFromArgs rest with INVALID_ARGS guidance that suggests the merged quoted form (text="Gesture lab"). Fill keeps consuming rest as its text payload; wait/is/replay-heal already handle rest explicitly. |
||
|
|
b6128c0088 |
docs: retire plans/perfect-shape.md — roadmap complete (#1003)
* docs: retire plans/perfect-shape.md — roadmap complete The perfect-shape roadmap (two-registry thesis: CommandDescriptor + PlatformPlugin, typed-result spine, folder DAG + layering lint, agent-cost, and the Apple apple+appleOs platform model with a non-breaking leaf wire) is substantively complete and merged. Per its own §5 retirement note, the durable decisions now live in ADR-0008 (command descriptor) and ADR-0009 (Apple/AppleOS), and current-state terms in CONTEXT.md; this removes the last plan file. - Delete plans/perfect-shape.md (plans/ is now empty and gone). - CONTEXT.md: add "Architecture (perfect-shape refactor, completed 2026-07)" end-state summary plus a "Deferred / next-minor" note (Phase 2c client-types narrowing, b.3 recording/providers facets, strict DAG back-edge inversion, legacy alias drops) so nothing is lost. - Repoint every remaining perfect-shape.md/§ reference (ADRs 0003/0008/0009, ci.yml, scripts/layering/check.ts, and the platform-plugin/apple comments) to ADR-0008/0009 or CONTEXT.md. No dangling references remain. Docs/comment-only; tsc, oxlint, oxfmt, and the layering DAG check all pass. * docs: repoint dangling perfect-shape section refs before retiring the roadmap Removing plans/perfect-shape.md left three comments citing bare section numbers with no surviving target. The rationales are already inlined, so drop the numbers (and point the do-not-flatten note at the durable ADR): - src/platforms/apple/plugin.ts: `(§7)` -> "do-not-flatten; see docs/adr/0009". - src/core/interactors/register-builtins.ts: "the §5.1 ... sketch" -> "an ... sketch". - scripts/layering/check.ts: drop `(§5.5 ...)`, keep the inline "re-export barrels only". |
||
|
|
db0e084c30 |
docs: retire plans/phase3-platform-plugin-progress.md; track remaining work in issues (#982)
The remaining Phase 3 Apple PlatformPlugin work (steps b + d) is now filed as GitHub issues under umbrella #972, so the standalone progress plan is redundant and a staleness hazard (it already drifted once re: cost.runnerRoundTrips). - Remove plans/phase3-platform-plugin-progress.md. - Repoint its references at the durable sources: perfect-shape.md (x3) and ADR-0009 now link the Phase 3 tracking issue #972; the plugin.ts step-b facet note points at ADR-0009 (+ issue #974). Design rationale stays in perfect-shape.md and ADR-0009; live status lives in the issues. |
||
|
|
26ac865c63 | refactor: consolidate Apple platform internals (#968) | ||
|
|
7a1640e53f |
refactor: move errors/redaction/device into src/kernel — Phase 5 slice 3 (#940)
* refactor: move errors/redaction/device into src/kernel — Phase 5 slice 3
Relocates the foundational primitive trio from src/utils/ into the kernel/ layer
(joining snapshot.ts from slice 2), per the target folder DAG in
plans/perfect-shape.md §5.5. A pure path codemod, no behavior change.
They form a closed cluster — device -> errors -> redaction, with redaction a
leaf — so kernel/ takes no upward dependency, and every importer becomes a clean
downward import toward kernel. errors.ts is the most-imported module in the
tree; device.ts the §5.5-named headliner. Moving all three atomically avoids a
half-state where one would import another across the utils/kernel boundary.
Imports rewritten by a resolve-based codemod (compares each specifier's resolved
path to the moved files, so the unrelated commands/management/device.ts and
other same-named files are untouched): 483 sites across 402 files. The two
platform-descriptor doc comments and the fallow health baseline key for
device.ts are updated to the new path; the contracts-schema-public guard that
asserts the error helpers pull no diagnostics/node: deps now reads kernel/.
Verified: tsc --noEmit, oxfmt + oxlint --deny-warnings, rslib build, full vitest
suite (2877 pass), fallow audit clean (411 changed files), Layering Guard empty;
kernel/ files import only within kernel.
* docs: update guidance references to kernel/{device,errors} after the move
AGENTS.md (Apple-family sync rule + normalizeError), ADR-0009, and
plans/apple-platform-consolidation.md still named the old src/utils/ paths.
Point them at src/kernel/. plans/perfect-shape.md's utils/device.ts mention is
left as-is — it describes the pre-move diagnosis.
|
||
|
|
29e19b8e3f |
docs: add ADR 0008 (command descriptor) + ADR 0009 (Apple consolidation) (#905)
Locks the two axis decisions and starts retiring plans/ into ADRs. ADR 0008 (Proposed) records the command-descriptor registry composing domain-owned facets and deriving the ~10 tables, bound by ADR 0003's four invariants; ADR 0009 (Accepted, groundwork shipped in #896) records the AppleOS leaf axis under one 'apple' Platform. perfect-shape.md links both and marks Phase 0 + Tier-A dedup as merged. |
||
|
|
93d5275e69 |
refactor: type-safe recording backends + exhaustive capability gating (#894)
* docs: add perfect-shape architecture roadmap
Captures the target architecture (two-registry thesis: CommandDescriptor +
PlatformPlugin over a clean folder DAG with a typed-result spine) and a sequenced,
strangler-fig migration path, grounded in a survey of the current codebase.
This PR implements the first two behaviorless Phase-0 items from that roadmap; the
larger registry work is deliberately deferred to later, independently shippable PRs.
* refactor: parametrize RecordingBackend by recording tag
RecordingBackend is now generic over the recording's platform tag, so each
backend's stop() receives an already-narrowed recording. This deletes all five
'recording as Extract<ActiveRecording, { platform: ... }>' casts — the textbook
discriminated-union-narrowing-by-cast anti-pattern — and makes a backend/tag
mismatch unrepresentable.
start() stays wide (DaemonResponse | ActiveRecording) because a device platform
does not map 1:1 to a recording tag (an iOS device resolves to either the 'ios' or
'ios-device-runner' recording). Device resolution returns a stop-less view
(RecordingStartBackend); stop is dispatched per active recording via the new
exhaustive stopActiveRecording(), replacing resolveRecordingBackendForRecording().
Behaviorless: pure type-level change, no runtime behavior change.
* refactor: make capability platform selection exhaustive
isCommandSupportedOnDevice resolved the per-platform capability bucket with an
if/else ladder whose final branch funneled every unmatched platform into
capability.web. That silently absorbs a future Platform with no compile error.
Replace it with selectCapabilityForPlatform(), an exhaustive switch over the
Platform union with a 'never' guard, so adding a new platform is a compile error
here instead of a silent web mis-gate. Identical behavior for all five current
platforms (ios/macos -> apple, android, linux, web).
* docs(adr): amend ADR 0003 for the single-declaration/derivation model
Ratifies the PR review caveat into the ADR itself: the daemon command registry
boundary is about ownership + the predicate interface, not the physical file a trait
is typed in. A derived/projected daemon registry is permitted only if it preserves
four invariants (daemon-owned declaration, unchanged predicate interface, no leakage
into public projections, one declaration per concern enforced by types). The original
decision stands; collapsing daemon policy into a public command registry remains
forbidden.
* docs: refine command axis to facet composition (ADR 0003-aligned)
- §2/§5.2: CommandDescriptor composes domain-owned facets (surface@commands,
capability@core, daemon@src/daemon) and projects them — compose-with, not
collapse-into. Adds the four ADR-0003 invariants.
- §6: mark the two shipped Phase-0 items (generic RecordingBackend<P>, exhaustive
capability selection); link the Apple plan from Phase 3.
- §5.1: Apple as the first PlatformPlugin instance, owning an AppleOS leaf axis.
- §8: before/after diagrams for the command axis + the two-axis summary.
* docs: add apple-platform-consolidation plan (AppleOS leaf axis)
One 'apple' Platform with an AppleOS discriminant (ios/ipados/tvos/watchos/
visionos/macos) rather than six Platform literals (which would collide with the
cross-platform 'target' axis). Captures the 4-investigator survey: ~85% of
platforms/ios is already the OS-agnostic Apple engine; the XCTest runner already
builds ios|macos|tvos; macOS is included as a distinct AppKit leaf (already
entangled). visionOS is scoped net-new work; watchOS is an unsupported sentinel
(XCUITest can't drive it). Before/after diagrams, per-OS readiness, sequencing.
|
||
|
|
a822325375 |
feat: add integrated device leasing (#890)
* feat: add integrated device leasing * fix: keep metro bearer token out of generated proxy profile The proxy connect profile is written to disk as a non-secret remote config, but it unconditionally copied `metroBearerToken` into that file, leaking the secret at rest. Mirror the cloud path, which keeps `daemonAuthToken` in-memory only: the token still flows through this connect via the returned flags, and later commands re-supply it via AGENT_DEVICE_METRO_BEARER_TOKEN. Extend the non-secret-profile test to assert the bearer token is absent from disk. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPa5Z9GBkeqoxVctC85N7e * fix: always release device lease on session close releaseSessionLease + sessionStore.delete ran only on the happy path, after several awaits (app-log/perf/snapshot teardown, platform close dispatch, runner stop) that can throw. A failed close therefore stranded the device lease until the inactivity expiry. Wrap teardown in try/finally so ownership is always freed; the original error still propagates after finally. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPa5Z9GBkeqoxVctC85N7e * fix: reconcile integrated device leasing * docs: simplify remote lease guidance * refactor: satisfy leasing fallow checks * fix: harden integrated device leasing * refactor: deepen device lease lifecycle * refactor: centralize lease scope projection * fix: harden proxy lease e2e flow * fix: address lease review feedback * refactor: tighten lease release cleanup * fix: simplify proxy startup output * fix: harden cloud lease identity * fix: color proxy startup output * fix: simplify proxy tunnel placeholder --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
d47cd30117 | feat: add agent-device proxy command (#844) | ||
|
|
5c83fe4e50 |
fix: improve daemon diagnostics and remove compact snapshots (#786)
* fix: improve daemon startup diagnostics * perf: relax query-sweep recovery budget * fix: make compact snapshot flag a no-op * fix: remove compact snapshot mode * test: align tvOS remote provider expectation * test: drop compact snapshot progress target |
||
|
|
fa1c1d55da |
refactor: localize command surface modules (#772)
* refactor: localize command surface modules * refactor: localize command runtime modules * refactor: tighten localized command exports * refactor: address command localization review * refactor: deepen batch command policy * refactor: split provider progress model * refactor: collapse command client facets * refactor: run progress metrics as TypeScript * refactor: remove obsolete command shims * fix: update localized snapshot output import * fix: preserve debug command localization |
||
|
|
fa8cce37d1 |
refactor(ios): snapshot capture plans with a structured quality verdict (#783)
* refactor(ios): snapshot capture plans with a structured quality verdict Implements ADR 0004's explicit-strategies decision as architecture (candidates 1+2 of the snapshot pipeline review): - Snapshot backend seam: three adapters (recursive tree, query sweep, private AX) behind one captureWithBackend dispatch. Each strategy declares its chain as data (regular: tree→queries→private-ax, compact: queries→private-ax, raw: tree→private-ax) and one plan runner walks it under a 20s umbrella budget so chained recovery can never stack past the 30s main-thread watchdog. Terminal policy is per-plan: raw rethrows AX failures (diagnostics preserve errors), interactive fails closed with runnerFatal invalidation. - Single quality classifier: one sparsePayloadReason predicate (with reason codes), one collapsed-leaf detector, replacing the three divergent sparse detectors (Swift structural, daemon count==1, CLI count<=3) that each patched a different failure shape. - Structured snapshot quality verdict on the wire (state, backend, reason, reasonCode, effectiveDepth, collapsedLeafIndexes): the daemon and CLI render warnings from it instead of re-deriving degradation from node shapes; budget starvation is no longer blamed on the app's accessibility. Legacy runner messages and daemon-side detectors stay behind a verdict-absent gate for mixed-version compat. - The verdict surfaces in --json (snapshotQuality) for agents; the generic sparse CLI hint is suppressed when a verdict explains it. Threading the verdict exposed two more hand-copy field drops (captureInteractionOutcomeAwareSnapshot, serializeSnapshotResult, client response mapping) - now carried alongside warnings everywhere. Verified live: Settings healthy (tree, no warnings), Settings compact under load (recovered/private-ax/budget), production login (sparse best-effort with honest warning), collapse fixture (healthy + collapsedLeafIndexes -> @ref warning), Bluesky Home (recovered/ private-ax, 24 nodes in 2s). Full unit suite 2327 passed, fallow clean, runner builds. * fix(ios): correct recovered-snapshot viewport and private-AX scope semantics Review follow-ups on the capture-plan refactor: - The query-sweep synthetic root doubles as the daemon's viewport (find.ts prefers on-screen matches inside nodes[0].rect), but it was built from candidate bounds, so off-screen controls below the screen could inflate it and win duplicate-label resolution. The root now uses the real finite viewport, falling back to candidate bounds only when viewport capture failed. - The private-AX backend applied --scope as a per-node text filter, hiding the matched container's children — diverging from regular snapshot scope semantics and contradicting the depth-cap hint that recommends scoped re-runs. Scope now selects the matched subtree: descendants inherit the match and only the normal option filters apply to them (in-bundle test covers a non-matching descendant). Verified live on Bluesky Home: scope homeScreen returns the 52-node subtree including non-matching descendants; compact root rect equals the screen (0,0,402,874). * fix(ios): fail closed on interactive AX failure, stamp fatal verdict, validate parser Three review findings on the capture-plan terminal path: - P1: the fail-closed guard required `best == nil`, but the query-sweep tier always returns a synthetic-root sparse payload that sets `best` — so an interactive recursive-tree AX serialization failure that no backend recovered returned a sparse snapshot instead of invalidating the cached target. Reaching the terminal already means no backend produced a usable tree, so the sparse `best` must not suppress the fail-closed path. Extracted the decision into a pure, unit-tested `resolveSnapshotPlanTerminal` (closes the terminal-ordering testability gap the architecture review flagged). - P2: `snapshotAccessibilityUnavailable` returned a payload with no `snapshotQuality`, leaving one planned sparse result on the legacy-message path. It now carries a sparse/ax-rejected verdict like every other planned snapshot, so downstream sparse handling keys off the verdict. - P2: `readSnapshotQualityVerdict` cast any string state/backend into the union, so a malformed object suppressed the legacy node-shape detectors. State and backend are now validated against their unions (unknown → verdict-absent → legacy detectors run); an unknown reasonCode is dropped rather than rejecting the whole verdict, so a forward-version runner still yields a usable verdict. Unit-covered: Swift resolveSnapshotPlanTerminal matrix + fatal-verdict assertion; TS parser accept/reject/forward-compat. Full suite 249 files / 2449 tests, fallow, lint, runner build green. |
||
|
|
3a02e514e1 |
refactor(ios): consolidate series batching onto the sequence runner command (#768)
* refactor(ios): consolidate series batching onto the sequence runner command Closes #767 Routes every Apple multi-press variant (plain, double-tap, hold, jitter) and swipe series through budget-chunked sequence requests, retiring the daemon-side tapSeries and dragSeries senders: - Add a doubleTap step kind to the sequence allowlist on both ends, mirroring the retired tapSeries doubleTapAt branch. - The single doubleTap interactor sends a one-step sequence and parses the result, surfacing step failures as errors. - Swipe series unroll ping-pong daemon-side into per-step endpoints; the runner's coordinate-drag path ignores durationMs exactly as the daemon-sent (non-synthesized) dragSeries did. - Extract runIosSequenceChunks so press and swipe share the chunking, aggregation, and global step-index rebasing. - Keep tapSeries/dragSeries runner handlers for wire compatibility with older daemons, annotated like interactionFrame; remove both from the preflight-skip allowlist (daemon never sends them) and update ADR 0005 / protocol-optimizations docs. This also closes the latent watchdog exposure where press --count N --interval-ms M routed to tapSeries and executed all pauses inside one 30s-watchdog main-thread block with no chunking. Behavior note: plain tap series now use the synthesized HID tap path on iOS non-tv (with runner-side tapAt fallback), matching the individual tap command instead of the retired tapSeries' XCUICoordinate taps. https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo * refactor(ios): drop dead series wire surface from the daemon - Remove chunkRunnerSequenceSteps: superseded by the budget-aware chunker; no production callers remained. - Remove tapSeries/dragSeries from the RunnerCommand union along with their orphaned fields (count, intervalMs, doubleTap, pauseMs, pattern) and protocol fixtures: this type is the send surface of the current daemon, which no longer sends either command. The Swift runner keeps serving both for wire compatibility with older daemons. - Retarget the ready-mutation preflight test from tapSeries to sequence. https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo * refactor(ios): remove retired series and frame wire commands entirely Drops the runner-side wire compatibility for tapSeries, dragSeries, and interactionFrame now that no daemon path sends them (series fuse into sequence since this branch; interactionFrame was fused into scroll in #760): - Swift: delete the three handler cases, performDragSeries, runSeries (no remaining callers), the CommandType enum cases, journal-retention and traits entries, and the Command fields (count, intervalMs, doubleTap, pauseMs, pattern) that existed only for them. The never-sent synthesized dragSeries branch goes with it. - TS: drop interactionFrame from the RunnerCommand union and isReadOnlyRunnerCommand, and its protocol fixture. - Update stale perf scenario labels referencing the retired commands. Verified dead before removal: no dynamic command construction anywhere (runner-command-recovery only echoes in-flight command ids), no raw-string references in Swift, no docs references. Helpers shared with live paths (synthesizedDragAt, doubleTapAt, keyboardAvoidingDragPoints, sleepFor) all retain callers. Compat: an old daemon paired with a runner built from these sources gets a CommandType decode rejection; the source-fingerprint check rebuilds a matching runner on the next session. https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
e6e2baf30f |
perf(ios): add lifecycle-safe runner sequence command for hot press series (#764)
* perf(ios): add lifecycle-safe runner sequence command for hot press series Adds a narrow 'sequence' runner command that batches an explicit allowlist of coordinate steps (tap, longPress, drag) into one lifecycle-tracked request with stop-on-first-failure and small bounded per-step results. iOS press series with hold/jitter now issue one sequence request per ~20-step chunk (also budgeted to stay under the runner's 30s main-thread watchdog) instead of one request per press. Sequence responses are journaled and retained, so lost-response recovery returns observed results without replaying the gesture sequence. Closes #669 * fix: perform every press in direct press series runDirectPressSeries guarded the awaited interaction itself with ??=, so presses 2..N were silently skipped once the first result was kept (affects Android series and doubleTap series; introduced in #512). The kept-first-result shape is preserved. * chore: unexport internal sequence chunk budget constant * perf(ios): make sequence eligible for readiness preflight skip Rebased onto main with #763 (healthy-mutation preflight skip) and #760 (fused scroll). Per the merge-order note, add 'sequence' to PREFLIGHT_SKIP_ELIGIBLE_RUNNER_COMMANDS so a successful sequence earns the next hot-command skip instead of always taking the conservative_command path. Extend the per-family skip tests and the allowlist enumeration in ADR 0005 and the protocol-optimizations doc. https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
3780ca6ed5 |
perf(ios): fuse scroll frame resolution and drag into one runner command (#760)
* perf(ios): fuse scroll frame resolution and drag into one runner command Non-tvOS scroll now sends a single mutating 'scroll' runner command. The Swift runner resolves the interaction frame and executes the same non-synthesized drag path, eliminating the separate read-only interactionFrame request per scroll. The command is lifecycle-journaled with retained response JSON so lost-response recovery returns the result without replaying the gesture. Closes #668 * perf(ios): make fused scroll eligible for readiness preflight skip #763 landed the healthy-mutation preflight skip with a note that the fused scroll command should join the allowlist once it exists. Add 'scroll' to PREFLIGHT_SKIP_ELIGIBLE_RUNNER_COMMANDS, drop the now-resolved code note, extend the per-family skip tests, and update the allowlist enumeration in ADR 0005 and the protocol-optimizations doc. https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo * test: complete scroll plan parity vector mirror Address review on the cross-language parity vectors: - mirror the Swift pixels-plan vector (down, 120px @ 300x600) in the vitest suite so every vector exists in both languages - add amount > 1 clamp and tiny-frame (2x2) vectors to both suites; the tiny frame engages every max(1, ...) floor and the .5 rounding cases where JS half-up and Swift half-away-from-zero must agree https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
462db525cc |
perf(ios): skip runner uptime preflight after recent healthy mutations (#763)
Reintroduces the #662 adaptive readiness-preflight skip with guardrails for the #702 failure modes. Recency is recorded only from healthy (non-runnerFatal) responses to an explicit mutating-interaction allowlist (tap, tapSeries, longPress, drag, dragSeries, swipe), scoped to the same appBundleId, capped at a 5s freshness window, and lives on the session object so it dies with every invalidation. Startup, no-recent-success, stale, app-switch, and non-allowlisted commands still preflight. A transport failure after a skip clears recency, carries the skip context through status recovery, and never routes into restart-and-replay. Closes #667 |
||
|
|
b2e4ace12c |
fix: focus booted iOS simulators with Device Hub (#750)
* fix: focus booted iOS simulators with Xcode Device Hub * fix: expose Device Hub opt-out * test: cover Device Hub screenshot retry focus --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> |
||
|
|
35f54a863c |
fix: resolve Maestro taps from regular snapshots (#709)
* fix: resolve Maestro taps from regular snapshots * fix: tighten Maestro snapshot compatibility * test: align Maestro snapshot freshness fixtures |
||
|
|
76cee982ba |
fix: stabilize iOS runner navigation taps (#702)
* 0.16.14 * fix: stabilize iOS runner navigation taps * fix: recover iOS runner after AX failures * docs: clarify AX-unavailable snapshot recovery * test: cover synthesized ios provider taps * test: cover iOS runner AX failure paths * fix: clarify sparse iOS AX recovery hint |
||
|
|
aa5b07faa5 |
fix: keep iOS runner hot across app closes (#700)
* fix: keep iOS runner hot across app closes * fix: keep iOS snapshots fast after relaunch * fix: bound compact iOS snapshots on broken AX trees * docs: clarify iOS snapshot backend strategy * test: update Settings replay selectors * fix: stabilize Settings replay selectors * fix: fall back for selector interactions * chore: simplify flat interactive snapshot query |
||
|
|
f2424f9d3a | refactor: centralize daemon command registry (#693) | ||
|
|
d087a62d1d |
fix: improve Maestro Android reliability and snapshot speed (#612)
* fix: improve maestro tab target swipes * fix: stop replay suites on pending timeout cleanup * fix: tighten maestro compat support plumbing * fix: resolve ios simulator url open targets * fix: speed up Android Maestro snapshots * fix: resolve android snapshot ci regressions * perf: reduce android snapshot helper overhead * docs: record persistent helper session architecture * chore: address snapshot session review * perf: add persistent Android snapshot helper session * fix: stabilize Android replay interactions * fix: stabilize Maestro visibility resolution * refactor: simplify Android keyboard parsing |
||
|
|
59d28e8446 |
refactor: add provider-first device lab tests (#542)
* refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor |