mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
main
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Michał Pierzchała
|
c93dcdbc90 |
feat: disclose selector resolution in interaction responses (#1193)
* feat: disclose selector resolution in interaction responses Implements ADR-0012 migration step 1 (decision 2). Adds an additive `resolution` field to press/click/fill/longpress responses: runtime-selector carries the full pre-action diagnostic shape (unique or disambiguated with matchCount/winnerDiagnostic/tiebreak/bounded alternatives), runtime-ref and native-ref carry the exact ref-provenance shape, direct-ios-selector carries the explicit not-observed marker, and coordinate/maestro-non-hittable-fallback stay inapplicable (no field). The comparator in selectors-resolve.ts now records which criterion (visible/deepest/smallest-area) decided each disambiguation without changing resolveSelectorChain's winner. Extends the ADR-0011 guarantee matrix with the resolutionDisclosure guarantee across all six dispatch paths, wires the shared response builder and MCP output schema, adds digest-level trimming (drops alternatives, keeps the verdict/counts), and proves via contract tests that resolution diagnostics are never ref-issued or MCP-pinned and cannot be reused as @ref targets. * fix: address resolution disclosure review findings * refactor: make resolution-disclosure choices self-evident Replace the direct-iOS/maestro message-sniffing (and its justification paragraph) with an explicit maestroFallback flag passed from the dispatch site that already owns the path decision, and shrink every why-this-is-OK paragraph to one-line constraint statements per the maintainer directive. * fix: usage-based maestro fallback disclosure + spec label-fallback Blocker 1: the runner-payload source now carries maestroFallbackUsed derived from the runner's actual execution outcome (the usedNonHittableFallback message bit RunnerTests+CommandExecution.swift reports, the same signal directIosSelectorFallbackDetails already keys on) instead of the permission flag. A fallback-allowed dispatch that hit its element normally discloses direct-ios/not-observed; only an actually-executed coordinate fallback is the inapplicable maestro cell. Contract tests cover both sides. Blocker 2: ADR-0012 decision 2 now defines the ref/label-fallback disclosure (runtime-ref trailing-label recovery via tryResolveRefNode's fallbackLabel; native-ref stays exact because the backend receives only the ref handle), amends the matrix-cell enumeration, layer-3 coverage list, and validation bullet, and the runtime-ref contract suite proves the label-fallback shape. * fix: honest runtime-ref registry cells for label recovery The disambiguation cell no longer claims refs identify exactly one node by construction — trailing-label recovery is a first-match lookup without the ranking, now an intentional waiver whose outcome the label-fallback disclosure surfaces per-response. resolutionDisclosure.via points at tryResolveRefNode (now exported), the resolver producing both exact and label-fallback, with direct unit coverage of both outcomes. * docs: correct native-ref exactness rationale and tiebreak doc Native-ref forwards fallbackLabel to the backend; exact is justified by non-observability of any backend-side label recovery, not by non-forwarding. The tiebreak doc now states the derived winner-vs-runner-up decisive margin. * fix: disclose Maestro fill fallback usage |
||
|
Michał Pierzchała
|
5c5fa012f7 |
feat: --settle returns the settled diff in the interaction response (#1101) (#1106)
* feat: --settle returns the settled diff in the interaction response (#1101) press/click/fill/longpress --settle executes the action, waits for the UI to go quiet (wait stable's loop, shared via stable-capture.ts), and returns the settled diff vs the pre-action tree in the same response — one round trip instead of the interact -> observe pair. - payload: changed lines only (bounded), summary counts, added-line refs, refsGeneration; best-effort (settled:false + hint on never-quiet content, never an action failure); --verify shares the settle captures - ref issuance: the settled tree becomes the session snapshot; a diff-carrying settle response clears snapshotRefsStale and the MCP layer merge-only re-pins added-line refs at the settle generation - grammar: --settle + --settle-quiet <ms> + --timeout <ms> (flag-sourced descriptor budget with new envelope:'widen' semantics mirroring wait) - ADR 0011: new settleObservation guarantee classified on every path with contract scenarios per enforced/delegated cell * test: give the two contention-flaky doctor scenarios explicit budgets The doctor provider scenarios sit at ~5s of real daemon-harness work on a loaded host and flake at vitest's 5s default during full-suite runs (the known contention flake AGENTS.md documents). Same in-file precedent as the Metro-probe scenario's 10s budget. * fix: move SettleParams to contracts to satisfy the layering DAG daemon/handlers/interaction-flags.ts imported the type across the daemon -> commands boundary (R2 commands-floor). The tuning params are part of the interaction contract like SettleObservation, so they live in contracts/interaction.ts and both layers import from there. * feat: keep settle diffs content-first — drop Key nodes, added lines win the cap Bluesky dogfood: a fill that summons the iOS keyboard spent 49 of the 80 capped diff lines spelling out QWERTY keys, and a screen transition with 269 removals could starve out the added lines entirely. Key-type nodes are now filtered from both diff sides (the [keyboard] container line still signals presence), and under truncation added lines — the ones carrying fresh refs — win slots over removals. * docs: state the core loop in the top-level help starting point Benchmarked with headless haiku/sonnet agents given only --help: both models skipped the help-workflow pointer and started with plain snapshot (38KB payloads they then had to re-read from files). One core-loop line at the starting point is what teaches snapshot -i and --settle to models that never read a second help page. * fix: preserve settle digest refs for mcp * fix: reduce settle fallow complexity * fix: surface settle output in CLI text * fix: complete settle handling for longpress * refactor: localize daemon timeout envelopes * refactor: deepen post-action observation * refactor: centralize post-action observation planning * refactor: derive settle capability from descriptors * refactor: trim settle descriptor helpers |
||
|
Michał Pierzchała
|
f3e07ff236 |
test: interaction contract suite with registry-driven coverage gate (ADR 0011 Layer 3) (#1092)
* fix: preserve the runner's non-hittable fallback marker through direct selector press
The direct iOS selector press handler spread successText('Tapped <selector>')
after the runner payload, clobbering the 'tapped via non-hittable coordinate
fallback' message that directIosSelectorFallbackDetails keys on — so
maestroNonHittableCoordinateFallbackUsed could never be true end-to-end (the
existing unit test passes because it mocks dispatchCommand above this layer).
Found by the ADR 0011 Layer-3 maestro-fallback contract scenario.
Share the marker string as MAESTRO_NON_HITTABLE_FALLBACK_MESSAGE and keep it
as the success message when the runner reports fallback usage.
* test: interaction contract suite with registry-driven coverage gate (ADR 0011 Layer 3)
test/integration/interaction-contract/ holds one scenario file per dispatch
path, each with a sibling .coverage.ts manifest declaring which guarantee
matrix cells it proves (scenario strings double as the vitest test titles).
index.ts aggregates the manifests statically, and a new Layer-3 gate in
src/contracts/__tests__/interaction-contract-coverage.test.ts fails when any
enforced (runtime/runner/delegated) cell lacks a scenario or when a scenario
claims a waived/inapplicable cell — coverage of the matrix is by
construction in both directions.
Path forcing is natural (no test-only env switch needed): selector/ref
targets take the runtime path, a tapTarget backend takes the native-ref fast
path, x/y takes the coordinate path, and simple-selector clicks on an iOS
provider transcript take the direct runner path (the transcript itself
proves which path ran via assertComplete).
Fixtures are the permanent Bluesky shapes: closed drawer, drawer + visible
twin, edge-grazing container, covered button, non-hittable cell.
Refs #1081
|