* refactor(capture-kit): relocate snapshot and recording zones into capture-kit
Move the ADR 0019 end-state capture zones into @agent-device/capture-kit:
- src/snapshot/** -> packages/capture-kit/src/snapshot/** (presentation,
freshness, scroll-edge-state, ios-snapshot-runtime, android occlusion)
- src/recording/** -> packages/capture-kit/src/recording/**
- src/core/snapshot-{chrome,state,tree-ingestion,node-lookup}.ts ->
packages/capture-kit/src/
- src/snapshot-quality/ test -> capture-kit presentation tree (directory
retires with its last file)
Pure renames: import re-pointing and gate updates follow in the next commit.
The snapshot-desktop-surface test parks in src/__tests__/ because it pins
the root eager-import-closure walker.
* refactor(capture-kit): re-point capture and recording consumers to the new subpaths
Rewires every consumer of the relocated snapshot/recording modules to the new @agent-device/capture-kit subpath exports, adds the 23 subpath entries to the capture-kit exports map, fixes the moved recording-scripts test's __dirname-relative paths for the deeper location, and records the completed migration in ADR 0019's end state.
* chore(gates): align layering, mutation, fallow and CI gates with the capture-kit relocation
Moves the executable-policy roots, presentation-owner constant, zone ranks, authority fixture, mutation sharding globs, stryker aliases, fallow baselines and the iOS workflow's android-owned paths-ignore entry onto the new packages/capture-kit paths, and extends the planted-red coverage to the new presentation-owner subpath.
* chore: point capture-domain source-of-truth comments at the relocated capture-kit modules
* test: point shutdown recording mock at capture-kit and cover interactor acquisition presentation
* test(capture-kit): update upstream presentation test imports
* chore(gates): follow relocated snapshot assembly in R74
* test(daemon): freeze prewarm deadline assertion clocks
* refactor: sink package-closed src modules into existing packages
Move closed modules into contracts, kernel, capture-kit, and ad-script,
and declare DaemonCommandDescriptor in core so R6/R9 can pin the remaining
provider-webdriver type cycle.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor: keep contracts and capture-kit off generic sinks
Move interaction-outcome, snapshot warning rendering, and inventory ALS
behind focused owners, and plant R18/R70 domain-shape gates so they
cannot return as package export-map growth.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor: drop moved implementation comments from owner modules
Names, types, and tests already carry those invariants; the relocated
files should not keep review-history or control-flow narration.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor: drop the empty snapshot-quality layering zone
W1 moved the verdict into capture-kit and this PR moved warning rendering
into snapshot-presentation, so the ranked zone no longer has production files.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(cli): advertise open --foreground and snapshot --actions in the workflow card
open --foreground (#1670/#1671) and snapshot -i --actions (#1665) shipped
with no mention in the compact `help workflow` card, so a planning model
never discovers either. Add one terse line each: the foreground fast-path
in Bootstrap, and the merged-element custom-action guidance in Validation
and evidence. Stays under the 9,000-byte compact-card budget (8493 -> 8908
bytes).
Adds two help-conformance bench cases per the repo's changed-guidance rule:
foreground-attach-single-sim (correct plan starts with `open --foreground`
in an unambiguous single-sim scenario, fail-closed alternative forbidden)
and merged-card-actions-not-directly-invokable (a merged Bluesky-style feed
card's actions list is evidence, not a selector). Both use a real pinned
sample rebuilt through the production snapshot renderer.
* fix(scripts): accept flag order in the foreground-attach conformance matcher
Flag order after `open` isn't semantically meaningful (`open --platform ios
--foreground` is exactly as correct as `open --foreground --platform ios`),
but startsWithForegroundOpen required --foreground to be the literal next
token after `open`. Rescoring the completed repeat=3 bench report shows this
docked codex:gpt-5.4-mini on all 3 trials even though its plan was
config-order noise, not a real deviation -- the no-positional/no-device
guarantee already comes from the forbidden checks. Loosened to require
--foreground anywhere on the open line; foreground-attach-single-sim now
scores 54/54 across both runners.
* fix: close workflow help conformance gaps
* fix: isolate concurrent remote connections
* refactor: harden remote connection state
* fix(cli): scope every emitted connect command to its own session
`scopeNextSteps` only reached `ConnectReadiness.nextSteps`, so two
command-bearing outputs still shipped unscoped:
- `providerArtifactNotes()` emitted `agent-device artifacts --json` as a
prose note, which never passes through that helper.
- `buildDeferredRuntimeNotice()` emitted `agent-device metro prepare
--remote-config <path>` independently in connection.ts.
On the shared-host concurrency path this branch fixes, following either
one resolves against the host-global active connection, so the artifacts
instruction can return another job's provider video and log URLs (#1659).
Both producers now take the connection state and format through one
exported `scopeCommand` helper, which is the single place a suggested
command is bound to its originating session. The metro config path is
shell-quoted alongside the session name.
Coverage: the BrowserStack route asserts human and JSON shapes carry one
`--session` per suggested command and that each emitted session resolves
back through `readRemoteConnectionState` to the connection that printed
it; `connection status` pins the scoped deferred-metro `nextStep`.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: support --settle on scroll and back (#1638)
Scroll-then-observe and back-then-observe are legitimate agent pairs, but
the post-action observation registry never grew past the touch commands, so
`--settle` on either was rejected with INVALID_ARGS — burning a tool call
each in AppControlBench's bsky-16.
Both commands now carry the `settle` descriptor trait, and every surface
derives from it rather than a hand list: CLI allowed flags, MCP/SDK input
fields, the flag-sourced timeout envelope, and MCP ref-pinning. The CLI
flag/metadata helpers moved out of the interaction family into
post-action-observation-grammar.ts (back is a system command), and
SETTLE_REF_ISSUING_TOOLS became a derivation — a hand list would have
silently stopped pinning the new commands' refs.
settleAfterInteraction and the new settleObservationCommand are two entry
points over one engine: same loop, storage, hints, and diff bounds, with the
target-less path supplying its own baseline and no proximity point. The
daemon reaches that command through the runtime surface, never by importing
`commands/` (R2) — the same seam the touch handlers use for press/fill —
and generic-settle.ts is loaded through a lazy `await import` returning a
closure, so the interaction runtime subgraph stays out of this dispatcher's
static graph (a static edge folded ~18 files into the daemon-server type
cycle; R10 caught it).
Both of generic-settle's orderings are load-bearing and tested: the baseline
is frozen before dispatch (and before the Android dialog preflight), and the
observation runs after markDeferredInteractionOutcome so settle's first
capture folds in the #1542 stabilization rather than racing it. The ADR 0014
"a settled diff publishes refs" rule moved to settle-ref-issuance.ts, shared
by both routes.
One divergence is deliberate: scroll/back resolve no element, so the diff
baseline is the session's STORED pre-action tree — "settled tree vs the
last tree you observed" — not press's freshly resolved pre-action capture.
Both commands also switch to preserve-daemon on timeout, which changes the
non-settle path too: with --settle their dominant hang mode is now a wedged
accessibility bridge, and a timed-out capture must not reset the daemon and
lose every session (#1105). The reviewed-set gate records it.
Live-validated on an iOS 26.2 simulator (Settings): scroll --settle settled
in 1786ms with a +6/-6 diff carrying fresh refs; back --settle in 771ms with
+15/-6. Alternating cost runs, one call vs the pair it replaces:
scroll 2.9-3.0s vs 5.3-5.6s, back 3.1-3.2s vs 4.7-5.1s. Those include the
#1627 deep-capture extension.
* fix: render settled-diff refs paste-ready in CLI output
A settled diff activates a PARTIAL ref frame (ADR 0014), which admits only
the pinned `@eN~s<gen>` form of the refs it issued. The unchanged-interactive
tail already rendered that way, but the diff's own added lines rendered the
bare `@eN` embedded in the snapshot line — so a CLI caller who copied the
ref the diff just handed them got `plain_ref_requires_complete_frame` and had
to append the generation by hand.
Added lines now render pinned when the response carries `refsGeneration`,
exactly like the tail. Removed lines render verbatim: they name elements that
just left the screen, and `SettleDiffLine` never gives them a ref.
This is not new to scroll/back — press/click/fill/longpress had the same gap
since #1101. MCP was never affected: its ref-pin store rewrites plain refs on
the way in, which is why the model never sees a suffix.
Live: `scroll down --settle` now emits `+ @e14~s218078 [cell] "Game Center"`,
and `press @e14~s218078` copied straight out of that line taps successfully.
* test: record the pinned-diff-ref bytes in the output-economy baseline
Rendering added diff-line refs pinned costs 8 bytes in the two settle CLI
text samples (two `~s<gen>` suffixes). The output-economy baseline is the
tripwire for exactly this, so the increase takes an explicit reviewed waiver
rather than a silent baseline bump — the same one the settled TAIL's pins
already carry, for the same ADR 0014 reason.
Only `bytes` moves: lines, refs, hints, and shape are unchanged, which is the
evidence that this is a suffix on existing refs and not a new payload.
Caught by CI, not locally: `pnpm test:unit` runs unit-core and
subprocess-stub only, while the Coverage lane runs every vitest project.
* test: prove the generic settle degrades when its runtime cannot be built
`createGenericSettleRuntime` catches and returns undefined so an observation
that cannot even start does not fail an action that already succeeded. That
was a claim in a docstring with nothing behind it — the one changed line the
coverage gate reported uncovered (95/96).
The test puts the session in the state the catch exists for: the router
handed us a session that is no longer in the store, so building the settle
runtime throws SESSION_NOT_FOUND. The response keeps its scroll result and
simply carries no settle payload. Removing the try/catch fails it.
* build: teach fallow that vi.mock reaches pinOwnProcessStartTime dynamically
Not from this PR: #1642 added `pinOwnProcessStartTime` on main, and its three
consumers reach it the only way a Vitest module mock can —
`vi.mock(path, async (importOriginal) => (await import('...')).pinOwnProcessStartTime(...))`.
Dependency analysis cannot follow that dynamic import to a consumer, so the
export reads as dead the moment any PR pulls that file into its audit scope.
This PR is the one that did.
The entry records the consumers by path and the reason, matching the
daemon route-handler entry directly above it, which exists for the same
dynamic-`import()` limitation.
* refactor: adopt the best of the parallel #1653 implementation
Two sessions independently built #1638 (PR #1650 and PR #1653) and converged
on the same architecture — trait in the registry, one engine with two entry
points, runtime-command seam, lazy import, preserve-daemon, stored-baseline
honesty. #1650 continues; this folds in what #1653 did better:
- The agent-facing help core loop (cli-help.ts) now names scroll and back as
settle-capable. Without this, the benchmarked closed-grammar help line kept
instructing agents that --settle is only for press/click/fill/longpress —
actively steering the AppControlBench models away from what #1638 shipped.
- issueSettleRefs moves into session-snapshot.ts, beside the partial-frame
primitive it wraps, deleting the single-function settle-ref-issuance module.
- Their seam tests: back reader→writer settle plumbing, back CLI settle
rendering, and a trait-less generic command (home) ignoring a stray settle
flag rather than observing or rejecting.
What #1650 had that #1653 lacked, for the record: the SETTLE_REF_ISSUING_TOOLS
registry derivation (without it, MCP never pins a scroll/back settle diff's
refs and the partial frame rejects every follow-up), BackCommandResult.settle
in contracts, back's MCP output schema, paste-ready pinned diff refs, and the
docs/changelog/baseline surfaces.
* bench: help-conformance case for settled scroll-to-find planning
The #1638 extension of the closed --settle grammar to scroll/back is the
feature's entire payoff — collapsing scroll-then-observe into one call — and
the closed command list is an enumerated N whose enumerator is this bench.
The regex over the help text proves the sentence exists; this case checks
whether a model plans differently because of it.
One focused case, deliberately not coached: a pinned visible-first snapshot
(rendered by formatSnapshotText, pinned by the sample-producers gate) whose
wanted row is summarized off-screen with no ref anywhere in the output. The
tempting pre-#1638 plan is `scroll` plus a separate `snapshot -i`; acceptance
is the single settled call. Scoring was verified against eight plan shapes in
both directions before recording.
Model-backed record (claude-haiku-4-5, 3 trials, current help): 0/3 — but the
decomposition is the finding. Settle eligibility GENERALIZED (3/3 trials put
--settle on scroll unprompted; the mutation-suffix framing concern did not
materialize) and the two-call habit is residual (1/3). All three trials failed
on `scroll @e3 down --settle` — the pre-existing #1366 scroll-takes-no-target
confusion, which the live CLI recovers with a dedicated hint but a single-shot
bench cannot. The recorded gap is therefore a first-30 doc gap (nothing
teaches that scroll takes no target), not a settle-eligibility gap; tuning the
case until it passes would just delete the evidence.
Three places smuggled structured data through untyped bags and re-read it
with runtime guards. Each gets an explicit typed boundary.
A. The resolution-suppression rule was encoded twice in
interaction-touch-response.ts — a spread ternary in the runner-payload
branch and an unconditional destructure used conditionally in the runtime
branch, with the ADR 0012 rationale living on only one source variant.
Both branches now read one `suppressesResolutionDisclosure(source)`
predicate through one `applyResolutionDisclosurePolicy` helper, where the
reason is stated once. The union field is renamed
`maestroCoordinateFallbackDispatched` (the dispatch path that ran) and
hoisted into a shared base. handleFillCommand's two-arm interactor.fill
call collapses to one.
B. `Interactor.type` narrows from `Record<string, unknown> | void` to
`TypeTextBackendResult | void`; the Apple runner boundary is the single
place the wire payload becomes that type. `maestroFallbackDetails` returns
a typed `{ used, extra }` instead of a bag both call sites re-read.
C. `details.candidates` meant two incompatible things. The device-domain
resolvers now key their list `devices`, so the shared renderer drops its
shape-disambiguation guards and the device list actually renders.
* fix: surface AMBIGUOUS_MATCH candidates and name find's supported actions (#1597)
AMBIGUOUS_MATCH errors now list the matching candidates (ref, role,
label/identifier) rendered the same way as snapshot -i lines, capped at
5 with a "+N more" marker. buildAmbiguousMatchError (the single
producer, src/daemon/handlers/find.ts) reuses formatSnapshotLine to
build the list; formatAmbiguousMatchCandidateLines (src/utils/output.ts)
renders it unconditionally on both text surfaces an agent actually
reads (CLI printHumanError and MCP formatToolErrorText) — previously
the candidates lived only in details, which neither surface printed.
find's "Unsupported find action: X" (e.g. from `find <text> press`)
now attaches a hint naming every action find actually supports and the
two-step recovery shape: run find "<text>" to resolve the ref, then
dispatch the gesture as its own command (press @eNN). The hint is a
single exported constant (UNSUPPORTED_FIND_ACTION_HINT) shared by both
throw sites — packages/selectors' raw-token parser and the CLI's typed
reader (src/commands/interaction/selectors.ts) — so they can't drift.
Matching semantics are unchanged; ambiguous rejection stays by-design.
The help-conformance corpus's AMBIGUOUS_MATCH quiz is updated: its
premise ("candidate refs were not shown") no longer holds, but with 3
identically-labeled candidates the lesson (don't guess a specific ref)
still holds.
* fix: guard the AMBIGUOUS_MATCH candidate renderer against device-domain shapes
Review on #1602 (P2): formatAmbiguousMatchCandidateLines ran for every
normalized error and stringified details.candidates unconditionally,
but device-domain AMBIGUOUS_MATCH/APP_NOT_INSTALLED errors
(findBootedAppleSimulatorWithApp, src/core/dispatch-resolve.ts) reuse
that key for { id, name } device objects with no `matches` field —
CLI and MCP would have printed "Candidates: [object Object]" for
those. The renderer now requires numeric details.matches AND every
candidate to be a string before rendering anything, restricting it to
buildAmbiguousMatchError's element-match shape; unrecognized shapes
render nothing, same as before this feature existed. Added regression
tests against the exact device-error shape on both text surfaces.
Also unexports AMBIGUOUS_MATCH_CANDIDATE_LIMIT (fallow flagged it as
an unused production export) — it has no consumer outside find.ts.
* feat(bench): renderer-pinned samples, topic-coverage gate, error quizzes; trim skillgym to agentic checks
The help conformance bench's quoted CLI output is now sourced from
scripts/help-conformance-sample-outputs.mjs, and every sample is rebuilt
through the real production renderers (settle output formatters,
printHumanError, formatSnapshotText, refMutationAdmissionResponse) by
scripts/__tests__/help-conformance-sample-outputs.test.ts — a rendering or
message change fails deterministically instead of leaving the bench grading
against output the CLI no longer prints. This retires the fabricated
recoverable-failure envelope (production never throws a textual settle
timeout; that case is replaced by a real DEVICE_IN_USE recovery quiz).
Bench cases move to scripts/help-conformance-cases.mjs and are enumerated
against the help-topic registry: helpTopicIds() is exported from cli-help,
and scripts/__tests__/help-conformance-topic-coverage.test.ts fails when a
help topic has neither a bench case nor an explicit waiver. New case
families: error-envelope recovery quizzes (device-in-use, stale pinned ref,
ambiguous find match, app-not-installed) pinned to real error text, topic
coverage for tv/web/react-native/debugging/workflow, and a metamorphic twin
of the settled-diff quiz.
The skillgym smoke suite shrinks from 119 cases to the 5 that measure what
only an agentic runner can show: skill routing plus output interpretation
with a proven local CLI help probe (local-cli-help-policy). Its embedded
samples now import the same pinned constants, replacing hand-transcribed
output that had already drifted from the renderer. Knowledge checks belong
to the bench; live fixture behavior belongs to the iOS simulator e2e suite.
* review: drive error samples through the real producers; enforce local-help on the routing smoke
The DEVICE_IN_USE, AMBIGUOUS_MATCH, and APP_NOT_INSTALLED parity tests no
longer hand-author the producer message before rendering: each drives the
actual producer — buildDeviceInUseBySessionError (extracted in
session-open.ts and called by the handler), buildAmbiguousMatchError (now
exported from find.ts), and buildAppNotInstalledError (extracted in
app-resolution.ts and thrown by the resolver). Because each factory is
exported from its producer file and called by the production path, dropping
the production call would make it test-only and fail
check:production-exports — the wiring is gate-enforced, not conventional.
open-and-snapshot now sets requireLocalCliHelp and
allowOnlyLocalCliHelpCommands, so the 'skill plus local help' claim is
observed rather than assumed; without them the case can pass on model prior
alone.
Michał Pierzchała