Commit Graph

18 Commits

Author SHA1 Message Date
Michał Pierzchała 7f61df30ae feat: add TV remote command (#1147)
* feat: add TV remote command

* feat: improve TV remote ergonomics

* test: cover tv-remote provider scenario

* fix: preserve focused Android TV nodes

* docs: tighten PR description guidance

* fix: remove d-pad command alias

* docs: clarify tv-remote hold syntax

* feat: add tv-remote longpress CLI sugar
2026-07-08 10:59:48 +02:00
Michał Pierzchała 5c5fa012f7 feat: --settle returns the settled diff in the interaction response (#1101) (#1106)
* feat: --settle returns the settled diff in the interaction response (#1101)

press/click/fill/longpress --settle executes the action, waits for the UI
to go quiet (wait stable's loop, shared via stable-capture.ts), and returns
the settled diff vs the pre-action tree in the same response — one round
trip instead of the interact -> observe pair.

- payload: changed lines only (bounded), summary counts, added-line refs,
  refsGeneration; best-effort (settled:false + hint on never-quiet content,
  never an action failure); --verify shares the settle captures
- ref issuance: the settled tree becomes the session snapshot; a
  diff-carrying settle response clears snapshotRefsStale and the MCP layer
  merge-only re-pins added-line refs at the settle generation
- grammar: --settle + --settle-quiet <ms> + --timeout <ms> (flag-sourced
  descriptor budget with new envelope:'widen' semantics mirroring wait)
- ADR 0011: new settleObservation guarantee classified on every path with
  contract scenarios per enforced/delegated cell

* test: give the two contention-flaky doctor scenarios explicit budgets

The doctor provider scenarios sit at ~5s of real daemon-harness work on a
loaded host and flake at vitest's 5s default during full-suite runs (the
known contention flake AGENTS.md documents). Same in-file precedent as the
Metro-probe scenario's 10s budget.

* fix: move SettleParams to contracts to satisfy the layering DAG

daemon/handlers/interaction-flags.ts imported the type across the
daemon -> commands boundary (R2 commands-floor). The tuning params are
part of the interaction contract like SettleObservation, so they live
in contracts/interaction.ts and both layers import from there.

* feat: keep settle diffs content-first — drop Key nodes, added lines win the cap

Bluesky dogfood: a fill that summons the iOS keyboard spent 49 of the 80
capped diff lines spelling out QWERTY keys, and a screen transition with
269 removals could starve out the added lines entirely. Key-type nodes
are now filtered from both diff sides (the [keyboard] container line
still signals presence), and under truncation added lines — the ones
carrying fresh refs — win slots over removals.

* docs: state the core loop in the top-level help starting point

Benchmarked with headless haiku/sonnet agents given only --help: both
models skipped the help-workflow pointer and started with plain
snapshot (38KB payloads they then had to re-read from files). One
core-loop line at the starting point is what teaches snapshot -i and
--settle to models that never read a second help page.

* fix: preserve settle digest refs for mcp

* fix: reduce settle fallow complexity

* fix: surface settle output in CLI text

* fix: complete settle handling for longpress

* refactor: localize daemon timeout envelopes

* refactor: deepen post-action observation

* refactor: centralize post-action observation planning

* refactor: derive settle capability from descriptors

* refactor: trim settle descriptor helpers
2026-07-06 20:18:44 +02:00
Michał Pierzchała 664b713130 feat(interaction): opt-in --verify evidence for press/click/fill (#1064)
* feat(interaction): opt-in --verify evidence for press/click/fill (#1047)

Adds an opt-in --verify flag that returns cheap post-action evidence
(foregroundApp, nodeCount, interactiveNodeCount, digest,
changedFromBefore) instead of requiring a full follow-up snapshot to
confirm a mutating command had an effect. The digest hashes the
(type, label, identifier) multiset of an interactive-only capture,
order-independent so it doesn't flip on harmless re-ordering; the
node tree is never serialized back to the client, only the digest and
counts. Default behavior (no --verify) is byte-identical to today.

Implements the approved design from the #1047 issue comment:
- src/utils/ax-digest.ts: new standalone digest module.
- Pre-action digest reuses the snapshot the resolution path already
  captures for ref/selector targets (zero extra cost); point targets
  opt into one extra baseline capture only when --verify is set.
- Post-action: one interactive-only capture through the same capture
  helper, digested and discarded.
- --verify threaded through the CLI flag schema, MCP input schema,
  interactionResultExtra allowlist, and MCP output schemas, following
  the same plumbing as --double-tap and the #1040 targetHittable
  precedent.
- The native-ref/direct-iOS-selector fast paths are skipped when
  --verify is set, since they bypass the resolution/capture path
  evidence depends on.

* test: cover press --verify with a provider scenario; classify the flag

CI's architecture-progress gate failed on 1 unclassified public flag.
--verify drives real device captures, so it belongs in the
device-observable list backed by an actual provider scenario rather than
the intentionally-outside bucket: the new scenario asserts evidence
(changedFromBefore, digest, nodeCount) on press @ref --verify, that the
verify capture's tree is never serialized into the response, and that
the transcript completes (snapshot -> tap -> verify snapshot; the @ref
path reuses the session snapshot as its baseline, so no extra
resolution capture entry exists).

* fix: type the verify scenario transcript entries

* fix: include interaction extras in the fill @ref response branch

The ref branch of dispatchFillViaRuntime rebuilt responseData from
backendResult/coordinates, dropping interactionResultExtra(result) — so
fill @ref --verify returned no evidence even though the post-action
capture ran (live E2E gap found in PR #1064 review). Spreading the
extras also gives fill @ref the same ref/refLabel/selectorChain (and
conditional targetHittable/hint) fields press @ref already returns.

Adds daemon tests for fill @ref --verify evidence and for the no-verify
path staying evidence-free with no post-action capture.
2026-07-04 10:48:54 +02:00
Michał Pierzchała d21b8ce32e feat: add doctor command (#883)
* feat: add doctor command

* fix: reduce doctor command complexity

* fix: classify doctor integration flags

* fix: simplify doctor setup

* refactor: split doctor checks

* fix: simplify doctor check set

* fix: include stopped android avds in devices

* fix: report doctor device inventory

* refactor: reuse device inventory selectors

* fix: summarize doctor inventory by platform

* fix: show metro cwd in doctor

* refactor: simplify metro doctor lookup

* fix: update doctor imports after apple consolidation

* feat: make doctor Metro probe controllable and surface hidden toolchain failures

Two gaps found while verifying the doctor command on a real environment:

- Metro host/port were uncontrollable from the CLI: --metro-host/--metro-port
  were rejected by allowedFlags, and readDoctorOptions only read them from
  req.runtime (populated by remote/connection profiles, never a plain CLI
  flag). The Metro check's own hint told users to 'pass the correct
  --metro-host/--metro-port', which did not exist. Declare the flags and read
  them from req.flags (runtime kept as fallback) so the probe can target any
  endpoint, e.g. from outside an RN/Expo project directory.

- A broken per-platform toolchain was silently hidden: readDoctorDeviceInventory
  dropped inventory failures whenever any other platform returned devices, so a
  broken Xcode or Android SDK still reported a green 'pass'. Keep the failures
  and surface each as a warn (device-<platform>) when other platforms have
  devices; scoped --platform runs stay quiet.

* fix: align doctor CI expectations

* feat: extend doctor preflight checks

* fix: keep doctor checks within ci gates

* fix: simplify doctor metro surface

* refactor: trim doctor bundle impact

* fix: restore useful doctor diagnostics

* refactor: reuse doctor output helpers

* refactor: share device inventory grouping

* refactor: keep doctor focused on preflight checks

* refactor: simplify doctor toolchain probes

* fix: keep scoped simulator hint generic

* fix: clarify doctor Xcode selection context

* fix: recognize provider scope in remote doctor

* fix: address doctor review gaps

* fix: keep doctor metro checks inferred
2026-07-01 14:24:32 +02:00
Michał Pierzchała a707348b9b feat: add hosted WebDriver provider support (#948)
* feat: add cloud webdriver artifacts

* fix: clean up local session after provider release failure

* fix: tag cloud webdriver provider requests

* feat: connect hosted webdriver providers

* docs: document hosted provider credentials

* refactor: tighten cloud webdriver provider internals

* refactor: consolidate cloud webdriver provider definitions

* refactor: collapse hosted webdriver runtime wrapper

* refactor: reduce cloud webdriver smell surface

* docs: clarify hosted provider interfaces

* fix: avoid regex slash trimming in webdriver urls

* docs: rename hosted providers to device clouds

* fix: align provider profile imports with remote modules

* fix: skip local android recovery for provider devices

* test: classify cloud provider integration flags

* fix: close active cloud connection session

* test: cover provider disconnect cli flow

* fix: make cloud webdriver sessions launchable

* fix: align cloud webdriver input gestures

* fix: avoid keyboard input during cloud scroll

* fix: constrain cloud webdriver scroll gestures

* refactor: isolate cloud webdriver scroll frame

* refactor: deduplicate cloud webdriver helpers

* refactor: tighten cloud webdriver action types

* fix: harden cloud webdriver release

* fix: polish provider disconnect diagnostics

* refactor: group connection profile helpers

* fix: repair rebased internal paths

* fix: satisfy cloud webdriver CI guards
2026-07-01 13:01:47 +02:00
Michał Pierzchała 65227c6719 refactor: absorb CLI parser into cli/parser/ — Phase 5 (#958)
Move the CLI argument/flag/help parser out of utils/ into a dedicated
src/cli/parser/ folder, per plans/perfect-shape.md §5.5 (utils/ hosts a 3k
CLI parser among its buried subsystems).

Files moved (3): args, cli-flags, cli-help (args->cli-help intra-set import
stays relative).

- git renames; importers repointed via the resolve-based codemod
  (64 importers; staying-utils/kernel deps recomputed to ../../)
- no public-export/rslib impact
- update scripts/integration-progress-model.ts import + fallow-baselines/
  health.json keys (args incl. :high impact variant)

Behaviorless path codemod. typecheck/lint/format/build/tests green;
integration-progress model still runs.
2026-06-30 13:56:04 +02:00
Michał Pierzchała 43d0a639d0 feat: leveled response views + --level knob with snapshot digest — Phase 4 (#942)
* feat: leveled response views + --level knob, with a snapshot digest — Phase 4

Add the agent-cost leveled-response system: a responseLevel knob
(digest | default | full) plumbed end to end behind a global --level flag
(mirroring --cost), and a per-command ResponseView registry applied in the
router on the success path.

- contracts: RESPONSE_LEVELS/ResponseLevel + meta.responseLevel + boundary
  schema whitelist. Plumbing mirrors --cost: cli-flags FlagDefinition +
  GLOBAL_FLAG_KEYS, AgentDeviceClientConfig + overrides, buildClientConfig,
  buildMeta. ResponseLevel exported from the public root.
- src/daemon/response-views.ts: the ResponseView registry. Seeds the snapshot
  digest — the full node tree (the dominant token sink) collapses to
  { nodeCount, refs: first 12 hittable/non-occluded refs with labels } plus the
  cheap top-level signals (truncated/visibility/snapshotQuality). full returns
  today's shape (nothing richer is computed yet).
- router graft (applyResponseLevelView + applyAgentCostGrafts): composes with
  the existing cost block. With responseLevel default (or unset) AND no
  registered view AND no --cost, the original response is returned UNCHANGED —
  byte-identical to today (Maestro .ad recompare safe). cost.nodeCount reads the
  original node tree so it stays accurate even after a digest.

Tests: snapshot view unit test (digest filters hittable/occluded, drops the
tree, keeps cheap signals; default/full passthrough); router graft test via an
injected view (default identity byte-identical, digest applies, full passthrough,
digest+cost composition, unregistered-command passthrough, boundary parse).

Verified: tsc, oxfmt + oxlint --deny-warnings, fallow audit clean, rslib build,
Layering Guard empty, 1106 daemon/contracts/client tests pass (incl. the
existing cost/typed-error grafts after the restructure).

* fix: repoint MCP output-schemas import to kernel/device (rebase fixup)

The kernel move (#940) deleted src/utils/device.ts; #941's
command-output-schemas.ts (merged after #940's codemod ran) still imported the
old path. Same one-line fix as #943; de-dups once that lands.

* fix: re-classify responseLevel flag in integration-progress model

The --level/responseLevel flag is a diagnostics/output flag (not device-
observable), classified in the exclusion bucket alongside --cost. (Lost in an
earlier rebase; re-applying.)
2026-06-30 07:53:28 +02:00
Michał Pierzchała 8cc6bb8b92 feat: add replay test reporters (#936)
* feat: add replay test reporters

* feat: support custom replay test reporters

* fix: satisfy reporter CI guards

* refactor: modularize replay test reporters

* refactor: trim replay reporter parsing

* refactor: simplify custom replay reporters
2026-06-29 20:37:41 +02:00
Michał Pierzchała 54178319d8 perf: reduce iOS simulator screenshot overhead (#927) 2026-06-29 15:53:19 +02:00
Michał Pierzchała 62f923dec4 feat: opt-in agent-cost wallClockMs behind --cost — Phase 4 slice 1 (#922)
* feat: opt-in agent-cost wallClockMs behind --cost

Add per-command wall-clock latency as a purely additive, opt-in response
field (cost.wallClockMs) gated behind a new global --cost flag.

The flag plumbs end to end mirroring --debug: cli-flags definition +
GLOBAL_FLAG_KEYS, AgentDeviceClientConfig/overrides, buildClientConfig,
buildMeta (meta.includeCost), the DaemonRequestMeta contract, and the
boundary parse in daemonCommandRequestSchema so it survives the HTTP edge.

The graft lives in request-router handleRequest (the seam that owns the
outer wall-clock incl. lock + execute + finalize). It mirrors the
conditional registerDownloadableArtifacts spread: when --cost is off OR the
response is an error, the response is returned untouched. Only on an opted-in
successful response is cost appended, so the default serialized DaemonResponse
is byte-identical to today (Maestro .ad recompare safe). Proven by the parity
test (flag-off identity, flag-on additive-only, error path, boundary survival).

Additive / semver-minor. MCP exposure and richer signals (roundTrips,
nodeCount) are deferred to follow-up slices.

* test: classify --cost flag as outside provider-backed integration

The integration progress guard (test:integration:progress:check) treats every
public CLI flag as either device-observable (requiring provider-backed coverage)
or intentionally excluded. --cost is a diagnostics/output flag (a purely additive
response field, not device-observable), so it joins json/help/version/verbose in
the 'config, output, diagnostics, and transport' exclusion bucket.
2026-06-29 13:50:12 +02:00
Michał Pierzchała 24cb2b622c fix: refine Apple provider pressure reporting (#877) 2026-06-25 17:34:50 +02:00
Michał Pierzchała ba825d8df2 fix: use desktop scroll events on macOS (#863)
* fix: use desktop scroll events on macOS

* fix: support paced macOS desktop scroll

* fix: clear scroll CI quality gates

* fix: address macos scroll review feedback

* refactor: simplify macos scroll plumbing

* fix: tighten scroll duration contract

* fix: limit apple scroll duration reporting
2026-06-25 07:38:21 +02:00
Michał Pierzchała 7739b71a25 refactor: centralize command family facets (#849) 2026-06-24 12:40:03 +02:00
Michał Pierzchała d47cd30117 feat: add agent-device proxy command (#844) 2026-06-23 17:20:08 +02:00
Michał Pierzchała bc5726e8b0 test: cover web provider scenario (#827) 2026-06-19 13:39:39 +02:00
Michał Pierzchała feb5309e38 fix: classify external xctest runner flags (#810) 2026-06-16 15:54:00 +01:00
Michał Pierzchała 5c83fe4e50 fix: improve daemon diagnostics and remove compact snapshots (#786)
* fix: improve daemon startup diagnostics

* perf: relax query-sweep recovery budget

* fix: make compact snapshot flag a no-op

* fix: remove compact snapshot mode

* test: align tvOS remote provider expectation

* test: drop compact snapshot progress target
2026-06-12 17:32:56 +02:00
Michał Pierzchała fa1c1d55da refactor: localize command surface modules (#772)
* refactor: localize command surface modules

* refactor: localize command runtime modules

* refactor: tighten localized command exports

* refactor: address command localization review

* refactor: deepen batch command policy

* refactor: split provider progress model

* refactor: collapse command client facets

* refactor: run progress metrics as TypeScript

* refactor: remove obsolete command shims

* fix: update localized snapshot output import

* fix: preserve debug command localization
2026-06-12 14:02:04 +02:00