mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
perf/1961-cli-compile-cache
77 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Michał Pierzchała
|
03c3984066 |
perf(contracts): granularize entry surfaces so hub importers stop evaluating the facade clump (#1969)
* perf(contracts): granularize entry surfaces so hub importers stop evaluating the facade clump
`@agent-device/contracts/platform` unions 32 vocabulary modules and
`/interaction` another 18. A file that value-imports either evaluates the whole
union to reach one function, and because permanent hubs sat behind them —
`command-descriptor/registry.ts`, `core/capabilities.ts`,
`interactors/register-builtins.ts`, `command-descriptor/platform-execution-entry.ts` —
that union rode into roughly half the unit suite's test graphs.
Give every vocabulary module its own entry subpath and move all value-importers
onto the module that owns the symbol. Type-only importers are left alone: `import
type` is erased, so it already evaluated nothing.
Measured with the #1950 eager-import-closure walker over all 974 unit-core test
files, against base
|
||
|
Michał Pierzchała
|
1f80c92a27 | refactor: derive every --settle surface from the descriptor trait (#1652) (#1945) | ||
|
Michał Pierzchała
|
d0547dcb97 |
refactor: complete the find cutover onto the request-bound runtime (#1944)
* refactor: complete the find cutover onto the request-bound runtime The deferred Wave 4 unit for #1739 (R35), unblocked by focus (R40) and type (R41). Find's read-only legs, focus leg, and type leg already ran bound; the one remaining direct platform execution was the mutating-target capture, which built createSelectorCaptureRuntime without a bound capture and fell through the legacy dispatch branch reserved for "the last one to migrate". - The mutating path now enters resolveBoundSelectorCapture — the selector family's shared admit-then-bind entry, which already named find in its intent table — and threads the bound capture into the target capture. - Find was that last one: `capture` on SelectorCaptureRuntimeParams is now required and the legacy fallback branch is deleted. The backend's `bound` becomes required-to-state, with the observation-free duration wait (`wait 400`) as the one declared absence — its runtime now carries no capture backend at all, so an accidental capture fails loudly instead of falling anywhere. - The descriptor flips to device-runtime with findRuntimePlanUses (the selector-text plans shared with get, plus focusRuntimeUse and typeTextRuntimeUse); the capability bucket and both overlay memberships (HARMONYOS_SUPPORTED_COMMANDS, WEB_QUERY_COMMANDS) are deleted. - R35 lands with the selector family's shared operation owners; the capture and backend tests move off the dispatch mock onto the bound seam, which is where the poll-deadline and private-ax-pin assertions actually live now. Wave 4 is complete: layering recognizes 26 migrated commands. * fix(find): one action-selected bind per mutating handler (ADR 0019 §9) Review P1 on #1944: a mutating find performed up to three separate facts/admit/bind projections — capture, then focus, then type re-admitted per leg. The request handler now resolves ONE action-selected plan and binds once: - New selector intents find-focus / find-type carry combined uses (capture + focusPoint, capture + focusPoint + typeText) through the same admit-then-bind path and plan machinery every selector capture uses; the new bind arms reuse the existing capture selectors, so the shared operation owners stay single. Delegated click/fill resolve targets on the plain capture pair. - The handler threads the one bind's operations to the shared executors: executeFocusPoint is extracted as the single lexical owner of the focusPoint call (R40's owner claim follows it), and executeBoundTypeText's runtime param narrows to the operations it actually uses so find can pass its own broader bind through it. - findRuntimePlanUses becomes the full action-selected set (eight uses), and the descriptor test pins each use's exact requirement list. - Regression: find focus and find type each assert exactly one facts inspection and one bindDevice call — the pre-fix handler fails both (two and three binds respectively). Live re-verified on iPhone 17 Pro at this head: find focus and find type both execute through the single bind, route synthesized-first-responder, typed text visible in the captured tree. |
||
|
Michał Pierzchała
|
81409f1a7c |
refactor: migrate type to the request-bound device runtime (#1935)
* refactor: migrate type to the request-bound device runtime
Wave 5 unit 2 for #1739 (ADR 0019), find's last blocker. `type "text"` and
`find <q> type "text"` now reach the device through one admitted, request-bound
`typeText` operation instead of the `handleTypeCommand` interactor leaf and its
dispatch-table arm.
- New `TypeTextRuntimeOperations` contract riding the same `Interactor` seam as
focus/screenshot/element-text; the operation returns the interactor's own
closed `TypeTextBackendResult`, so Apple route evidence passes through and
every other owner types blind, exactly as before. The iOS synthesized-type
commit wait (#1676) is Apple-interactor-internal and moves nowhere.
- The interaction backend's `typeText` member exists only when the `type`
handler admitted and bound a runtime — no caller can fall back to legacy
dispatch, so the command keeps exactly one execution path (R41).
- `executeBoundTypeText` reproduces the retired leaf byte-for-byte: leading-ref
rejection with the same hint, space-joined positionals, the 0-10000 delay
bound, and only textEntryRoute surviving from the owner's result. Its parse
pins moved from the dispatch-level tests into the daemon runtime test.
- Exact-owner facts replace the capability bucket (apple sim+device, android
all-but-simulator-row, harmonyos emulator+device, linux device, web device,
vega unavailable, providers wherever their interactor is reachable) and
`type` leaves HARMONYOS_SUPPORTED_COMMANDS / WEB_INTERACTION_COMMANDS.
- The Linux desktop replay types a digit on real hardware; the coverage
manifest promotes `type` contract -> live with the two-sided count pins.
- Android/webdriver facts helpers extracted (androidTouchFact, interactorCell)
to keep inspectFacts under the complexity gate.
`find` stays legacy: both of its direct execution legs now share bound
runtimes, so the atomic R35 cutover is next.
* refactor(type): single-pass daemon routing, shared binder source, owner cell tests
Review follow-ups on #1935.
- The type handler now calls the bound executor directly: the interaction-
runtime hop validated and formatted what executeBoundTypeText validates and
formats again, so it is gone — no boundTypeText backend member, no second
result rebuild. The ADR 0014 frame expiry moves to the handler.
- New contracts/interactor-operation-binding.ts: one local resolver and one
fail-closed provider resolver shared by the screenshot, focus, and type
binders — three private copies retired, provider error text preserved.
- provider-limrun/interaction-operations.ts: the interactor-backed interaction
cells move out of the app-log owner (586 -> 563 lines, below its pre-unit
size); text interaction is composed by that owner, not defined in it.
- Every owner runtime test now pins the focusPoint/typeText fact cells and
bound-operation presence for its exact kinds: apple, android (incl. the
synthetic-simulator refusal), harmonyos, linux, web, vega (refusal + hint),
webdriver (reachability-gated, incl. inactive session), limrun (live +
recovery). The webdriver unsupported-capability row documents that
interaction gates on interactor reachability, not capture declarations.
- The Linux replay assertion is now change-sensitive: type "555" then wait for
a 555 node — no calculator button carries that label, so the wait passes only
if the keystrokes landed in the display; deleting the type step turns it red.
* fix(replay): give the calculator focus before the Linux type assertion
The change-sensitive wait exposed what the review predicted: the typed digits
never landed, because `focus 100 100` clicks the DESKTOP and takes keyboard
focus away from the calculator. The old broad assertion masked exactly this.
The retries then wedged on a latent quirk: attempt-1 leaves the pointer at
(100,100), and the next attempt's `xdotool mousemove --sync` to the same point
waits for a motion event that never comes, so every retry dies at the focus
step with a 10s timeout — which is why the lane reported step 7, not the
failing wait.
New tail: `focus 100 100` (R40 evidence + survival assert), then
`click "label=1"` — a resolved press inside the window that restores keyboard
focus, proves pointer input lands in the app, and moves the pointer off
(100,100) so retries cannot trip the mousemove no-op hang — then `type "55"`
and `wait "label=155 || text=155 || value=155"`. No button is labelled 155, so
the wait passes only if the typed keystrokes reached the display.
* refactor(type): delete the fallow SDK typeText surface, drop dead surface fields
Thermo-nuclear review follow-ups (reviewed at 82fb8c2dc; the three-hop relay
it names was already deleted in
|
||
|
Michał Pierzchała
|
17da776350 |
feat(ios): add snapshot backend conformance (#1930)
* feat(ios): add snapshot backend conformance * fix(ios): load built SDK at live runtime * test(client): isolate snapshot forwarding regression * refactor(snapshot): keep backend capability metadata internal * fix(test): merge backend conformance imports * fix(snapshot): keep backend forcing internal * refactor(snapshot): isolate backend capability fixtures * refactor(snapshot): keep capability governance internal * fix(ios): align snapshot actionability contract |
||
|
Michał Pierzchała
|
46eff36f85 |
refactor: migrate focus to the request-bound device runtime (#1925)
* refactor: migrate focus to the request-bound device runtime Wave 5's first unit (#1739, ADR 0019). `focus x y` and `find <q> focus` now reach the device through one admitted, request-bound `focusPoint` operation instead of the `handleFocusCommand` interactor leaf and its dispatch-table arm. - New `FocusRuntimeOperations` contract with local and provider interactor binders, mirroring the screenshot/element-text seam rather than inventing a second way for one operation class to reach its mechanics. - Exact-owner facts replace the capability bucket: apple simulator/device, android emulator/device/unknown, harmonyos emulator/device, linux device, web device, vega none, providers wherever their interactor is reachable. That is the retired bucket's cell table, restated as facts. - `focus` leaves BASE_COMMAND_CAPABILITY_MATRIX and both hand-maintained overlays (HARMONYOS_SUPPORTED_COMMANDS, WEB_INTERACTION_COMMANDS). - R40 is the new parametrized cutover row; `focusPoint` has exactly one owner. - The `x y` positional parse moves to utils and is shared with the still-legacy touch siblings, so a migrated command cannot drift from them. `find` stays legacy: this unit owns its focus leg only, its `type` leg still dispatches, and R35 waits on the Wave 5 `type` unit. * test(focus): cover the owning interactor binders, lower the find ratchet Review follow-ups on #1925. P1: focus-runtime.test.ts bound a fake focusPoint, so deleting the interactor call inside bindLocalFocusInteractor left focus a successful no-op with every test green. Adds packages/contracts/src/focus-runtime.test.ts, which executes both binders and asserts resolver context, positional (x, y) forwarding, the structured missing-provider failure, and that an already-cancelled request never resolves an interactor at all. Two planted mutants confirm it bites: removing `await interactor.focus(input.point.x, input.point.y)` and transposing its two arguments each fail exactly the two forwarding tests, while the daemon-level focus and find suites stay green — which is the gap the reviewer named. Coverage: find.test.ts shrank to 1204 lines when its focus assertion moved off the dispatch mock; the ratchet pin follows it down. * test(focus): add live Linux focus coverage to the desktop replay The Linux `focus` claim rested on the provider scenario at command-contract level. The desktop replay runs on real Linux hardware in the Smoke lane, so it now runs a coordinate focus and re-asserts the session survived it. Coordinate, not selector: the step exists to prove the migrated `focusPoint` path executes on real hardware, so it must not be able to fail on match ambiguity or CI layout drift. Reclassifies focus contract -> live in the Linux coverage manifest and updates the two pinned counts. The manifest gate is two-sided — a live claim must name a command the replay actually invokes — so the claim cannot drift from the file. |
||
|
Michał Pierzchała
|
1f8fdd0b5d |
fix: preserve Maestro clickable-first ordering (#1917)
* fix: preserve Maestro clickable-first ordering * test: cover Android Maestro clickable-first path * fix: keep Maestro fixture Android-only * fix: reveal Android Maestro targets in smoke scenario * fix: quote Maestro smoke assertion text * fix: retain Android Maestro clickability evidence |
||
|
Michał Pierzchała
|
56f2671a66 | perf: reduce cold iOS runner startup latency (#1927) | ||
|
Michał Pierzchała
|
be51870118 |
fix: make iOS scroll release controlled (#1906)
* fix: reduce iOS scroll overshoot * fix: make iOS scroll release controlled * fix: make controlled iOS scrolls deterministic * fix: preserve continuous drag sampling |
||
|
Michał Pierzchała
|
17bdca76cc |
refactor: migrate wait to request-bound runtime (#1875)
* refactor: migrate wait to request-bound runtime * fix: preserve native selector wait observation * fix: classify wait observations as conditional * refactor: compact conditional runtime declarations * fix: isolate selector runtime intents |
||
|
Michał Pierzchała
|
494eb52f66 |
refactor: migrate get to the request-bound device runtime (#1877)
* refactor: migrate get to the request-bound device runtime `get` declares `elementReadRuntimeUse` (required `captureSnapshot`, preferred `readTextAtPoint`), admits once from exact owner facts, refuses before binding, and binds exactly once. Its capability bucket, the static HarmonyOS/Web command sets that augmented it, and `requireCommandSupported` admission for `get` are gone; `'get'` leaves the `createSelectorRuntime` capability union. The neutral `readTextAtPoint` operation replaces the branch-per-family legacy `read` dispatch on the `get` path. Every local family and both providers now classify it exhaustively — Web, HarmonyOS, Vega and every provider row report it unavailable, which is behaviour-preserving because the legacy dispatch had no arm for them and threw on every call before falling back. R36 is the new parametrized cutover row. * fix(get): admit before the direct-iOS fast path; close the element-read outcome Review blockers on #1877. 1. `dispatchGetViaRuntime` could complete the direct-iOS selector query before `resolveBoundGetRuntime`. Once `get` declares `device-runtime`, ADR 0019 requires resolve -> admit -> bind before anything in the request path operates, so admission now runs first for every target shape and the fast path is a fast path *within* an admitted request. Regression: an eligible direct selector cannot operate when facts refuse admission. 2. `readTextAtPoint` returned `Promise<string>` and `readTextForNode` caught any throw and fell back, assigning a typed diagnostic after an untyped failure. It now returns a closed `ElementTextReadOutcome`; fallback happens only for the contract's classified reasons; unexpected errors propagate. The reason union is derived from its runtime list so the two cannot drift, and an unhandled reason is a compile error at the consumer. This retires the generic catch the start record promised. * feat(daemon): land the selector capture seam with get as its first consumer Takes ownership of the request-bound selector capture seam from #1876, which cannot ship standalone: with find's cutover deferred it had no consuming command (ADR 0019 §10) and was not dead-code clean (check:production-exports 19 -> 20). `get` is its first consumer, so it lands here. Adopts find's handoff as given. The one shape change, approved by the coordinator: the selector family gets its own capture uses carrying a PREFERRED `readTextAtPoint`, declared ALONGSIDE the snapshot uses so `snapshot`/`diff` keep binding exactly what they bind today. The read is surfaced through the existing arms of `bindSnapshotCaptureRuntime`, reusing the same selectActiveAppSnapshot / selectSnapshotWithoutActiveApp selectors — no second plan-to-operation dispatch. `get` now runs through `createBoundSelectorRuntime`; `resolveBoundGetRuntime` and its test are deleted as superseded, and `'get'` leaves the `createSelectorRuntime` capability union. The legacy read adapter survives for `find <q> get text` and is selected by which command constructed the runtime — never by failure, family, environment, or flag — so `get` cannot reach it. It retires in find's cutover, where the last consumer moves. * refactor: retire the read dispatch alias across both selector consumers Read-only `find` now constructs a BOUND selector backend, so `get text` and `find <q> get text` execute the same bound `readTextAtPoint` instead of one binding it and the other dispatching the legacy `read`. This moves find's READ LEG only: find's descriptor stays LEGACY_PLATFORM_EXECUTION and it claims no cutover row. With no consumer left, the whole chain goes: the `read` registry entry and its `dispatch: {}` projection, `DISPATCH_HANDLERS.read`, `handleReadCommand`, `interaction-read-legacy-dispatch.ts`, and the duplicate platform reader branches it carried. `read` was the only `dispatch-alias` descriptor, so that catalog group goes too. Deleting the registry entry drops 'read' from DescriptorDispatchCommandName, which makes a surviving DISPATCH_HANDLERS.read a compile error rather than something R36 has to police. R36 now claims the retirement it can prove. `find.test.ts` is over the size tripwire, so its handler invocation is extracted to find-handler-fixture.ts and the pin lowered 1237 -> 1221. * refactor(daemon): apply the seam addendum after #1876 was re-scoped Two edits, per find's ADDENDUM.md: 1. `includeRects` returns to `buildRuntimeCaptureInput`. It was removed from #1876 as unconsumed; the selector capture path is genuinely its first consumer (a Web rect capture requests bounds explicitly), so it lands here under the same rule that moved the seam. `snapshot`/`diff` pass nothing. 2. The per-capture `signal` is dropped, not restored. `CaptureSnapshotInput` has no such field on this stack — it moved to `wait` (#1875) with the regression that proves per-poll abort and quiescence. `get` captures once per resolution and never polls, so nothing here needs it. The seam test and fixture coverage for it moves with the contract rather than being kept against a field that no longer exists. * refactor(get): retire the direct-iOS selector shortcut `get` declares device-runtime, so its request path must reach the platform only through operations R36 declares. `dispatchDirectIosSelectorGet` reached `runAppleRunnerCommand` through a path the row declares no operation for; admitting before a bypass is not executing through the seam, so the bypass is removed rather than ordered after admission. Every target shape — including the simple iOS `id=` selector — now resolves through the bound capture. `queryDirectIosSelector` itself stays: `offscreen-target-probe.ts` still consumes it and it remains single-copy. `dispatchDirectIosSelectorIs` belongs to `is` (#1883). Two get-only helpers (`readDirectIosGetSelector`, `buildDirectIosGetResult`) became unreachable and are deleted with the caller. Declaring `querySelector` as a fact-admitted preferred operation was rejected on duplication, not correctness: the offscreen probe takes a plain session and cannot consume a bound operation, so it would ship the query twice until Wave 5 moves the probe — the deferred-duplication shape this PR was already overruled for on the `read` alias. It returns as a declared, §9-measured operation in a later unit that also moves the probe. Cost, stated plainly: `get text id=…` loses its tree-capture skip on iOS. No fallback was added and the latency is not recovered elsewhere. R36's singularExecution claim is now what the code does rather than aspirational. * refactor: ride the Interactor seam for the element read; drop the bespoke host Two operations of the same class were reaching their mechanics two different ways: `findText` rides `Interactor` via `localInteractors.resolve`, while `readTextAtPoint` had its own host port. That is duplication of MECHANISM, so the read now rides the same seam. `Interactor` gains `readTextAtPoint?`, implemented on the Apple, Android and Linux interactors where those mechanics already live. `src/platform-runtime-element-text-host.ts` and its `elementText` host wiring are deleted; the contract binds through the resolver exactly as the snapshot runtime does. Size honesty: this removes an 89-line module but the four readers still have to exist, so they moved into the interactors rather than vanishing. Net production change is ~4 lines, not ~89. The duplication of mechanism is what is actually fixed; Wave 5/6 retires the seam for both operations together. Also from the size investigation: - `ElementTextRuntimeExecution` was byte-identical to `SnapshotRuntimeExecution`; removed and reused, as `find-text-runtime.ts` does. - Removed a stranded, stale comment in `selector-capture-binding.ts` that still claimed a duplication this branch had already retired. - `FrozenUnavailablePlatformRuntimeFacts` is derived from its input type rather than restated, removing a 14-line clone group my new cell had pushed over the detector threshold. * refactor: migrate is to the request-bound device runtime (#1883) * refactor: migrate is to the request-bound device runtime `is` declares the shared selector capture use, admits once from exact owner facts, refuses before binding, and binds exactly once. Its capability bucket, the static HarmonyOS/Web command sets that augmented it, and `requireCommandSupported` admission for `is` are gone; `'is'` leaves the `createSelectorRuntime` capability union. Admission now runs BEFORE the direct-iOS selector fast path. ADR 0019 requires resolve -> admit -> bind before anything in a `device-runtime` command's request path reaches the device, so that query becomes a fast path *within* an admitted request rather than a way around exact-owner facts. The rule is documented once, on `createBoundSelectorRuntime`, replacing the two duplicated call-site comments `get` and `is` were each carrying. Declared behaviour change: `is` takes the active-app plan split, so the facts decide per family. On iOS `appBundleId` is the XCUITest attach target — with no tracked app the runner's own process comes to the foreground, displaces the app under test, and the capture then answers confidently about the runner's own blank screen. An iOS `is` on a session with no tracked app is now a typed SESSION_NOT_FOUND refusal carrying the `open` hint. Refusing beats displacing-and-lying. Android captures the real launcher in that state and is unchanged, which is what the platform facts already encoded. The two Apple watchOS cells move from capability-admitted-then-runner-failure to a typed unavailable refusal, the same classification snapshot, diff, and get already landed. R37 is the new parametrized cutover row. `find` keeps `createSelectorRuntime` and its `requireCommandSupported` call, so `captureData` stays optional and `captureSnapshotWithInteractor` stays: this unit is not the last selector unit. * fix(is): a failing iOS assertion fails instead of exiting zero Reverses part of #557, on thymikee's explicit instruction. `is` is an assertion: the docs state it "exits non-zero on failure". The direct-iOS fast path broke that contract — it reported a failed predicate as a completed command, so on device $ agent-device is text id=… "Wrong Expected Text" Passed: is text (exit 0) because `{ok: true, pass: false}` reaches `isCliOutput`, which renders "Passed: is <predicate>" without reading `pass`. A failing assertion reported as success lets a replay run on past a broken state. Now: Error (COMMAND_FAILED): is text failed for selector id=…: expected="Wrong Expected Text" actual="Apple Account, …" (exit 1) The renderer needed no patch: a negative can no longer produce a success envelope, so it is correct by construction. Direction chosen deliberately. Making the two paths agree could have gone either way, and "an agent asked a question and got an answer" is a real argument for the other one. This follows the DOCUMENTED contract rather than merely the incumbent behaviour, and the alternative is a far larger change: a zero-exit `is` would alter every platform and path, break scripts that rely on it failing the shell, and needs its own PR, docs, and probably a major version. It is also already how `is hidden` and `is exists` behave end to end. PASSING assertion, and that arm still answers with zero captures (pinned). Only the negative falls through — what #557's own summary asked for, "preserving snapshot fallback for misses", refusing fallback only for hard failures like ambiguity. The fall-through was #557's own design, never armed: the `| null` return and the caller's `if (!payload) return null;` guard were unreachable. This makes that dead guard live. Measured on iPhone 17 (median of 9, warm daemon): predicate holds 0.14s / 0 snapshots, unchanged; predicate fails 0.25s / 1 snapshot. ~+0.11s on failing assertions only. Correctness gain beyond the envelope: the fast path evaluates a ONE-NODE tree, so `visible` cannot see the ancestor geometry a list row inherits and its negative can be wrong. Falling through re-asks the real tree and can turn a spurious negative into a pass. The #557 pin moved with its reasoning at the pin site. * fix(layering): let a cutover row state a data-only admission retirement Review blocker on #1883: R37 claimed `legacyRetirement.routeNames: ['WEB_QUERY_COMMANDS_WITH_IS', 'HARMONYOS_IS_SUPPORT']`. Neither identifier has ever existed. They satisfied the non-empty shape check while proving nothing — the vacuous registry claim AGENTS.md warns about, and a green gate that would stay green if the deletion were reverted. The cause was the model, not the row. Every `LegacyRetirementClaim` form names something that must NOT exist, which a row can always satisfy by inventing a name. `is` retired no module, route, or dispatch projection because it had none: its legacy admission was a capability bucket plus membership in two static platform command sets, so its real retirement is a DATA deletion the model could not express. Rather than patch around that with sentinels or a per-command policy file — both forbidden by the playbook — this generalizes the model. `staticCommandSets` names the sets themselves and is proven from both sides: each must still be DECLARED in production source, and must no longer list the command. A fictional set fails the first half; a skipped deletion fails the second. That is what an identifier-shaped claim cannot state. R37 now claims HARMONYOS_SUPPORTED_COMMANDS and WEB_QUERY_COMMANDS, which is the deletion it actually performed. Planted red, both halves, against the real gate: [R37 is-runtime-cutover] 2 violation(s): (is cutover row):1 — claims retired static command set 'WEB_QUERY_COMMANDS_WITH_IS', which no production source declares (is cutover row):1 — claims retired static command set 'HARMONYOS_IS_SUPPORT', which no production source declares [R37 is-runtime-cutover] 2 violation(s): src/core/capabilities.ts:59 — static command set WEB_QUERY_COMMANDS still admits is so the exact claim that shipped is now rejected by name, and so is restoring the membership it claims to have removed. Mechanism cases live with the other planted-row tests; layering goes 177 -> 181. * test(is): pin the exit-code guarantee independently of what answers the predicate Prep for the Blocker 1 retirement, which deletes `buildDirectIosIsResult` — the function the #557 reversal fixed. The reversal's guarantee must not evaporate with it, so it gets a case that does not know how the daemon decided. `is` is documented to "exit non-zero on failure". The reversal proved that at the JSON envelope; nothing pinned it at the CLI boundary, which is where the defect was actually visible (`Passed: is text`, exit 0). This asserts the CLI contract directly: a `predicate_failed` response exits 1 and never renders as passed. It survives the retirement untouched, because it asserts the outcome rather than the path. Planted red with the exact pre-#1739 envelope the shortcut produced (`{ok: true, data: {pass: false}}`): `exitSpy.calls` is `[]` — no exit call at all — so the case fails, which is the regression it exists to catch. Unpushed on purpose: the restack will carry it into the retirement cycle. * refactor(is): retire the direct-iOS selector shortcut thymikee's ruling (option b). `is` declares `device-runtime`, so its request path must reach the device only through the operations R37 declares. It did not: a simple iOS `id=`/`label=` target was answered by a direct XCUITest querySelector without any capture, ordered after admission but not executing through the seam. This is not retired because it was wrong. `wait` hypothesized that the degenerate one-node evaluation mis-answers `is visible` for off-viewport nodes, traced it through the code convincingly, then tested it on device and it did not reproduce — XCUITest's own query is conservative about visibility, so the degenerate evaluation never gets the chance. It is retired because it was an undeclared, unmeasured bypass that made R37's singularExecution claim false: the same class of untruth as the sentinel retirement names fixed in the previous commit. Declaring querySelector as a real operation instead was rejected for a concrete reason: offscreen-target-probe.ts consumes queryDirectIosSelector with a plain session and cannot take a bound operation, so declaring it now would ship it twice until Wave 5 moves the probe — the deferred-duplication shape that got get's read deferral overruled. It returns as a declared, fact-admitted, section 9-measured operation in the unit that also moves the probe. Retired: dispatchDirectIosSelectorIs, its call site, buildDirectIosIsResult, and resolveDirectIosSelectorQuery — each had exactly one caller, all on this path — plus the ResolvedDirectIosSelectorQuery type they orphaned and two imports. queryDirectIosSelector itself stays: the offscreen probe still consumes it and it remains single-copy. Latency cost, stated plainly and not softened: a held predicate on a simple iOS selector goes from ~0.14s with no capture to ~0.25s with one, measured as the median of 9 warm runs on iPhone 17. There is no fallback and no fast path. R37's comment finally describes the code: "every predicate answers from the resolved tree" was written while the shortcut existed. Its scope is now stated too, so it is not read as absolute — the Android foreground-blocker diagnostic still reaches adb on the failure path, where it cannot produce or change a verdict; that edge is pre-existing, co-owned with wait, and recorded as Wave 6 denominator work with R22's appState as its declared replacement. Seven tests lost their subject. Those whose only content was the shortcut's own mechanics are deleted; the outcome-level ones are retargeted and keep asserting what survives. --------- Co-authored-by: agent <agent@local> * fix(contracts): a falsely advertised element read fails as a contract bug An owner whose facts advertised `readTextAtPoint` but whose interactor cannot perform it was reported as `{ status: 'unreadable', reason: 'surface-not-readable' }`. That put a contract violation inside the closed reason set that licenses falling back to the captured tree, so `get text` answered from potentially stale snapshot text precisely because the runtime lied about itself. ADR 0019 §2 requires the mismatch to fail as `runtime-contract-invalid`; it now throws. Removing the only producer of `surface-not-readable` made that reason dead: no path can reach it, since an interactor that HAS the read maps a blank or absent answer to `no-text-at-point` via `elementTextRead`. Dropped from the union, its consumer switch arm, and both test lists. `classifiedFallbackReason`'s `never` arm stays — it is what makes adding a reason a compile error rather than a silent untyped fallback. Deduplication found while auditing the change: - `invalidRuntimeContract` was module-private in `platform-runtime.ts`. It now owns its own module so both runtime modules share one construction. It is deliberately not exported through the platform facade: that facade must stay exhaustive over its sources, which would make this a public symbol with no external consumer. - The 8-field runner execution projection was written out three times (`snapshot-runtime-capture-input.ts`, `interaction-read.ts`, `screenshot-runtime.ts`). One `runtimeExecutionFromContext` now serves all three; `screenshotExecutionFromContext` keeps its name and delegates, since `ScreenshotRuntimeExecution` and `SnapshotRuntimeExecution` are the same type. Dropping a field here silently strips request id, log/trace paths, XCUITest overrides, or runner lease context — an operation that still answers but runs unconfigured, which is exactly the defect the wait unit hit as a P1. Red before green: with the old guard restored the new regression fails with "Missing expected rejection" — the call resolves instead of throwing, which is the silent degradation it exists to forbid. --------- Co-authored-by: agent <agent@local> |
||
|
Michał Pierzchała
|
fed7251336 | fix: calibrate iOS scroll execution (#1905) | ||
|
Michał Pierzchała
|
2eeb9c644f | fix(maestro): refresh target after selector miss (#1886) | ||
|
Michał Pierzchała
|
9ce1ef7587 |
feat(snapshot): move scope into presentation (#1855)
* feat(snapshot): move scope into presentation Use one preorder label/identifier/value policy across Swift and TypeScript, keep scoped iOS acquisition conservative, and remove the daemon's second scope pass. Non-vacuity: label-only matching failed identifier/value parity fixtures; Android pass-through failed its boundary test; disconnecting Swift applyScope produced eight scope/depth/projection failures. * fix(snapshot): select scopes with presented content * docs(snapshot): describe presentation-owned scope * docs(snapshot): record contribution-aware scope * refactor(snapshot): drop unrelated provider churn * fix(snapshot): accept healthy empty scoped capture * refactor(snapshot): isolate empty-scope admission * fix(snapshot): align scope ownership across runtimes * test(snapshot): pin post-wire scope owner * test(snapshot): retain find test shrink |
||
|
Michał Pierzchała
|
d8e03aea9b |
refactor: migrate screenshot to request-bound runtime (#1878)
Retires the last dispatchCommand edges for screen capture: the generic-route command, the sparse-snapshot fallback, and the Android snapshot-timeout evidence capture all admit exact owner facts and bind once (ADR 0019, cutover rule R39). --overlay-refs becomes part of the declared use, so a target that can capture pixels but not a tree is refused before anything is written to disk. |
||
|
Michał Pierzchała
|
79dddaf781 | refactor: tighten viewport runtime facts (#1873) | ||
|
Michał Pierzchała
|
37b1bc8cbd |
refactor: migrate viewport to request runtime (#1864)
* refactor: migrate viewport to request runtime * fix: preserve viewport cutover evidence |
||
|
Michał Pierzchała
|
294654a3a5 |
fix(android): resolve snapshot scope once and disclose the API 23 occlusion-scan gap (#1832 C1/C2) (#1846)
* fix(android): resolve snapshot scope once and disclose the API 23 occlusion-scan gap (#1832 C1/C2) - Android resolves --scope inside its projection only, under the shared scope specification (matchesSnapshotScope in @agent-device/contracts/snapshot: first document-order match over label/value/identifier, empty on no match). The daemon post-wire scopeSnapshotNodes pass skips the android backend, so scope has one owner and one no-match semantics instead of BFS+fallback followed by document-order+empty. - Golden table contracts/fixtures/snapshot-scope-policy.json is asserted against the predicate, the Android projection, and the daemon pass; the Swift runner twin (#1797) consumes the same table. - androidSnapshot.occlusionScanUnavailable discloses helper trees without drawing-order (API 23), where the covered-sibling pruner cannot run. Disclosure only; C1 stays open until occlusion moves to the daemon annotator. * fix(android): resolve scope over the presented tree and stop dropping it on interaction captures Adversarial review findings on the first commit: - BLOCKER: captureSnapshotData spread `snapshotScope: undefined` over flags, so an interaction capture (press/click/fill/longpress/hover --scope, --settle observation) reached the Android platform unscoped while buildSnapshotState still saw the scope. The post-wire pass used to rescue it; after skipping android it returned the unscoped tree. One effective scope now feeds both. - Scope resolves over the PRESENTED nodes of the requested projection, not the acquired tree, so an acquired match that membership drops no longer empties the snapshot, and Android matches the domain iOS's pass uses. - Slicing after the walk keeps ancestor context (hittable/collection/chrome) above the scope root, so scoped -i is a subset of unscoped -i; --depth stays scope-relative. - Shared findSnapshotScopeRange/reindexSnapshotNodes so the daemon pass and the Android projection run one implementation; scope slice extracted to ui-hierarchy-scope.ts (mirrors its test). - parseUiHierarchy moved to a test fixture module (it had no production caller left). - Golden rows sharpened (value row no longer matches via label on Android); the Android leg runs raw AND regular. CHANGELOG entry; docs wording corrected for iOS/@ref. * fix(layering): keep the contracts snapshot façade exhaustive over snapshot-scope * fix(android): scope to the first match whose subtree still has presented content Review P1 on #1846: with scope resolved strictly over presented nodes, `snapshot -i --scope panel` answered "no nodes" whenever the matched container was a structural view membership drops — even though the button inside it was exactly what was asked for — and `--depth 0` then hid a node the response prints at depth 0. The scope root is now the first document-order match whose subtree contributes at least one node to the requested projection, and the result is that subtree's presented nodes re-rooted at depth 0. Both failure modes die: a decorative match membership drops no longer empties the snapshot, and a dropped container still scopes to its content. `--depth` under scope filters the depths the response emits, so a node shown at depth 0 survives `--depth 0`. Tests: the golden legs stay raw+regular (bare TextViews cannot survive -i, so an -i leg would measure membership, not scope) with the projection interplay pinned by two dedicated tests on actionable shapes; the 'not re-scoped after the wire' case now runs a real parsed scoped tree instead of fabricated depth-0 siblings. |
||
|
Michał Pierzchała
|
d76e0f94e9 |
refactor: migrate snapshot to device runtime (#1779)
* refactor: migrate snapshot to device runtime * refactor: complete snapshot runtime policy cutover * test: enforce snapshot owner-facts admission * refactor: consolidate desktop snapshot capture * fix: scroll to visible iOS smoke targets * fix: close snapshot cutover alias bypasses * fix: constrain snapshot admission identity flow * fix: enforce snapshot admission through owner facts * fix: adapt replay source tests to snapshot runtime |
||
|
Michał Pierzchała
|
a853734f0c |
fix(webdriver): give cloud session creation its own budget and stop leaking billed sessions (#1782)
* fix(webdriver): give cloud session creation its own budget and stop leaking billed sessions Cloud lease allocation ran under the generic 30s/1-retry request policy, so BrowserStack iOS real-device session creation (45-90s) aborted client-side at ~60s on most runs. Each timed-out POST /session still completed server-side and, being non-idempotent, was retried — leaving two billed provider sessions per failed open with no id to release them. - POST /session is its own phase: a 180s create budget (default), zero retries, and no request-bound abort, so the daemon always learns the session id. - lease_allocate carries a 300s allocation budget surfaced to providers as LeaseLifecycleContext.deadline, and a matching 330s client envelope that preserves the daemon on timeout (a reset would SIGKILL mid-create and orphan every billed session the daemon held). - The request's cancellation signal is ownership evidence: a session that completes after the requester left is released, not registered; a create that the transport gives up on surfaces typed evidence (provider + lease) so an operator can find and stop the maybe-orphaned session. Closes #1774 * refactor: one canceled-request error, and tighten the #1774 shapes Review pass over the session-create fix: - The canceled-request error had nine hand-rolled copies (src/request/cancel, maestro shared, exec, retry, install-source x2, and the new provider one). It now has one definition in @agent-device/kernel/errors: createRequestCanceledError(details?, cause?) + isRequestCanceledError + REQUEST_CANCELED_REASON. Callers add evidence or a sharper hint; the reason itself is not overridable, so nothing can build one the predicate misses. - lease_allocate's timeout bundle moves beside INSTALL_TIMEOUT_POLICY in the registry (same {...DEFAULT, envelopeMs, onTimeout} shape); the request timeout constant stays exported from timeout-policy like its siblings. - Transport: fetch helper returns Response's own ok/status; the timeout reason const is private behind isWebDriverRequestTimeout. - Client: one-use options type inlined; the two deadline helpers share one floor. - Session-manager tests: shared makeRuntime/jsonResponse/afterEach restore. Net -29 lines with the feature in. * chore: keep the canceled-request reason private to the kernel * fix: typed cancellation everywhere + own the AWS remote-access ARN through startup Second-order follow-ups the #1774 refactor made cheap: - markRequestCanceled aborts the request signal WITH the kernel's typed canceled error as its reason. Every signal.throwIfAborted(), aborted fetch, and 'throw signal.reason' in the daemon (20+ sites) now surfaces a canceled request as such instead of a bare DOMException that normalized to UNKNOWN — and no site has to know the factory exists. - AWS Device Farm prepareSession owns the remote-access ARN from the moment create-remote-access-session answers: a startup timeout, the allocation deadline, or a canceled request now stops it before the failure surfaces (previously a timed-out startup left a RUNNING billed session behind — the same leak class as the WebDriver session, one phase earlier). The startup wait is capped by LeaseLifecycleContext.deadline and wakes on cancellation. - BrowserStack's pre-session local app upload honors the request signal (an upload is not billed, so plain abort is right there). - lease_heartbeat/lease_release share lease_allocate's preserve-daemon policy: the rationale — the daemon owns billed sessions; a reset orphans them all — applies verbatim. Each AWS ownership test proven red without the guard (3/3). * refactor: dedupe billed-resource cleanup and lease-signal wiring Shrink pass — same behavior, less duplication: - releaseOnFailure(primaryError, release) in webdriver-utils replaces the two identical 'best-effort stop the billed resource, attach cleanupError to the primary AppError' helpers (WebDriver session + AWS remote-access ARN); shared errorMessage too. - The lease handler pulls the request signal from getRequestSignal(requestId) like every sibling handler, instead of threading a requestSignal arg through LeaseHandlerArgs and the request-handler chain. Drops the field, the wiring, and five mechanical test edits; the handler test now proves the request-bound signal (abort it, watch the provider's signal flip) rather than arg identity. - Inlined the one-use requestHeaders back into fetchWebDriver. Handler-signal test proven red without the wiring. * fix(lease): the daemon releases a lease allocated for a gone requester; honest release evidence Review follow-up. The provider was doing the daemon's job: it treated the request signal as 'ownership evidence, not an interrupt' and needed three paragraphs to say so. The daemon owns the request, so it now decides — generically, for every provider — what happens to a lease that finished allocating after its requester left: release it (provider + registry) and answer with the canceled error. - lease.ts: after allocate returns, isRequestCanceled(requestId) → releaseAllocationForGoneRequester(). Release evidence is claimed ONLY on a clean release (no warnings, no throw); a WEBDRIVER_SESSION_DELETE_FAILED release is reported released:false with providerSessionId + a stop-by-hand hint (thymikee's finding: the previous evidence was success-shaped even when DELETE failed). - WebDriverSessionManager: the createOwnedSession/releaseCanceledSession trio is gone; allocate is plain 'create with a budget; on failure clean up' again. - LeaseLifecycleContext.signal is just cancellation, like everywhere else; the ownership-semantics comments on the contract, client, registry, AWS prepare and utils shrink to what the code no longer says itself. - Tests: the two provider-level cancellation tests move to the daemon handler (where the logic now lives), plus the failing-DELETE regression; both proven red without the post-allocate check. * fix(aws): the allocation deadline bounds remote-access startup, not the 120s default Live iOS real-device run: startup needed ~128s and hit the standalone 120s default while the daemon's 300s allocation budget still had room — the new ownership guard correctly stopped the ARN, but the open failed for no reason. When the daemon supplies a deadline it is the bound; the default only applies standalone. Rerun: open in 112s, snapshot, clean close, session STOPPING. * test(aws): pin that the allocation deadline outlives the 120s startup default; drop empty import Review follow-ups on 7f9d1481a: a virtual-clock test (Date.now advanced 10s per poll, RUNNING at 150s, deadline 300s) that fails on the old min(default, deadline) logic and passes now; and the empty 'import {} from kernel/errors' left in maestro/shared.ts is removed. * refactor: finish the dedupe — one release path, kernel errorMessage, AWS on releaseOnFailure Code-quality review at 7f9d1481a: 1. aws-device-farm.ts still carried its own copy of releaseOnFailure (the dedupe commit's script aborted before reaching it and I mis-verified). Now uses the shared helper; private copy deleted. 2. Empty 'import {} from kernel/errors' in maestro/shared.ts removed (273870099). 3. errorMessage() lives in @agent-device/kernel/errors; the two copies this PR had added (lease.ts, webdriver-utils.ts) import it. Sweeping the pre-existing copies is a follow-up. 4. lease.ts has ONE release path: releaseLease(registry, provider, lease, request, ctx) → { released (registry), provider } used by both the lease_release case (wire shape unchanged) and the gone-requester branch, which folds a throwing provider release into releaseError. 'released' now means the same thing in both; the provider verdict is a separate 'providerReleased' (warnings-free, no throw) that drives the stop-by-hand hint. -~35 lines. 5. sessionCreateTimeoutMs is Omit-ed at the WebDriverTransportOptions boundary instead of Pick-ed back out internally. * fix(lease): 'released' on a canceled allocation means the billed session is confirmed gone Re-review at 3665ea06: unifying the release path had made the cancellation error report released:true from the daemon's registry record while the provider DELETE had failed — success-shaped again, with the operator verdict demoted to a second key. Fixed at the source of the ambiguity: - LeaseReleaseOutcome names its bookkeeping field registryReleased. - On the canceled error, 'released' is true only when registryReleased AND the provider released without warnings AND without throwing; the registry record is exposed as 'registryReleased'. The stop-by-hand hint keys on 'released'. - lease_release keeps its existing wire field ('released' = registry; provider cleanup rides in 'provider'), unchanged. - Regressions: failed DELETE and throwing release both pin released:false / registryReleased:true (+ providerSessionId, warnings|releaseError, hint); both proven red on registry-only semantics. * ci: retrigger default-setup CodeQL Run 32051017472 is wedged on GitHub's side: status=completed with Analyze (python) still queued and Analyze (java-kotlin) failed only at SARIF upload (503, 'No server is currently available'). It can be neither cancelled nor rerun, and default-setup CodeQL has no dispatchable workflow, so a new push is the only way to get a fresh run. No source change. * test(webdriver): assert the typed timeout contract on the shared-budget probe main's #1790 tightened this test to expect the raw TimeoutError DOMException, which this PR intentionally normalizes into AppError{reason: webdriver_request_timeout}. On the merge ref the two met and Coverage went red. The regression now asserts the structured contract and that the second request's budget is the shared remainder (~118ms of 200 after an 80ms first call). |
||
|
Michał Pierzchała
|
60f6356b04 |
fix: read replay scripts on the caller and ship them with the request (#1810)
* fix: read replay scripts on the caller and ship them with the request Closes #1802 * test: assert the caller-side replay path as a substring, not a hand-escaped regex * perf(cli): load the Maestro engine only when a replay entry is a flow The command registry evaluates every command family on CLI startup, so the replay script-source builder's static @agent-device/maestro import put the YAML parser on the --help path. It now loads on demand behind the format check, and the startup import-closure guard covers the engine the way it already covers node:http. * refactor: share the replay request field vocabulary across the CLI and client views The new replay script-source flags appear in both CliFlags and CommandExecutionOptions, which fallow flagged as a clone; ReplayRequestFields declares them once. The test-suite handler's missing-sources rejection now travels the typed-error path its sibling rejections already use, so the fix adds no branch to handleSessionReplayCommands. |
||
|
Michał Pierzchała
|
f843dc2df1 |
fix(scroll): keep saturated scroll gestures out of the status bar; gate Android replays from android/emulator (#1781 A1) (#1820)
* fix(scroll): keep saturated scroll gestures out of the status bar; gate Android replays from android/emulator (#1781 A1) `pnpm gate replay-android` failed 4/8 whenever it ran after the full-tier Android E2E (replays-nightly run 32107665052, job 95620294899): 05-app-lifecycle, 06-swipe-gestures and both fixture replays diverged under "A system surface covers the app". The E2E was not the cause. Reproduced on a pixel_7 / API 36 AVD with the same cutout geometry CI's `avdmanager --device pixel_7` produces (status bar 136px, not the 63px of a plain 1080x2400 skin): - `03-scroll-discovery.ad` runs `scroll up 3`. The scroll planner clamps travel to the viewport minus a 5% band, so the touch-down landed at y=120 — inside the 136px status bar — and pulled the notification shade instead of scrolling. On API 36 the app window is edge-to-edge, so the reported viewport starts at y=0 and includes that bar. - The shade then covered every replay until `04`'s `back` closed it. Native readdir order on the runner (03, 05, 06, fixture/02, fixture/01, 04, 01, 02) put four files in that window; the last green run (2026-07-30) had 04 right after 03, so the pull was masked. Fix in the product, not the lane: DEFAULT_EDGE_PADDING_FRACTION 0.05 -> 0.1 in the TS scroll planner and its Swift port. Every real Pixel has a cutout (5.7% of a Pixel 7's height) and an iPhone's Dynamic Island status bar is 6.9%, so any saturated `scroll up` opened the shade / Notification Center for real agents too. Parity vectors updated in both suites plus a Pixel 7 regression vector (1080x2400, amount 3 -> touch-down y=240 > 136). Second contamination the same order exposed once the shade was gone: `fixture/02-selector-routes-covered-diagnosis.ad` is a #1715 reproduction recipe that FAILS BY DESIGN at step 9 (covered-target refusal) and leaves the device in landscape, yet the gate enumerated `test/integration/replays/android` recursively. iOS keeps gate replays in `replays/ios/simulator` and fixture recipes in `replays/ios/fixture`; Android now mirrors that: the six Settings replays move to `replays/android/emulator`, `test:replay:android` points there, and `fixture/` stays E2E-owned (`full:fixture-replays` already runs 01 by path). android.yml and the workflow-evidence fixture follow the path; the replay-compat manifest keeps the historical paths it pins at released tags. Verified live (Pixel 7 geometry, API 36, --retries 0): control run at main head in CI order reproduces exactly CI's 4/8; with the fix, `pnpm gate replay-android` 6/6 in both native and CI order, and `03` leaves Settings on screen (scroll up 3 now touches down at y=240). * test(scroll): drive the TS and Swift scroll-plan parity vectors from one table (#1820 review) The two suites hand-mirrored the same vectors and #1820 had to edit both by hand — the drift class the repo already closes for the tap-point rule via contracts/fixtures/tap-point-policy.json. The scroll vectors (plus both planner constants, pinned behaviourally on a 1000px axis) now live in contracts/fixtures/scroll-gesture.json; scroll-gesture.test.ts and RunnerTests+ScrollGesture.swift iterate it. Verified: vitest 10/10; the four XCTests run on an iOS 26.2 simulator with the unit flag on (Executed 4 tests, 0 failures). Also: test/ci/android-workflow-evidence.json says what it guards. Follow-up for content-safe viewport bounds + discovery order: #1821. |
||
|
Michał Pierzchała
|
8db36299e4 |
feat(web): add hover command for hover-gated UI (#1783) (#1786)
* feat(web): add hover command for hover-gated UI (#1783) Add a first-class `hover <x y|@ref|selector> [--settle]` verb, admitted on web only, that moves the pointer without pressing via the agent-browser backend (mouse move). It rides the existing targeted-touch pipeline (ref/selector/coordinate resolution, occlusion/off-screen guards, settle observation, response builder, recording) through a new optional Interactor/backend `hover` op that only the web provider implements. Touch platforms have no hover state: capabilities advertise it on web only and iOS/Android/Linux reject it at admission with a --platform web hint; longpress stays the mobile hold-gesture verb. Closes #1783 * fix(hover): native hoverRef route for web @ref, android coverage pin, revert skill edit Review follow-ups on #1786: - hover @ref on web now dispatches through the provider's own element handle (agent-browser `hover <ref>`) via a new backend hoverTarget, mirroring click/fill's ADR 0011 native-ref path — web ref frames carry no rects, so the coordinate route could never resolve them. The shared preflight + exact-ref dispatch is extracted into dispatchNativeRefInteraction and used by tap/fill/hover; the guarantee matrix native-ref row now lists hover. - Daemon regression test is production-faithful: rect-less web ref frame, scoped provider, asserts no coordinate dispatch. Selector→coordinate and provider hoverRef tests added. - Android emulator coverage summary pin 2/53 → 3/54. - skills/agent-device/SKILL.md reverted (out of scope, AGENTS.md rule). - Docs/help disclose that --settle with @ref on web shares click's existing limitation; use a selector or coordinates for the settled diff. * test: drive hover through the apple output guard; cover direct hover dispatch The provider-integration apple-leak guard partitions every public command into driven/skipped; hover was neither, which failed Integration Tests and took Coverage down with it. Drive it (it reaches the Apple capability refusal, which is scanned like any other error response). Also cover the direct-dispatch handleHoverCommand seam. * test(web): drive hover @ref in the provider-backed web scenario The integration-progress gate requires every public command to be referenced by a provider-backed scenario. Add hover @ref to the web desktop flow: it must reach the provider's hoverRef handle (never a coordinate) and be recorded on the session without fabricated x/y, like click @ref. |
||
|
Michał Pierzchała
|
0d3b7413c5 |
fix: prevent private AX subtree leaks at source (#1807)
* fix: prevent private AX subtree leaks at source * fix: preserve values in settle signals * fix: normalize settle signal semantics |
||
|
Michał Pierzchała
|
f378050586 |
feat(snapshot): attach a fallback screenshot to sparse captures (#1764)
A sparse verdict already tells the caller to use a screenshot as visual truth, which made that screenshot the guaranteed next command on every unreadable screen — a second round trip to obey advice we authored. The user-facing `snapshot` dispatch now takes the shot itself and links the path in its warnings. The fallback is deliberately hung off `dispatchSnapshotViaRuntime` and skipped for internal observations: selector resolution, settle, and wait polling reach `captureSnapshot` directly, so a wait polling an unreadable screen cannot turn into a screenshot per poll. A failed shot is swallowed — the verdict's own warning still carries the manual remedy, so the fallback can never fail the snapshot that was asked for. Sparse captures also say when the screen is the app's problem. Only the `sparse-tree` reason code is evidence about the app: every backend reached the screen and it published no semantic content, which is the same emptiness assistive tech gets. `ax-rejected`, `budget`, `no-nodes` and `capture-failed` are limits of this tool and stay unattributed, so readers are not sent to file bugs against code that is not broken. |
||
|
Michał Pierzchała
|
d8a7d03faf |
refactor: route application lifecycle through runtime facts (#1759)
* refactor: route application lifecycle through runtime facts Moves the canonical `open`, `prepare`, `close` and internal `runtime` descriptors behind package-owned lifecycle bindings admitted from device runtime facts, while daemon request/session policy and public response construction stay put. Based on main, which already carries the boot unit, the parametrized cutover gate and the apps unit. Readiness is package-owned there, so the Apple and Android bindings call ensureAppleReady/ensureAndroidReady rather than a root readiness bag; ensureAppleReady gained an onColdBootStart hook so open keeps warming the runner cache in parallel with a cold boot, and a narrow markBooted port publishes readiness' fresh observation so a flow still makes one simctl listing. Cutover rows take R24-R27, clear of the accepted catalog and the sibling install stack, and cutoverTableDefects rejects a duplicate rule id. Two defects this unit introduced are fixed here rather than shipped: `open <app> <url>` dropped the URL on a first open, and test-IME activation was first fatal on an unobtainable helper and then over-caught. Helper unavailability is a typed non-activation outcome now; fence, lock and post-record failures propagate. The duplication the unit had accumulated is gone: one runtime-admission module instead of five per-command copies, one direct-lifecycle binding factory instead of six hand-rolled packages, one transport-hint predicate, one session finalization path, and no identity-wrapper module. * fix: allocate lifecycle cutover rows after deployment * chore: preserve lifecycle union reconstruction * fix: reconcile lifecycle runtime stack * refactor: tighten lifecycle runtime topology * refactor: remove superseded runtime adapters * fix: preserve stacked runtime cutovers * test: preserve migrated runtime ownership * test: move Android deployment retry ownership * test: extract runtime hint fixtures * fix: preserve lifecycle stack invariants * fix: complete lifecycle runtime cutover * fix: remove lifecycle cutover residue |
||
|
Michał Pierzchała
|
66cca1a5b8 |
refactor: route install commands through platform runtime (#1758)
* refactor: route install commands through platform runtime * fix: preserve stacked runtime facts * fix: align deployment facts with shutdown runtime * refactor: simplify capability facts projection * style: format capability facts projection * fix: preserve migrated capability ownership * fix: propagate deployment artifact cancellation * refactor: move Harmony deployment mechanics into package * refactor: move Apple deployment tools into package * refactor: move Android deployment tools into package * refactor: inject deployment temporary storage * refactor: remove superseded deployment helpers * fix: preserve provider deployment transport |
||
|
Michał Pierzchała
|
5855dfc2e0 |
refactor: route shutdown through device runtime (#1757)
* refactor: route shutdown through device runtime * fix: cover shutdown cutover review gaps * fix: propagate shutdown cancellation * fix: move shutdown mechanics to platform owners * fix: pass device to shutdown fact fixture * test: cover shutdown facts in session state fixtures * test: simplify Android shutdown assertions * fix: preserve Apple shutdown cancellation |
||
|
Michał Pierzchała
|
39cd4d346a |
refactor: route appstate through platform runtime (#1755)
* refactor: route appstate through platform runtime * test: keep appstate capability fixture below complexity limit * test: cover appstate required readiness fact * fix: align appstate facts with boot readiness * fix: keep appstate use declaration minimal * fix: close appstate parity and ownership gaps * docs: record final appstate size accounting * fix: merge neutral runtime imports * docs: align final appstate size totals * fix: remove stale runtime dependency edges * docs: correct appstate size accounting * refactor: keep runtime-use factory internal * docs: itemize runtime-use relocation * fix: move appstate queries into runtime packages * refactor: retire root foreground query paths * refactor: share Android foreground parser ownership * fix: preserve Android appstate parser precedence * docs: keep appstate evidence in review artifacts * fix: keep appstate runtime loading lazy * fix: fail closed for stale limrun appstate * fix: preserve limrun recovery and abort appstate * fix: narrow limrun exact-owner recovery * fix: allocate appstate cutover rule * fix: reconcile appstate with merged main * style: format harmony runtime test * fix: allocate appstate rule id * fix: allocate appstate layering rule * fix: remove stale app command admissions * fix: close appstate layering regressions * fix: align Harmony capability parity with runtime facts * test: cover limrun recovery-only readiness * fix: keep Limrun recovery binding app-log only * fix: parse Android app state in linear time |
||
|
Michał Pierzchała
|
c7565cb1f8 |
refactor(snapshot): clean snapshot ownership (#1754)
* refactor(snapshot): clean snapshot ownership * fix(snapshot): address ownership review feedback |
||
|
Michał Pierzchała
|
eabc936a0f |
refactor: route apps through request runtime (#1756)
* refactor: route apps through request runtime * test: remove stale apps adapter mock * fix: clean apps runtime replay artifacts * fix: remove stale runtime test exports * refactor: simplify apps runtime admission * test: exercise runtime use through facade * fix: close apps runtime admission gaps * fix: disambiguate doctor app inventory callback * fix: keep capability fixture below complexity limit * fix: keep HarmonyOS app inventory fail-closed * test: type HarmonyOS app admission fixture * fix: restore HarmonyOS app inventory parity * test: align HarmonyOS readiness fixture * fix: preserve HarmonyOS doctor app parity * test: type HarmonyOS doctor fixture * fix: isolate HarmonyOS doctor policy * fix: align apps cutover with shared rule catalog |
||
|
Michał Pierzchała
|
b13c06338e |
refactor: route boot through readiness runtime (#1747)
* refactor: route boot through readiness runtime * fix: separate boot admission from readiness * fix: register boot cutover policy * refactor(runtime): move readiness into platform owners * fix(test): tolerate provider temp cleanup races |
||
|
Michał Pierzchała
|
97c87eec4d |
refactor(registry): exhaustive platformExecution discriminator (ADR 0019 §6) (#1740)
* refactor(registry): make the platform-execution discriminator exhaustive
ADR 0019 §6 (amended): every command descriptor declares its platform-execution
mode explicitly. Adds the `none` mode to `CommandPlatformExecution`, removes the
silent `{ kind: 'legacy' }` default at registry entry, and annotates all 76
descriptors so the migration denominator is machine-readable.
Part of #1739 (wave 0)
* fix(registry): react-devtools executes delegated platform behavior
`react-devtools start` on a Limrun Android instance dispatches internal
`runtime port-reverse`, which reaches a provider device runtime, so ADR 0019 §6
`none` is false for it. Reclassify as `legacy` and add the derived coherence gate
that catches delegated platform execution: if a CLI route for command R
dispatches command D, R may declare `none` only when D is `none`.
Part of #1739 (wave 0)
* fix(registry): attribute CLI dispatches by occurrence, not command name
Subtracting attributed command NAMES let a stray dispatch hide behind a routed
one that names the same command, so the gate's totality claim did not hold.
Dispatch sites now carry their source offset and attribution subtracts
occurrences.
Part of #1739 (wave 0)
* fix(registry): unresolvable CLI daemon-send targets fail the gate
An unknown literal or computed command target resolved to undefined and never
entered the scan, so a dispatch could evade attribution by naming a target the
gate could not read. Daemon-send envelopes are now located by their send call and
an unresolvable target is reported instead of skipped.
Part of #1739 (wave 0)
* refactor(cli): own injected daemon dispatches at a typed construction seam
The syntactic scan recognized only a direct sendToDaemon call whose first
argument was an inline object literal, so a variable envelope or a computed
callee was omitted from every result. Rather than teach the scanner more shapes,
the CLI's injected dispatches now flow through one typed construction point whose
route/command pairs are declared, and the gate reads that declaration instead of
recovering it from syntax.
Part of #1739 (wave 0)
* fix: constrain injected daemon transport handoffs
|
||
|
Michał Pierzchała
|
74eab2a554 |
refactor: route selector-resolution structural stages into typed policy (#1744)
* refactor: route selector structural stages into typed policy #1649 landed the per-caller ambiguity matrix and deliberately left four structural columns out: occlusion, off-screen, hittable-ancestor promotion, and the poll budget were per-caller pipeline code, so declaring them would have been an unverifiable claim (nothing consumed them; flipping one left the suite green). This adds the missing half as a table with runners. `SELECTOR_PIPELINE_POLICIES` (src/core/selector-pipeline-policy.ts) gives each caller ONE row naming its ambiguity contract plus its four stages, and every stage is reached only through a runner that reads the row: - occlusion -> selectorPipelineCandidates (candidacy) and resolveSelectorPipelineTarget (refusal). Acting rows exclude covered nodes and refuse covered targets; `find` and the diagnosis probe keep them as candidates and refuse at the target; reads and `wait` ignore them. - promotion -> resolveSelectorPipelineTarget. The per-call-site `promoteToHittableAncestor: boolean` is gone: click/press/longpress name `promotedTarget`, fill/focus/scroll/drag endpoints and the native-ref preflight name `resolvedTarget`. `find`'s below-the-root variant is a declared value rather than a second local helper. - off-screen -> throwIfOffscreenInteractionTarget, which now takes the row and returns the node untouched (no iOS rescue round trip) for observation rows. - poll -> selectorPollBudget, which createWaitPolling derives its deadline and inter-poll delay from; the two wait loops carry a budget, every other row carries none and cannot be polled. Behavior is byte-identical. The acting refusal keeps its exact node, label and details in every branch (promotion declines to retarget away from a covered node, so the "both covered" case names the same node it always did), and `find` carries the occlusion verdict to the focus/type seam rather than raising it early, because find click/fill still delegate that refusal to the interaction leaf's own error shape. selector-pipeline-policy.test.ts drives EVERY row through EVERY runner, including the rows whose answer is "skip" — the half that used to be an absence of code, and an absence cannot fail. Each stage was proven red by flipping its cell (occlusion, promotion, off-screen, poll, plus the declare-only-what-is-enforced guard). The ADR 0011 occlusion/nonHittable `via` pointers for the runtime tree paths now name the runner that makes the decision, not the predicate it applies. Closes #1656; prework for #1739 (waves 4-5). * docs: state constraints instead of narrating the refactor Comment pass over #1656: drop the "used to be per-caller code" / "not module constants" / "rather than an omission" narration — a comment should say what a future edit must respect, not what the previous shape was — and compress the find occlusion-verdict and poll-budget notes to the constraint they actually carry. * refactor: make the selector pipeline the only door to the engine Review of #1744: the structural rows were declared but bypassable. Read and wait routes composed `selectorPipelineCandidates(row, nodes)` with the raw `resolveSelectorChainWithPolicy(..., row.resolution)` and never entered the promotion or off-screen stages, so flipping a read row's `promotion` or `offscreen` changed only the policy unit tests — production `get`/`is`/`wait` were unaffected, which is the unverifiable-column failure #1656 exists to remove. Callers could also pair one row's candidate set with another row's ambiguity contract, and `find list` reached the engine directly. The owning interface (src/core/selector-pipeline.ts) now runs every stage a row declares, skips included, and the stage functions are private to it: - `resolveSelectorPipeline` — single-target rows: candidacy, ambiguity, the replay-guard hook, promotion, occlusion, off-screen. - `listSelectorPipelineMatches` — `reject-candidates` rows, returning the candidate set AND the tree the row sees, so ranking and equivalence classification judge the same nodes candidacy produced. - `runNodePipelineStages` — the node stages for a target from a non-chain matcher (`@ref`, find's fuzzy locator) or a narrowed candidate set. A row whose off-screen stage refuses must supply a refusal shape, so flipping an observation row to `refuse` fails on its real route instead of silently observing. `find list` now names a `readList` row (the new `reject-candidates`/no-rect ambiguity row) instead of calling the engine. R17 selector-pipeline-ownership (scripts/layering/) makes the bypass structurally inexpressible: only the owner may import the engine entry points. Proven against a planted import in selector-read.ts, which the repo-wide scan rejects with the entry points that replace it. Flips now fail through REAL command routes, verified one at a time: readUnique.occlusion/offscreen/promotion and wait.occlusion via get attrs / is / wait; readAny.offscreen via is exists and find; readList.occlusion via find list; promotedTarget.promotion via runtime click. The wait route test needed an advancing clock first — with the frozen one a refused wait spun instead of failing, so the flip hung rather than asserting. * refactor: drop find's dead candidate binding The selector branch bound the row's candidate set and never read it: only the acting classification needs that tree, and find's locator branch brings its own matcher. Names what actually governs the locator target — the shared node stages below, not a candidate set it never had. * refactor: reserve the selector engine behind the pipeline owner Review of #1744 (three blockers). **Listing rows no longer claim stages they cannot run.** `find <q> list` resolves to a candidate SET, so promotion, the off-screen guard and a poll budget have nothing to apply to — a listing has no single element to retarget, keep on screen, or wait for. `readList` now declares only the two stages a listing executes (`SelectorListPolicy`: resolution + occlusion), and the narrower shape is load-bearing: `runNodePipelineStages` and `selectorPollBudget` take the full row, so handing them a listing row is a compile error rather than a silently skipped stage. Pinned with `@ts-expect-error` — widening `readList` makes the directives unused and fails the typecheck. **The engine door is a specifier, not a symbol.** R17's regex could not see a namespace import, a re-export, or a deferred `import()`, none of which mention the symbol it matched. The two engine entries moved to `@agent-device/selectors/engine`, and R19 enforces over the resolved import graph, where every one of those forms is the same edge. Proven on the repo-wide scan by planting each form into a shipped route: namespace import, dynamic import, and `export *` laundering all come back red. `resolveImportEdges` drops an edge whose specifier resolves to nothing, so a specifier rule goes quiet — not red — if the subpath is ever retired. The gate now says that out loud instead of scanning clean. **R19, not R17.** #1750 allocates R17/R18. Verified free against origin/main and that PR's diff, then validated by real merges in both directions: the uniqueness gate passes either way and the three ids stay distinct. The gate itself is new (`scripts/layering/rule-ids.ts`): two branches taking one free number do not conflict in git, so nothing caught R17 twice. Matching whole string literals is what separates a declaration from prose that names a rule, and it is what let the gate see #1750's `const RULE = '…'` shape — the first version missed it and would have been vacuous. `main`'s two pre-existing collisions (R11, R13) are listed as known, not pinned by equality, so #1750 lands in either order without breaking this. Also: the root façade now exposes no resolver at all, and its surface test pins both doors. * fix(layering): make each rule-id allowance expire with its collision Review of #1744: `KNOWN_RULE_ID_COLLISIONS` filtered the exact R11/R13 collision strings, so once #1750 renames those rules apart the entries would keep waving those very collisions through if anyone reintroduced them. "Inert" was wrong — a stale allowance fails open, permanently. `ruleIdCollisionFailures` now checks the transition from both sides: a collision nobody allowed fails, AND an allowance whose collision is absent from the scan fails as a stale allowance. The entry therefore has to be deleted in the same change that removes the collision, and the list burns down to empty, which admits nothing. #1750 is still open, so the transitional entries stay for now (option (b)). Verified against a scratch tree carrying that PR's rename: leaving the list untouched reports both entries as stale; deleting them is clean; and reintroducing `R11 names contracts-implementation-authority and package-boundaries` afterwards is rejected. The last of those is also a unit regression, so the post-transition guarantee is pinned rather than argued. |
||
|
Michał Pierzchała
|
f18f8b076f |
refactor(contracts): consolidate per-domain defineUse wrappers into one neutral defineUse (#1741)
* refactor(contracts): consolidate per-domain defineUse wrappers into one neutral defineUse
ADR 0019 §9: use declarations share one neutral defineUse; per-domain
currying wrappers around runtimeUse<PlatformRuntimeOperations>() add a
module per domain for no information. network-runtime-plan.ts,
logs-runtime-plan.ts, screen-recording-runtime-plan.ts, and
app-log-resource-recovery.ts each re-derived their own curried alias
(defineNetworkUse, appLogUse, defineScreenRecordingUse, and an inline
instantiation) from the same generic factory with the same type
parameter.
Export defineUse = runtimeUse<PlatformRuntimeOperations>() once from
platform-runtime.ts (where runtimeUse lives) and re-export it from the
platform facade. Every runtime-use declaration (networkDumpUse,
networkAdmissionUse, the app-log uses, the screen-recording uses, and
appLogRecoveryUse) now builds through that single export; the
per-domain wrappers are deleted.
Type-level only: no required/preferred keys changed, and no use
declaration was added, removed, or altered. Existing deepEqual
assertions in network-runtime-plan.test.ts, logs-runtime-plan.test.ts,
and screen-recording-runtime-plan.test.ts already pin every produced
use object's exact {required, preferred} shape, so they double as the
before/after regression proof that this refactor is behavior-neutral.
Part of #1739 (wave 0).
* fix(contracts): move defineUse into platform-runtime-operations.ts
Review: defining defineUse in platform-runtime.ts required importing the
concrete PlatformRuntimeOperations catalog into the generic runtimeUse
primitive module, while platform-runtime-operations.ts already imports
generic runtime types from platform-runtime.ts. That's an avoidable reverse
type dependency — the lower generic primitive depended on its concrete
aggregate catalog. The unchanged SCC file count didn't prove this harmless;
it counts cycle members, not newly introduced back-edges.
Move defineUse = runtimeUse<PlatformRuntimeOperations>() into
platform-runtime-operations.ts, alongside PlatformRuntimeOperations.
platform-runtime.ts no longer imports the concrete catalog. Re-export
defineUse through the platform facade from its new source module; every
call site keeps importing it from @agent-device/contracts/platform
unchanged, and the three contracts-internal call sites now import it
directly from platform-runtime-operations.ts.
Validation: tsc (full workspace + examples/sdk), check:layering (136/136,
type-cycle count unchanged at 46), and check:affected --run (473 files /
3939 tests) all clean at the new head.
|
||
|
Michał Pierzchała
|
602b7a2995 |
refactor: narrow perf API to actionable evidence (#1731)
* refactor: narrow perf API to actionable evidence * fix: address perf API review feedback * fix: preserve deprecated Android CPU metrics |
||
|
Michał Pierzchała
|
1b2e786128 | refactor: move screen recording onto platform runtime (#1724) | ||
|
Michał Pierzchała
|
cdc754e6ed |
perf: speed up iOS agent recovery and streamline CLI guidance (#1700)
* Avoid interactive children in parent taps * docs: streamline no-skill CLI help * perf: recover faster from sparse iOS trees * fix: preserve selector context for blocked parent taps * fix: preserve coordinate text-entry focus * fix: preserve thin parent touch targets * fix: fail closed for unscoped iOS typing * test: isolate replay lock fixture * test: share node integration process |
||
|
Michał Pierzchała
|
b15c502318 |
refactor: extract platform network runtime (#1702)
* refactor: extract platform network runtime * fix: preserve platform network recovery routes * test: guard network parser placement |
||
|
Michał Pierzchała
|
b1ed5353d1 |
refactor: extract platform log runtime (#1701)
* refactor: extract platform log runtime * fix: clear terminal app log recovery markers * fix: preserve scoped app log tooling * fix: preserve app log cancellation * fix: handle large changed coverage diffs * fix: harden Limrun runtime identity * refactor: tighten platform log runtime * fix: close app log trust gaps * fix: accept canonical session path aliases * refactor: extract durable capture kit * fix: refresh retained log marker admission * fix: rotate app logs after process relaunch |
||
|
Michał Pierzchała
|
13cc90ffc6 |
fix: harden Android snapshot and fill reliability (#1708)
* fix: harden Android automation reliability * test: isolate CLI flush integration * test: close Android review gaps * test: register CLI transport fixture * test: consolidate CLI subprocess fixture |
||
|
Michał Pierzchała
|
c06bed9f77 |
refactor: extract platform device inventory runtime (#1699)
* refactor: extract platform inventory runtime * fix: preserve scoped Apple inventory tooling * fix: preserve Apple tool cancellation * refactor: tighten platform inventory boundaries |
||
|
vw2x
|
4b432fb59b |
feat: add HarmonyOS support (#1683)
* feat: add HarmonyOS device automation foundation Add HDC-backed discovery, snapshots, application lifecycle, and core mobile interactions. Route HarmonyOS through the platform registry and client contracts. Cover parsing and capability parity with focused tests. * feat: support HarmonyOS HAP deployment Install and reinstall signed HAP archives through HDC. Resolve bundle identities from module metadata and relaunch after package replacement. Extend deploy routing and capability coverage for HarmonyOS. * feat: add HarmonyOS single-pointer gestures Execute pan, fling, and swipe plans through HDC uiInput primitives. Derive scroll coordinates from the live ArkUI viewport. Keep unsupported multi-touch gestures explicitly rejected. * refactor: split session inventory command handling Separate session, device, capability, and app inventory response paths. Preserve the public inventory response contract while reducing handler complexity. * feat: support HarmonyOS keyboard actions Route HarmonyOS enter, return, and dismiss through HDC key events. Expose supported keyboard actions through the system command metadata. Keep keyboard visibility inspection explicitly unsupported. * fix: reject unsupported HarmonyOS drag gestures Keep drag unavailable until HDC can preserve source and destination hold semantics. * feat: add HarmonyOS app log streaming 1. Stream HarmonyOS app logs through PID-scoped hilog sessions.\n2. Record HarmonyOS app identity during bundle-id opens for app-scoped commands.\n3. Cover backend routing and bundle identity resolution. * feat: report HarmonyOS foreground app state 1. Read the foreground HarmonyOS mission through aa dump.\n2. Expose HarmonyOS appstate with package and ability metadata.\n3. Add parser coverage for foreground and missing-state cases. * fix: advertise appstate through capabilities 1. Classify appstate in the command descriptor capability matrix.\n2. Surface supported appstate commands in capability inventory.\n3. Cover the advertised Android capability contract. * feat: sample HarmonyOS process performance 1. Sample HarmonyOS process CPU and resident memory through HDC.\n2. Expose the verified metrics through the shared perf command.\n3. Keep frame and memory snapshot collection explicitly unavailable. * feat: clear HarmonyOS app state 1. Add HarmonyOS settings clear-app-state through bundle cleanup.\n2. Force stop the app before clearing data and cache.\n3. Reject all unverified HarmonyOS settings explicitly. * docs: document HarmonyOS support 1. Describe HarmonyOS HDC prerequisites and HAP installation.\n2. Add HarmonyOS to platform discovery and product documentation.\n3. Document verified performance limits for the public HDC surface. * fix: preserve HarmonyOS deploy session identity 1. Bind a resolved HarmonyOS bundle after install or reinstall.\n2. Keep app-scoped logs and observability available after deployment.\n3. Cover session identity preservation for HarmonyOS reinstall. * test: lock HarmonyOS capability boundary 1. Add an independent HarmonyOS capability-matrix oracle and exact advertised-command regression test. 2. Document current HDC-backed support and evidence-based unsupported command boundaries. * refactor: simplify HarmonyOS shared platform boundaries 1. Split device selection and settings dispatch into focused helpers without changing behavior. 2. Keep HarmonyOS serial selection and lock-policy classification covered by regression tests. 3. Remove Fallow complexity findings from the HarmonyOS diff against upstream main. * fix: bound default HarmonyOS HDC commands 1. Apply a 15 second timeout to ordinary HDC operations. 2. Preserve operation-specific timeout budgets for installation and capture paths. 3. Add regression coverage for default and overridden HDC timeouts. * feat: add HarmonyOS screen recording Implement physical-device whole-screen recording through the system recorder and HDC media transfer. Reject unsupported HarmonyOS recording scopes and export flags. Cover capability routing, media retrieval, cleanup, and simulator rejection. * feat: report HarmonyOS HDC readiness Add an HDC version check to the HarmonyOS doctor flow. Document HarmonyOS as a supported doctor platform and cover the result. * refactor: simplify HarmonyOS recording checks Reduce recording validation and test complexity without changing behavior. * test: cover HarmonyOS platform contracts Synchronize public platform expectations across CLI, MCP, replay, and inventory tests. Mock HarmonyOS inventory probes to preserve concurrent test behavior. * test: model HarmonyOS recording capability Require a physical HarmonyOS device in the independent capability parity oracle. * test: cover HarmonyOS input and lifecycle paths Exercise HDC input, lifecycle, installation, and relaunch command sequences. * test: cover HarmonyOS device observability paths Exercise discovery, screenshot validation, and process performance sampling. * docs: define HarmonyOS CI hardware policy Keep HDC hardware validation local and require mocked CI contract tests. * fix: honor HarmonyOS app inventory filters * fix: bound HarmonyOS app inventory classification 1. 限制应用元数据分类并发并为默认清单设置整体时限. 2. 将请求取消信号传递给 HarmonyOS 应用清单读取. 3. 补充失败时中止在飞读取且不继续排队的回归测试. * fix: preserve HarmonyOS inventory failure causes 1. 保留触发应用元数据分类失败的原始错误, 避免被取消同级任务覆盖. 2. 补充总时限中止在飞读取且不启动排队任务的回归测试. 3. 验证后序任务失败时保留默认筛选的恢复提示. |
||
|
Michał Pierzchała
|
18291ba8e2 |
perf: collapse app-driving startup turns (#1693)
* perf: collapse app-driving startup turns * fix: align foreground open guidance |
||
|
Michał Pierzchała
|
6c0fcb64a1 |
fix: reject distinct ambiguous mutation targets (#1667)
* fix: reject distinct ambiguous mutation targets * fix(ios): scope the raw-match rejection to mutating dispatches `RunnerTests+Interaction.findElement` applied the new fail-closed classification to `querySelector` as well as press/type, because the read call site takes the default `allowNonHittableFallback: false`. With one visible/hittable match and one non-hittable same-selector duplicate the query started returning AMBIGUOUS_MATCH where it previously selected the hittable element, and `queryDirectIosSelectorOrFallback` preserves that error for read callers — so `get`, `is`, and `wait` surfaced an error instead of their prior answer. `classifyDirectSelectorCandidates` now takes a `rawMatchPolicy`. Mutations keep `.rejectDistinctMatches` (the default, so no mutation call site changes); `queryElement` passes `.preferHittableMatch`, restoring the prior read rule: prefer the single hittable match, ambiguous only when hittable matches compete, and never adopt the Maestro coordinate fallback. The Maestro expected-point path is untouched. Covers the one-hittable + one-non-hittable read, competing hittable reads, and the non-hittable-only read. ADR 0011's amendment now states the scope. * test(ios): execute selector read ambiguity regression --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
4b89482a06 |
fix(daemon): open --foreground P1 hotfix — selector rejection, interactive snapshot, capture-failure masking (#1671)
* fix(daemon): open --foreground P1 hotfix — selector rejection, interactive snapshot, capture-failure masking Three P1s from the post-merge review of #1670, all at the session-open-foreground dispatch seam: 1. Explicit device selectors were silently overwritten: the resolved-device rewrite pinned --udid/--platform over whatever the caller passed, so `open --foreground --udid B` with sim A sole-booted silently opened A. Now fails fast with INVALID_ARGS (matching the existing app-positional rejection) on --udid/--device, and on --platform other than ios; an explicit --platform ios passes through. 2. The promised interactive snapshot was never requested: the composed snapshot dispatch forwarded the open request's flags untouched, without snapshotInteractiveOnly — so the capture was NOT the `snapshot -i` path the doc comment promised and returned no interactive presentation. The composed request now sets snapshotInteractiveOnly: true (the exact key the CLI maps -i to and the snapshot runtime reads as interactiveOnly). 3. A capture failure masked the successful open: returning the snapshot error discarded openResponse even though the session exists, so a retry of `open --foreground` failed with "close the current session first". Open success + snapshot failure now returns ok with an explicit initialSnapshotError {code, message} detail and a rendered warning that the session IS open and how to capture manually (snapshot -i). Regressions added for all three: explicit-selector rejection (udid/device/both/non-iOS platform + ios pass-through), the composed dispatch carrying snapshotInteractiveOnly, and the snapshot-failure path returning ok + warning + usable session. * fix(cli): render the composed open --foreground snapshot on default stdout and project initialSnapshotError through the public surfaces Post-merge review on #1671 found the daemon fixes never reached the public boundaries: openCliOutput ignored the nested snapshot (the one-call promise held only under --json), and initialSnapshotError was daemon-only — absent from AppOpenResult, Node normalization, and serializeOpenResult, with the normalized shape truncated to code+message. - default open output now renders the composed interactive tree through the same snapshotCliOutput path snapshot -i uses - AppOpenResult carries initialSnapshotError as the FULL daemon error (hint/details/diagnosticId/logPath preserved) through normalization and serialization Worker-authored; committed by the coordinating session after the worker stalled twice mid-push. Tests: output.test.ts + session-open-foreground (26 pass), typecheck, oxfmt. * refactor(client): one daemon-error normalizer + client-route regressions for initialSnapshotError Review follow-ups on #1671: normalizeInitialSnapshotError duplicated the target-shutdown error normalization and pushed the module over the fallow complexity threshold; consolidated into a single internal normalizeDaemonError (table-driven, full shape incl. retriable/supportedOn) used by both result paths, projected through normalizeOpenForegroundComposition. Client-route regressions: createAgentDeviceClient().apps.open now proves the full initialSnapshotError shape (hint/details/diagnosticId/logPath/retriable) survives normalization, and that a malformed one is dropped — deleting the boundary normalization fails these tests. Also rebased onto current main. * fix(daemon): a thrown initial-snapshot capture failure gets the same successful-open contract Review P1 on #1671: dispatchSnapshotViaRuntime rethrows ordinary capture/runner exceptions; the composition only handled a returned { ok: false }, so a thrown failure escaped to the router and failed the whole open after the session was created — retrying then wedged on the existing session. The catch normalizes the rejection (kernel normalizeError, same conversion the router applies) into the shared openWithInitialSnapshotFailure path: ok response, full-shape initialSnapshotError, session-usable warning. Rejecting-mock regression added alongside the returned-failure case. |
||
|
Michał Pierzchała
|
13bc70f24f |
refactor(find): resolve a mutating find's target once, not twice (#1654) (#1669)
* fix(find): resolve a mutating find's target once, not twice (#1654) A mutating `find click`/`find fill` captured the screen, matched by locator under the `findAct` policy, promoted to a hittable ancestor, minted `@eN` off the node it chose — and then re-entered the interaction leaf by bare `@ref`, which looked that ref up AGAIN via resolveSnapshotForRef. The second lookup reads the SESSION frame tree, not the fresh capture find matched against, so the two could disagree: it could hand the action a different node than find picked, or refuse as unresolvable a ref find had observed a moment earlier. find now passes the node itself. `internal.findPreresolvedTarget` carries the resolved node and its tree over the in-process invoke hop, and the ref branch adopts them instead of resolving the ref a second time. What is NOT skipped: the guarantees. Occlusion, hittable-ancestor promotion, and the off-screen guard all still run, on that node, at the same symbols the ADR 0011 `runtime-ref` cells name. Only the LOOKUP is replaced. Recording, ref-frame effects, settle/observation, and deferred-outcome marking are untouched — they live in the dispatch wrapper, not in resolution, which is why the invoke hop is kept rather than bypassed the way find focus/type bypass it. The channel is a second field rather than widening `findResolvedTarget` because they answer different questions: that flag governs ref-frame admission and staleness (policy), this governs resolution (lookup), and focus/type set neither. It is `internal`-only, so it never crosses the wire and carrying live node references is sound. ADR 0011 re-check: the `runtime-ref` disclosure cell gains a second producer, adoptPreresolvedRefTarget, reporting `exact`. Truthful rather than borrowed — the ref is minted off the node handed over. It cannot report label-fallback, which is right: label recovery is a property of looking a stale ref up, and this path performs no lookup. Tests pin the behavior by tap coordinates, so they name which tree the leaf resolved against, plus a control proving the ordinary @ref path is unchanged and a case where the session tree cannot resolve the ref at all — that one can only pass if no second lookup runs. Verified revert-sensitive: removing the short-circuit fails 3 of the 4, and the control stays green. Out of scope, unchanged: the ADR 0011 `native-ref` path (web provider clickRef/fillRef only), where the ref is the provider's own element handle. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117wfrvC6MRDJUWdRBErNEF * test(find): prove the production route, and correct the divergence claim (#1654 review P2) The regression tests built `internal.findPreresolvedTarget` by hand and called handleInteractionCommands directly, so they proved the leaf consumes the channel but never that find ATTACHES it. Either producer could have stopped doing so and they would all have stayed green — the exact gap #1649's review named, reappearing one layer up. Four tests now drive the real handleFindCommands with invoke wired to the real handleInteractionCommands. Two assert click and fill each carry the selected node (separate producers, separate forwarding hops, so asserted separately); two advance the session tree between find's match and the leaf's resolution and assert the dispatched point is still find's node. That last pair also corrects the record. The claim that the leaf resolves against a different tree than find matched is too strong for the common path: `omitRefFrameSnapshot` (interaction-runtime.ts) makes find's internal dispatch skip the authorized frame tree and resolve against `session.snapshot`, which find's own capture just wrote — so the second lookup normally AGREES, and the two non-diverged tests above pass with or without the fix. The divergence is real but narrower: it needs something to advance `session.snapshot` between find's match and the leaf's resolution, which `refreshAndroidRefSnapshotIfFreshnessActive` does on the ref path. Measured at this head, the old code taps (60,720) "Delete" where find matched "Save" at (310,510) — a wrong-element mutation, now pinned by both click and fill. So the change's value is what #1654 asked for — one resolution end to end — plus closing that window, not a fix for a divergence occurring on every find. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117wfrvC6MRDJUWdRBErNEF * fix(find): tighten resolved target provenance --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
04c33f9e1e |
feat(ios): expose AX custom actions on merged accessibility elements (#1665)
* feat(ios): expose AX custom actions on merged accessibility elements
Apps that merge a card into one accessibility element for VoiceOver (React
Native's `accessible` prop) publish the card's real affordances as
UIAccessibilityCustomActions rather than as child elements. Our snapshot showed
only the merged node, so an agent looking for a feed card's options control had
nothing to aim at and fell back to coordinate guessing.
`snapshot --actions` now names them:
@e8 [link] "feedItem-by-whiskers.test" actions: ["Reply", "Repost", "Open post options menu"]
Opt-in, because the AX server cannot serve custom actions in a bulk tree
request: adding the attribute makes testmanagerd's reply decoder reject the
nested arrays a custom action serializes into, drop the reply, and time the
request out (~65s vs ~110ms). Only per-element reads answer, at ~100ms each, so
the runner reads at most 12 labelled childless nodes and stops at the
capture-plan deadline. The request pins the private-AX backend, since no other
backend can read the attribute, and reports that as its own `requested-backend`
verdict so a deliberate pin never renders as a degradation warning.
Invocation is not shipped: the actions are readable but not invocable from the
runner. RunnerAXSnapshotBridge.h records the five APIs that were tried.
* fix(ios): disclose a capped custom-action pass, and read on-screen elements first
Two gaps in the first cut.
An element the bounded pass never reached rendered identically to one with no
custom actions, so a capped capture silently taught the reader that later feed
cards have no affordances — the exact mis-inference this feature exists to
prevent. The runner already counted reads against candidates; it now carries
both to the response, and the verdict renders one response-level line when the
pass was incomplete. A complete pass stays silent, and "never asked" stays
distinguishable from "read none" (the key is absent, not (0, 0)).
The obvious remedy for a capped pass would be a scoped re-run, but scope is
applied when the Swift walk builds nodes, long after the read pass, so it does
not redirect the budget at all — measured: reads=12 candidates=18 with and
without --scope. Rather than print a remedy that does nothing, the read pass now
orders candidates on-screen first. That makes the budget land on elements an
agent can act on, and makes the disclosed remedy true: scrolling changes the
on-screen set, so a re-run reads elements the previous pass could not.
Also states plainly, in the flag help and the tool/SDK field description, that
the names are for planning: nothing invokes them, so the affordance is reached
through the element's detail screen, the same control exposed elsewhere, or
coordinates.
* test(snapshot): pin the custom-action coverage pair in the verdict shape assertion
* chore(scripts): classify the --actions flag in the integration progress model
The completeness gate flagged snapshotCustomActions as unclassified, which is
what it is for. It gets its own bucket rather than joining the provider-scenario
table: the values come from the private AX client inside the runner process, and
the fake runner derives its behavior from fixture tables that cannot fabricate
custom actions, so there is no provider-backed scenario to claim. The owning
coverage is named instead — runner XCTest unit, snapshot-lines, snapshot-quality.
* fix(ios): fail closed on unserviceable --actions, bound each read, cap output, and count actions in identity
Four review findings.
1. `--actions` with `--raw`, or on any target that is not an iOS simulator, used
to succeed and return nodes with no actions — a requested capability silently
no-opped, indistinguishable from "this screen has none". Both now fail closed.
The raw pairing is rejected at the shared request seam (INVALID_ARGS) so CLI,
Node client and MCP answer alike before any device work; the platform case is
rejected once the session device is resolved (UNSUPPORTED_OPERATION), naming the
resolved target. `diff --actions` was already rejected as an unsupported flag.
2. The per-element AX read had no timeout, so one wedged element could consume
the whole capture budget. Each read now runs off-thread behind a 1s wait. A
timed-out element counts as unread, never as "read, and it has no actions", so
the existing partial-pass disclosure already covers it.
3. The element budget bounded element count only; one element could still return
an unbounded list of unbounded names. Capped at 8 names of 80 characters, and
clipped elements are counted into the coverage so a truncated list is disclosed
rather than silently presented as complete.
4. Action names were rendered unescaped, and no comparison key read them. Names
now get the same escaping as text previews plus control-character folding, so an
app-authored name cannot split or corrupt a line. `actions` joins the diff
comparable key, the unchanged-comparison projection, and — the sharper bug —
the snapshot presentation key, without which `snapshot` followed by
`snapshot --actions` on a still screen answered "unchanged" and never delivered
the actions that were explicitly requested.
* fix(ios): contain a hung custom-action read instead of accumulating orphans
The 1s read deadline frees the caller, but the underlying AX call is a
synchronous XPC round trip that cannot be cancelled — it keeps running. On a
global concurrent queue that meant repeated `snapshot --actions` against a
wedged element piled up orphaned reads, all using the shared XCAXClient
concurrently. The deadline was containment for the capture, not for the runner.
Since the call cannot be cancelled, contain it instead:
- every read runs on one dedicated serial queue, so a wedged call can never be
joined by a second concurrent user of the shared client;
- a single-flight guard refuses to dispatch at all while an abandoned read is
still outstanding, so a repeated capture adds no work — the dispatch counter
stands still;
- the read pass stops at that point rather than paying a deadline per element
on reads that would all be refused, and reports `blocked` so the capture stays
honest. That gets its own line, because the partial-pass remedy (scroll and
re-run) cannot clear a hang and would send the reader in circles.
Recovery needs no reset: when the hung call finally returns, in-flight drops to
zero and reads resume.
The regression drives a fake AX client that never returns, and asserts the three
things the fix exists for — exactly one in-flight read with no further
dispatches across repeated captures, immediate returns with the skip disclosed
instead of the scroll remedy, and reads working again once the wedge clears.
* ci(ios): execute the custom-action runner regressions instead of only compiling them
The iOS workflow runs a targeted -only-testing list, so a runner test that is
not named there is compiled by the build step and then never executed. All seven
custom-action tests were in that gap — including the containment regression,
which is the only executable proof that a hung AX read cannot accumulate
orphaned in-flight reads.
Red/green against the containment regression, with the fix reverted to its
pre-fix concurrency behavior (global concurrent queue, no single-flight guard,
no blocked exit):
RED in-flight 6 (want 1), dispatches 6 (want 1), each repeat paid the full
1.004s deadline (want <0.2s), blocked=false (want true), and the
in-flight drain never completed — "Exceeded timeout of 5 seconds".
GREEN 7/7 pass, containment regression in 1.02s.
|
||
|
Michał Pierzchała
|
e14c9d8d7b |
fix(cloud): unblock iOS snapshot and gate cloud fill on text-entry focus (#1658) (#1666)
* fix(cloud): unblock iOS snapshot and gate cloud fill on text-entry focus (#1658)
Two bugs isolated to the cloud-webdriver iOS path.
`snapshot`/`diff` refused every capture on a live BrowserStack session with
SESSION_NOT_FOUND, instantly and without a driver round trip. The app-session
guard they ran belongs to the local XCUITest runner, which must attach to a
target app; a cloud capture reads the provider's own driver session and needs
no app identity, so it now applies to local Apple targets only. The session
was empty in the first place because the provider open path skips local app
resolution wholesale — no simctl/devicectl reaches a hosted device — and
dropped an explicitly spelled bundle id along with it. A dotted, non-deep-link
target is the bundle id under the same convention resolveIosApp applies
locally, so a cloud `open com.example.app` now records it.
`fill` tapped and sent its keys in back-to-back requests. A WebView input —
an OAuth page in a Safari view controller — takes first responder
asynchronously, so the keys landed with nothing focused while the command
still answered "Filled N chars"; tapping and filling as two separate commands
worked only because the round trip between them gave the field time to focus.
The cloud interactor now waits on the same signal the Apple runner uses, the
software keyboard going from hidden to shown after its tap, and discloses what
it observed as `textEntryReadiness` so a fill with no witness cannot pass for
a filled field. Where keyboard visibility cannot witness the focus move —
back-to-back fills into one form, the shape that failed most often — it spends
the runner's full readiness budget rather than racing the app with a short
settle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Zjbzf7HdziX9SzFpNP7WW
* fix(cloud): let a new bundle-id open replace the tracked cloud iOS app
Adopting an explicitly spelled bundle id on a provider-backed open (the
fix that makes snapshot/diff work at all) also made a previously dead
precedence rule live: the provider branch returned currentAppBundleId
first, so once a first open had populated it, `open com.a` followed by
`open com.b` left the session still reporting com.a to every
appBundleId-gated command.
The local path does the opposite, and is the convention this branch is
meant to mirror: resolveIosApp returns a dotted target unchanged and
never consults the session's current app. Only its deep-link branches
prefer the tracked id. Flip the provider branch to match — an explicit
bundle-id target wins, and everything the branch cannot name (deep
links, display names, bare open) still falls back to the tracked id.
* fix(cloud): fail a witness-less cloud fill instead of reporting it filled
Review follow-ups on #1658.
`not-observed` was still a success: it sent the keys and answered "Filled N
chars", and nothing renders `textEntryReadiness` in default CLI output — so the
exact silent success this branch exists to remove survived whenever focus never
happened. A tap that raises no keyboard now fails with
`text_entry_focus_not_observed` and sends no keys, leaving the field untouched
rather than half-written, and the readiness vocabulary keeps only outcomes that
describe a fill that did type.
The readiness budget was advertised but not enforced at the request boundary:
each keyboard probe inherited the client's 30s default, so one hung probe could
hold a 2s wait for far longer. Probes now carry their own bound, threaded
through the client as a per-request timeout override.
The probe also swallowed every error as "this driver cannot answer", which
degraded a dead session, an auth rejection, or a grid outage into a blind text
entry. Only a positively classified unimplemented route counts as unsupported
now — classified on the W3C error code rather than the status, since `unknown
command` and `invalid session id` share HTTP 404 — and everything else
propagates.
The provider scenario proved request ordering against a stub that always
accepted keys. Its fake now models the device: focus lands a beat after the
tap, and keys arriving while the keyboard is down are accepted and dropped,
exactly as an unfocused field does. The tests assert the field's own value, and
both go red against the pre-fix `fill`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Zjbzf7HdziX9SzFpNP7WW
* fix(cloud): witness the tapped field's focus before a cloud fill types
Review of cc23f2b found two ways a fill could still report success
without evidence that OUR tap focused the field it was aimed at.
P1. `settled-keyboard-up` and `settled-unknown` both typed and returned
normal success. Keyboard visibility can only witness that *a* field took
focus, never *which*: filling a second field in an already-open form
reads the same before and after, so a missed tap left the first field
focused and `POST /keys` — which the driver routes to whatever holds
first responder — appended to it while every request returned 200.
Failing those closed outright would have broken ordinary multi-field
form fills, which do work: a live AWS Device Farm run types both fields
of a WebView login correctly. So witness focus properly instead. W3C
`GET /element/active` answers the question keyboard visibility cannot —
is the thing focused now the thing I tapped — and answers it whether or
not the keyboard was already up. That becomes the primary signal
(`focused-element`); the keyboard transition stays as the fallback for
drivers without the route, and a keyboard already up on such a driver
now refuses rather than typing.
The test is identity, not geometry. Containment of the tap point looks
like the obvious rule and is wrong: focusing a field can re-lay it out.
On a live iPhone 16, tapping Safari's collapsed address bar expands it
into a taller field that no longer covers the tapped point, and a
containment-only rule refused a fill that plainly worked. So a tap that
MOVES focus counts, with containment as the second half of the test —
re-filling the already-focused field moves nothing, and only geometry
tells that from a tap that missed. Both readings are taken before the
tap, since each is evidence only as a change.
P2. The 2s budget bounded the loop but not the calls inside it: every
probe got a fixed 1500ms, so one begun near the deadline finished well
past it. Both the probe timeout and the sleep are now capped by the
remaining budget.
Also fixes a related escape the review did not name: the poll loop had
no catch, so one transient grid error aborted a fill the next poll would
have satisfied. Probe failures are now tolerated within the budget, but
a budget that expires without a single answered probe rethrows, so a
dead session surfaces as itself rather than as "the tap missed".
The provider scenario gains the two-field case the review asked for: it
begins keyboard-up with the email field focused, misses the password
tap, and asserts no keys reach the email field.
Verified on AWS Device Farm iPhone 16 / iOS 18.0 at this exact tree:
address bar (the re-layout case) and both WebView login fields all
report `focused-element`, the second with the keyboard already up, and
the device reads back `tomsmith` and a 20-character password.
* fix(cloud): refuse a cloud fill no focus probe can witness, and bound the composite probe
Two blockers from the review of
|
||
|
Michał Pierzchała
|
cc943400a9 |
feat(daemon): [RFC] prototype foreground-attach convenience (#1670)
Prototype `open --foreground`: on a fresh session with no app argument, auto-resolves the target from the sole booted iOS simulator's sole foreground app (reusing the exact same ambiguity-detection probe that enriches the SESSION_NOT_FOUND hint), then attaches the initial interactive snapshot to the response by composing the existing snapshot-runtime dispatch. Collapses the documented 3-call snapshot-fails -> read-hint -> open -> snapshot-succeeds dance into a single call for the unambiguous case, while failing closed (AMBIGUOUS_MATCH) with no guessing otherwise. First-pass RFC, not reviewed — see PR body for the design tradeoff writeup, live before/after evidence, and scoped-out follow-ups. |