Commit Graph

7 Commits

Author SHA1 Message Date
Michał Pierzchała a830ac8df2 feat: add Linux command evidence lane (#2017)
* feat: add Linux command evidence lane

* fix: assert Linux find result shape

* fix: read Linux find result envelope

* fix: reset Linux calculator before diff

* fix: release Linux session before reset

* fix: guard Linux evidence session reset

* fix: forward Linux evidence timeout

* fix: tighten Linux evidence assertions

* fix: preserve Linux replay session identity

* fix: close Linux replay session before reset

* fix: share Linux evidence daemon state

* fix: keep Linux swipe evidence in bounds

* fix: keep Linux artifact gap honest
2026-08-25 07:56:13 +02:00
Michał Pierzchała 759f175332 refactor: move Wave 5 touch commands to platform runtime (#1987)
* refactor: move touch commands to platform runtime

* fix: require direct selector touch binding

* fix: address touch runtime review

* fix: classify maestro direct click guarantee

* fix: distinguish Maestro direct selector dispatch
2026-08-24 14:54:44 +02:00
Michał Pierzchała 296447707e refactor: migrate back/home/orientation/tv-remote/keyboard to the request-bound device runtime (#1955)
* refactor: migrate back/home/orientation/tv-remote/keyboard to the request-bound device runtime

Continues the ADR 0019 platform-runtime migration (Wave 5 generic leaves):
five generic-route commands move off dispatchKnownCommand/Interactor legacy
dispatch onto fact-owned admission, one bind per handler. keyboard uses the
R35 action-selected single-bind pattern (status/dismiss/enter each admit and
bind independently). All 8 owner runtime packages gained fact-cell tests for
the new operations; six smoke-coverage integration oracles and nine
daemon/capability unit test files were updated for the retired capability-
catalog admission these commands no longer carry.

* refactor: extract shared interactor-resolution prelude in keyboard-runtime

bindKeyboardStatus/Dismiss/Enter repeated the same signal-check +
resolveInteractor call; factor it into resolveKeyboardInteractor so each
binder is a two-line call instead of a six-line copy. No behavior change —
the three contract-module mutants planted earlier in review still kill on
this shape.

* fix: refuse watchOS admission for back/home/orientation/keyboard; pin tv-remote non-TV parity

P1: watchOS has no constructible Apple interactor (XCUITest cannot drive its UI, ADR-0009),
matching the existing captureScreenshot/captureSnapshot/readTextAtPoint/findSelector pattern
in this same file. appleBackFact/appleHomeFact/appleMobileInputEligible admitted every
Apple OS but tvOS/macOS, wrongly including watchOS. Facts now refuse watchOS explicitly for
back, home, orientation, and keyboard dismiss/enter, with a fact-cell test asserting no
binding for every one of them.

P2: verified the daemon's generic-route capability gate already reproduced the retired
per-platform tv-remote hint text (message stays the generic "<command> is not supported on
this device", hint carries the owner-specific text) for every device that could reach
dispatch in the old system -- the retired handleTvRemoteCommand's own "supported only on TV
targets" check was unreachable there and only exercised by a test calling dispatchCommand
directly. Added a daemon-level test pinning the exact iOS and Android-mobile hint strings to
make that parity explicit instead of implicit.

Also fixes a fallow complexity finding the P1 test edit introduced by splitting the fact-cell
assertions into five small named helpers instead of one large function.

* fix: stop orientation-runtime.test.ts's router-join test from hitting real adb

Root-caused the CI-only Coverage failure (unreproducible locally in isolation,
reproducible 2/2 in the full CI run): every generic-route leaf this migration
touches carries `androidBlockingDialogGuard: true`, and `dispatchGenericCommand`
calls `ensureNoAndroidBlockingDialogReady` unconditionally for any
`platform: 'android'` session reaching the real request router -- regardless of
whether admission is fact-based or capability-based. That check calls
`getAndroidBlockingDialogFocus`, which shells out to the real `adb` binary.

orientation-runtime.test.ts's "request router joins..." test used a synthetic
`platform: 'android'` device through `createRequestHandler` (the real router),
without stubbing the platform ADB layer -- only the runtime gateway was mocked.
On a host with a real `adb` binary (my machine) the subprocess fails fast and
`allowFailure` tolerates it, costing ~800ms-1.1s but still succeeding. On a host
with no `adb` binary at all (CI's Coverage job, a plain unit-test lane with no
Android SDK) the spawn itself throws, which isn't something `allowFailure`
catches, producing exactly the observed `ok: false` unsupported-operation
response.

back/home/tv-remote's equivalent router-join tests already use Apple/Vega
devices, so they never reached this path. Switched orientation's fixture to
match -- Apple, since the fixture's facts/execution are fully synthetic and
platform-agnostic regardless.

Also: renamed the widely-shared 'emulator-5554'/'ios-simulator' device-id
literals in back/orientation/tv-remote/keyboard-runtime.test.ts to file-scoped
ids. Device claims for a `local-family` owner binding hit the real on-disk
`require-owner` claim file (keyed only by canonical device id), and 27+
pre-existing test files already share 'emulator-5554'; this migration added
three more consumers of it under a `require-owner` policy that reaches real
admission, which was worth eliminating as a source of doubt even though it
wasn't the actual root cause here.

* refactor: extract navigation/keyboard concepts into sibling modules; test the real Android dialog-guard path

packages/platform-apple/src/runtime.ts and packages/provider-limrun/src/app-log-runtime.ts
grew past the repo's 500-line extraction threshold. Move the new back/home/orientation/
tv-remote/keyboard facts and bindings into packages/platform-apple/src/navigation/runtime.ts
(new sibling module, matching deployment/runtime.ts's existing pattern), and the new keyboard
facts/bindings for limrun into the existing packages/provider-limrun/src/interaction-operations.ts
(which already held the sibling navigation logic).

Also fix orientation-runtime.test.ts's router-join test: it previously swapped its device
fixture from Android to Apple to dodge the real adb-backed blocking-dialog guard, which masked
the Android route that was actually failing in CI. Keep the Android fixture and stub
getAndroidBlockingDialogFocus instead, the same seam request-router-android-modal.test.ts
already uses.

* refactor: adopt granular contracts subpaths for back/home/orientation/tv-remote/keyboard

Following main's #1969 (facade granularization), give each of this branch's five
new contract modules their own package.json entry subpath and move every
value-importer (owner runtime packages, the daemon binders, and their tests) off
the wide @agent-device/contracts/platform facade onto the specific module that
owns the symbol — the same convention #1969 established for the rest of the
vocabulary. Keeps this migration's files out of the contracts-entry-closure gate
and out of the eager-evaluation cost #1969 measured for the daemon's permanent
hubs (registry.ts, dispatch.ts).

* refactor: shared navigation/keyboard binder table; dedupe keyboard admission; drop restated types

Addresses the review's finding 1 (seven per-owner copies of the same
"fact-keyed table of interactor binders" pattern) by extracting
bindAdmittedLocalInteractorOperations/bindAdmittedProviderInteractorOperations
into packages/contracts/src/interactor-operation-catalog.ts. Each owner now
requests the subset of back/home/setOrientation/tvRemote/keyboard{Status,
Dismiss,Enter} it admits, instead of hand-writing
`facts.operations.<key>.available ? bind…(resolver) : {}` per operation.
Applied across all seven call sites (apple, android, harmonyos, vega, linux,
webdriver, limrun) and collapsed limrun's two separate bind functions
(navigation, keyboard) into one shared call.

Finding 3 (resolveBoundKeyboardRuntime copy-pastes admit-then-wrap three
times): extracted a local admitKeyboardAction<...> helper mirroring
resolveBoundGenericRuntime's admit-then-defer shape, so the three action
branches (status/dismiss/enter) share one admission path.

Finding 6 (execute* helpers hand-restate a contract that can drift): back/
home/orientation/tv-remote/keyboard's execute functions are now typed off
`BoundDeviceRuntime<typeof xRuntimeUse>` (derived from the actual bind-use
value) instead of a hand-written `Readonly<{ operations: Readonly<{...}> }>`
shape. Also fixed provider-limrun's `RuntimeOperationUnavailability |
{ available: true }` restating RuntimeOperationFact by hand — folded away
entirely once the bind functions it typed were removed.

Finding 7 (naming/placement): platform-apple/runtime.ts's misleadingly-named
`captureOperations` bucket (held deployment/network/recording/find, not just
capture) collapsed into one flat `operations` object now that the navigation
bucket is a single function call instead of six ternaries.

Exported RuntimeAdmissionRequest from runtime-admission.ts (needed by the new
keyboard admission helper). Added packages/contracts/src/
interactor-operation-catalog.test.ts for the new shared binder table.

pnpm typecheck, check:fallow, check:layering, and the full unit-core suite
(1010 files / 7513 tests, one known contention-flake excluded) are green.

* refactor: split generic-mutating command traits from the legacy dispatch pair

Addresses the review's finding 5: GENERIC_MUTATING_LINUX_DEVICE_COMMAND_TRAITS
bundled two orthogonal things (daemon/recording traits, and the legacy
capability+dispatch pair migration strips), forcing every migrated
descriptor to hand-expand the constant minus two fields plus an explanatory
comment.

Split into GENERIC_MUTATING_COMMAND_TRAITS (the shared daemon/recording
traits) and LEGACY_LINUX_DEVICE_EXECUTION (the dispatch/capability pair).
back/home/orientation/tv-remote (this migration) and focus (an earlier one,
same pattern, previously a stale reference to the retired constant name)
now spread the trait constant directly instead of hand-expanding it; the
still-legacy `scroll` descriptor spreads both pieces, equivalent to the
retired constant.

pnpm typecheck, check:fallow, check:layering, and the registry/daemon test
suites are green.

* refactor: table-ify packages/contracts/src/keyboard-runtime.ts's three-way duplication

Finding 3's second half: the three bindKeyboardX functions and six
bindLocal/ProviderKeyboardXInteractor entry points differed only by method
name and label string. Replaced with one generic bindKeyboardAction<Key>
dispatching off the operation key (interactor[key], resolved from a small
label table) plus two shared local/provider dispatch helpers the six named
exports each call with their own key — collapsing three copies of the bind
logic into one and six near-duplicate entry-point bodies into one line each,
while keeping every exported name and type signature unchanged.

pnpm typecheck, check:fallow, check:layering, and pnpm check:affected --run
are green.

* refactor: parameterize runSessionOrSelectorDispatch with an execute strategy

Addresses the review's finding 2: handleKeyboardCommand re-implemented
runSessionOrSelectorDispatch's orchestration step for step (session/selector
guard, device resolve, ref-frame expiry, record) instead of reusing it,
because the shared function had no seam for keyboard's bind-and-execute
admission — only the legacy requireCommandSupported + dispatchCommand path.
That left the shared orchestrator with one caller instead of two, and set a
precedent that would fork a new copy for each of the 28 remaining
session-route migrations.

Gave runSessionOrSelectorDispatch an `execute` parameter: the orchestration
(guard, resolve device, admit-then-execute, expire ref frame if mutating,
derive and record next session) stays in one place, and callers supply their
own admission/execution strategy. Extracted `legacySessionDispatchExecute`
for the still-legacy capability-gate-then-dispatchCommand shape
`handleTriggerAppEventCommand` (the remaining legacy caller) now passes
explicitly, and `keyboardSessionExecute` for keyboard's bind-and-execute
shape. Deleted the now-fully-redundant `executeBoundKeyboardCommand` — its
result recording duplicated what the shared orchestrator's tail already
does.

pnpm typecheck, check:fallow, check:layering, the full daemon test suite
(321 files / 2271 tests), and pnpm check:affected --run are green.

* refactor: extract limrun facts-runtime.ts; discriminate KeyboardDismissResult by owner

app-log-runtime.ts was still 589 lines after the shared-abstraction fixes; moves fact
assembly (limrunAppLogFacts/limrunAppLogRecoveryFacts/limrunLifecycleFacts/deploymentOptions)
to a new facts-runtime.ts and the shared device-identity predicate to device.ts, the leaf
both files already depend on. app-log-runtime.ts is now 336 lines.

KeyboardDismissResult was an 11-field optional bag with executeKeyboardDismiss separately
re-deriving platform from the device and projecting subsets by hand. Each owner (android,
apple, harmonyos) now tags its own result with a `kind` discriminant, so an owner can only
ever produce its own shape, and the daemon derives the wire `platform` label from `kind`
instead of guessing from the device a second time. Wire output is unchanged.

* fix: expire ref frame before the mutating call, not after; extract session/selector dispatch; derive catalog operations from facts

runSessionOrSelectorDispatch awaited execute(device, session) — which bundled admission
and the mutating invocation together — before expiring the ref frame, so a rejecting or
timed-out invocation left a stale frame active (ADR 0014 requires expiry immediately
before the mutating call, with no success-only rollback). Split the execute thunk into
`prepare` (admission only) + a deferred `execute` invocation, so the orchestrator can
expire between them regardless of how the invocation resolves. Added a regression test
proving the frame still expires when the invocation rejects.

Extracted runSessionOrSelectorDispatch and its keyboard/trigger-app-event callers into a
new session-selector-dispatch.ts, matching this file's own convention of one file per
command-group (session.ts shrinks from 571 to well under its 500-line budget).

bindAdmittedLocalInteractorOperations/bindAdmittedProviderInteractorOperations accepted
both a facts object and a separately hand-maintained `operations` array naming the same
keys — a second source of truth that could drift from what the facts actually admit.
Removed the array; the binder now walks the fixed set of navigation operations and lets
each owner's own facts decide what binds, exactly as before but with one source of truth.

* style: reformat legacySessionDispatchExecute call in session-selector-dispatch.ts

* refactor: derive catalog operation list from one canonical tuple; move keyboard orchestration tests

NAVIGATION_INTERACTOR_OPERATIONS was declared as a plain readonly array independently
of the NavigationInteractorOperation union it walked, so a future union member could
compile without ever being added to the walk list, silently preventing an admitted
fact from binding. Made the tuple the single canonical value: the union type is now
derived from it via `(typeof TUPLE)[number]`, so LOCAL_BINDERS/PROVIDER_BINDERS'
Record<NavigationInteractorOperation, ...> completeness is checked against the same
tuple, not a separately hand-kept list. Added a regression test binding all seven
operations at once to pin the runtime walk, independent of the type-level guarantee.

Moved the four keyboard-orchestration tests (the two ADR 0014 ref-frame seam tests
plus the two session/selector-guard tests) out of the mixed appstate/perf test file
into a new session-selector-dispatch.test.ts, colocated with the file they exercise.
Strengthened the rejection regression test to assert the frame is already expired
from inside the rejecting keyboardDismiss callback itself, pinning the exact
pre-invocation seam rather than only checking the end state after the dispatch settles.

* fix: restore back/home/orientation/tv-remote/keyboard-runtime exports lost in rebase

Rebasing onto origin/main dropped these five package.json export entries during
conflict resolution (the granular-subpath commit's package.json changes silently
lost during merge). Restored, confirmed by pnpm typecheck across all 17 workspace
packages and the full unit-core suite (1023 files / 7581 tests).

* test: pin the exact point the live iOS email field value goes missing

Two prior CI runs on this PR saw the seeded email field ("ada@example") end up
containing only a typed suffix (".test") by the time the flow reads it back at
the end — after fill, keyboard dismiss, coordinate refocus, and type. Since this
PR touches executeKeyboardDismiss's response shaping, the reviewer asked to
disprove keyboard dismiss as the cause rather than assume the pre-existing
dropped-keystroke flake pattern applies.

Added two read-back checkpoints: right after seeding (before dismiss runs at
all) and right after dismiss (before the coordinate refocus + type steps that
follow). If both hold "ada@example", the loss happens during refocus/type, not
dismiss — matching the documented flake, not a regression in this PR's diff.

* refactor: make keyboard status/enter owner-discriminated too; trim review-round prose

KeyboardStatusResult and KeyboardEnterResult were bare objects; executeKeyboardStatus
and executeKeyboardEnter derived the wire platform label from device.platform via
keyboardPlatformLabel, the same re-derivation already fixed for dismiss. Each owner
now tags its own result with a kind (android's status/enter as 'ime-probe' and
'android-acknowledged', harmonyos's enter as 'harmonyos-acknowledged', apple's enter
as 'visibility-echo'), and the daemon derives platform from a kind-keyed lookup table
for all three actions. keyboardPlatformLabel and its isIosFamily import are gone —
nothing derives platform from the device anymore. Android and HarmonyOS's enter
acknowledgments are structurally identical (empty besides kind), so the discriminant
alone — not result shape — is what tells the daemon which owner actually ran.

Added a harmonyos enter test alongside the existing ios/android ones so all three
owners are covered for both dismiss and enter's kind-to-platform mapping.

Also trimmed several comments that narrated which PR review round motivated them
down to just the durable invariant or rationale — the type shape, test names, and
assertions already carry the proof.
2026-08-24 10:40:27 +02:00
Michał Pierzchała 3bf3ff130a fix: Linux input/a11y defects from #1935 (click miss, typed '=', GTK4 text) (#1949)
* diag: instrument Linux CI to gather evidence for #1935 input/a11y defects

Temporary — adds a diagnostic step that dumps raw AT-SPI interfaces/actions
for gnome-calculator's digit buttons, tests a raw xdotool click at a
button's own rect (bypassing our promotion logic), and isolates the typed
'=' character in several configurations. Will be removed once the real
fixes land.

* diag: harden diagnostic step against bash -e and AT-SPI registration races

The prior version crashed 7s in: GH Actions runs steps under bash -e, and
an unguarded python3 heredoc threw (iterating a dict instead of a list
when the app wasn't found yet), aborting the rest of the script silently
under continue-on-error. Guards every fallible command, and replaces the
fixed 2s sleep with inspect.py's own poll-until-found loop.

* diag: test WINDOW coordtype and static Text.get_text call (round 3)

Round 2 proved Component.get_extents(SCREEN) returns (0,0) for every
non-toplevel widget (real click miss confirmed on-screen), and
Text.get_text() throws — a documented PyGObject binding collision with
the deprecated 1-arg Accessible.get_text(). This narrows to the two
candidate fixes before writing them: does CoordType.WINDOW give usable
relative offsets, and does Atspi.Text.get_text(accessible, ...) (static
call) return the real typed text.

* fix(linux): resolve click-miss, dropped '=', and GTK4 text exposure defects

Three defects surfaced by CI on #1935 (Linux Smoke lane), all confirmed
live via instrumented CI runs before being fixed here:

1. Click misses its target: Component.get_extents(Atspi.CoordType.SCREEN)
   returns (0, 0) as the origin for every non-toplevel widget under this
   GTK4 build — confirmed by a raw click at the computed rect center
   landing on the window's own header-bar button instead of the intended
   digit button. CoordType.WINDOW gives correct, distinct per-widget
   offsets, so get_rect() now computes screen-absolute rects as that
   offset plus the enclosing top-level frame's own (correct) screen
   origin, threaded through traverse_node() alongside the existing
   window-title tracking. Complementary hardening: role "label" is now
   excluded from `hittable`, since GTK4 wraps every button's caption in a
   same-rect "label" child, and the shared cross-platform promotion logic
   in interaction-targeting.ts would otherwise retarget a click from the
   button onto that non-interactive label.

2. Typed '=' never arrives: a single isolated synthetic keystroke sent
   right after a focus change is unreliably delivered — confirmed live,
   both `xdotool type -- "="` and `xdotool key equal` sent alone produced
   no character at all, while multi-character bursts always landed in
   full. typeLinux and sendKey now wait a short settle margin before
   dispatching to xdotool/ydotool, absorbing the race regardless of which
   action last changed focus.

3. GTK4 apps expose no editable text: accessible.get_text_iface().get_text()
   throws "Atspi.Accessible.get_text() takes exactly 1 argument (3 given)"
   — a documented PyGObject binding collision between Text.get_text and
   the deprecated 1-argument Accessible.get_text, silently swallowed as
   "no text" by the broad exception handler. get_text_value() now calls
   the unbound Atspi.Text.get_text(accessible, ...) form, which correctly
   returns the real content.

The Linux smoke replay is restored to exercise all three fixes together
(click a resolved digit button, type a full calculation including the
'=' keystroke, wait on the computed result through the tree) instead of
staying at the weakened, contract-tier assertions the defects had forced.
The coverage manifest promotes click and type from command-contract to
live accordingly.

* fix(linux): drop unproven keyboard-settle and hittable changes per review

Addresses thymikee's review on #1949 (both points correct):

P1: the keyboard settle (typeLinux/sendKey) was unjustified. The cited
diagnostic evidence for a dropped '=' actually shows the opposite —
"100+55=" and "5=5" both computed correctly with zero settle, proving
'=' was delivered in every multi-character burst tested. Sending '='
alone to an empty entry showing a blank display is normal calculator
semantics (nothing to evaluate), not a lost keystroke. The likelier
explanation for the original "100+55" screenshot (run 32487868346) is
that its attempt-3 hit the already-fixed mousemove --sync hang, not an
independent keyboard-dispatch defect. Reverted; no keyboard-dispatch
change was needed.

P2: the `role_name != "label"` hittable narrowing was extra surface
beyond what the click-miss fix required. The corrected AT-SPI coordinates
alone fix the observed miss — the button and its same-rect label child
resolve to nearly identical centers, so descendant promotion still lands
inside the button either way, and the replay can't distinguish which
node it actually targeted. Reverted; only the coordinate fix remains.
2026-08-21 20:02:17 +02:00
Michał Pierzchała 81409f1a7c refactor: migrate type to the request-bound device runtime (#1935)
* refactor: migrate type to the request-bound device runtime

Wave 5 unit 2 for #1739 (ADR 0019), find's last blocker. `type "text"` and
`find <q> type "text"` now reach the device through one admitted, request-bound
`typeText` operation instead of the `handleTypeCommand` interactor leaf and its
dispatch-table arm.

- New `TypeTextRuntimeOperations` contract riding the same `Interactor` seam as
  focus/screenshot/element-text; the operation returns the interactor's own
  closed `TypeTextBackendResult`, so Apple route evidence passes through and
  every other owner types blind, exactly as before. The iOS synthesized-type
  commit wait (#1676) is Apple-interactor-internal and moves nowhere.
- The interaction backend's `typeText` member exists only when the `type`
  handler admitted and bound a runtime — no caller can fall back to legacy
  dispatch, so the command keeps exactly one execution path (R41).
- `executeBoundTypeText` reproduces the retired leaf byte-for-byte: leading-ref
  rejection with the same hint, space-joined positionals, the 0-10000 delay
  bound, and only textEntryRoute surviving from the owner's result. Its parse
  pins moved from the dispatch-level tests into the daemon runtime test.
- Exact-owner facts replace the capability bucket (apple sim+device, android
  all-but-simulator-row, harmonyos emulator+device, linux device, web device,
  vega unavailable, providers wherever their interactor is reachable) and
  `type` leaves HARMONYOS_SUPPORTED_COMMANDS / WEB_INTERACTION_COMMANDS.
- The Linux desktop replay types a digit on real hardware; the coverage
  manifest promotes `type` contract -> live with the two-sided count pins.
- Android/webdriver facts helpers extracted (androidTouchFact, interactorCell)
  to keep inspectFacts under the complexity gate.

`find` stays legacy: both of its direct execution legs now share bound
runtimes, so the atomic R35 cutover is next.

* refactor(type): single-pass daemon routing, shared binder source, owner cell tests

Review follow-ups on #1935.

- The type handler now calls the bound executor directly: the interaction-
  runtime hop validated and formatted what executeBoundTypeText validates and
  formats again, so it is gone — no boundTypeText backend member, no second
  result rebuild. The ADR 0014 frame expiry moves to the handler.
- New contracts/interactor-operation-binding.ts: one local resolver and one
  fail-closed provider resolver shared by the screenshot, focus, and type
  binders — three private copies retired, provider error text preserved.
- provider-limrun/interaction-operations.ts: the interactor-backed interaction
  cells move out of the app-log owner (586 -> 563 lines, below its pre-unit
  size); text interaction is composed by that owner, not defined in it.
- Every owner runtime test now pins the focusPoint/typeText fact cells and
  bound-operation presence for its exact kinds: apple, android (incl. the
  synthetic-simulator refusal), harmonyos, linux, web, vega (refusal + hint),
  webdriver (reachability-gated, incl. inactive session), limrun (live +
  recovery). The webdriver unsupported-capability row documents that
  interaction gates on interactor reachability, not capture declarations.
- The Linux replay assertion is now change-sensitive: type "555" then wait for
  a 555 node — no calculator button carries that label, so the wait passes only
  if the keystrokes landed in the display; deleting the type step turns it red.

* fix(replay): give the calculator focus before the Linux type assertion

The change-sensitive wait exposed what the review predicted: the typed digits
never landed, because `focus 100 100` clicks the DESKTOP and takes keyboard
focus away from the calculator. The old broad assertion masked exactly this.

The retries then wedged on a latent quirk: attempt-1 leaves the pointer at
(100,100), and the next attempt's `xdotool mousemove --sync` to the same point
waits for a motion event that never comes, so every retry dies at the focus
step with a 10s timeout — which is why the lane reported step 7, not the
failing wait.

New tail: `focus 100 100` (R40 evidence + survival assert), then
`click "label=1"` — a resolved press inside the window that restores keyboard
focus, proves pointer input lands in the app, and moves the pointer off
(100,100) so retries cannot trip the mousemove no-op hang — then `type "55"`
and `wait "label=155 || text=155 || value=155"`. No button is labelled 155, so
the wait passes only if the typed keystrokes reached the display.

* refactor(type): delete the fallow SDK typeText surface, drop dead surface fields

Thermo-nuclear review follow-ups (reviewed at 82fb8c2dc; the three-hop relay
it names was already deleted in 3d44b6dea — these are the residuals).

- typeTextCommand had zero production callers after the direct-executor
  routing: the daemon was its only consumer, and the released SDK types over
  the wire (executeCommand('type')), not through the embedded runtime catalog.
  Deleted: the command, its Options/Result types, the catalog registrations,
  the AgentDeviceBackend.typeText member, and every fixture/pin that kept the
  dead surface alive. R41's retirement claim now names typeTextCommand, so a
  revival fails the gate. One parse/compose owner remains: executeBoundTypeText.
- TypeTextInput.options.surface and FocusPointInput.options.surface were dead
  clones — never set by a projector, never read by a binder. Removed both.
- The Linux replay click disambiguates with role=button: the calculator tree
  carries [text] digit nodes beside the buttons, so a bare label=1 was an
  AMBIGUOUS_MATCH rejection on attempt-1 — which parked the pointer at
  (100,100) and made every retry hang in the xdotool no-op move, reporting as
  the step-7 focus timeout.

Live-verified on iPhone 17 Pro at this head: coordinate focus -> bare type ->
route synthesized-first-responder -> text observable in the captured tree.

* fix(ci): lower the settle.test.ts pin, add pixel evidence to the Linux type wait

- settle.test.ts shrank to 2359 lines when its dead typeText fixture member
  left; the ratchet pin follows it down (the history-backed gate caught the
  gap on CI while the local affected run had not re-selected the ratchet).
- The Linux replay now screenshots the calculator right after the type step.
  The lane uploads test/screenshots/replays/*.png on pass and fail, so a
  wait-155 timeout becomes diagnosable from the artifact: display showing 155
  means a tree-exposure gap; an empty display means the input never landed.
  The 20-ref divergence dump cannot show the entry node either way.

* fix(replay): assert the Linux type through the computed result

Run 32485981780's typed-state artifact settled both open questions with one
image: the display shows "55" — the bound typeText keystrokes LAND on Linux
CI — while the wait for that value timed out, so the calculator entry does not
expose its text to selectors; and the display shows no leading "1", so the
resolved click on button 1 missed its target entirely.

Both discoveries leave the replay: the click dependency goes (a pre-existing
click-coordinate defect is not this unit's evidence chain), and the assertion
moves to where the tree can answer — the typed string is now a full
calculation ("100+55=") whose `=` creates a history row, and the wait matches
the computed 155. No button carries that label and the typed string never
contains it, so the wait passes only if the keystrokes executed; deleting the
type step turns it red. The typed-state screenshot stays as per-run pixel
evidence either way.

* fix(linux): guard the no-op --sync mousemove; assert typed digits via value=

Two artifact PNGs decided this. Run 32485981780 shows "55" in the entry while
the wait for it timed out on a mismatched target; run 32487868346 shows
"100+55" — digits and + land, the = keystroke does not, and the retries died
in the focus step again because the previous commit removed the pointer-moving
click.

- moveTo now probes `xdotool getmouselocation --shell` and skips the --sync
  move when the pointer already sits on the target: a no-op move emits no
  motion event and hangs until the action timeout, which is what reported
  every failed replay retry as its first coordinate step. A failed probe never
  blocks the move. The provider test pins both sequences, including the
  skip case.
- The replay types digits only ("155") and waits on `value=155`: the AT-SPI
  dumper reads the entry's Text interface into the node's value and the
  selector engine matches it — the earlier "exposure gap" conclusion came
  from a pair that never tested the matching value. No button carries 155, so
  the wait passes only if the keystrokes executed.

* fix(replay): keep Linux type at contract tier — GTK4 exposes no entry text

Run 32490373693 closed the investigation: the typed-state artifact shows
"155" in the calculator entry while the wait for value=155 timed out with no
interactive filtering in play and a matcher that does compare node.value. The
AT-SPI dumper's get_text_iface() route returns nothing for GTK4
gnome-calculator, so no tree-level assertion on typed text can hold on this
lane today.

The replay keeps the type step and uploads the typed-entry screenshot every
run — live pixel evidence that the migrated typeText path lands keystrokes on
real Linux hardware — and closes with the survival assertion. The manifest
claim returns to the contract tier with the reason written at the entry, and
the count pins follow. The GTK4 exposure defect joins the Linux input-defect
chip; fixing PyGObject-vs-GTK4 blind through CI rounds is not a sane loop.

The moveTo no-op guard from the previous commit stays: it is why this run
finally reported the true failing step on every attempt instead of the
step-7 hang.
2026-08-21 17:15:35 +02:00
Michał Pierzchała 46eff36f85 refactor: migrate focus to the request-bound device runtime (#1925)
* refactor: migrate focus to the request-bound device runtime

Wave 5's first unit (#1739, ADR 0019). `focus x y` and `find <q> focus` now
reach the device through one admitted, request-bound `focusPoint` operation
instead of the `handleFocusCommand` interactor leaf and its dispatch-table arm.

- New `FocusRuntimeOperations` contract with local and provider interactor
  binders, mirroring the screenshot/element-text seam rather than inventing a
  second way for one operation class to reach its mechanics.
- Exact-owner facts replace the capability bucket: apple simulator/device,
  android emulator/device/unknown, harmonyos emulator/device, linux device,
  web device, vega none, providers wherever their interactor is reachable.
  That is the retired bucket's cell table, restated as facts.
- `focus` leaves BASE_COMMAND_CAPABILITY_MATRIX and both hand-maintained
  overlays (HARMONYOS_SUPPORTED_COMMANDS, WEB_INTERACTION_COMMANDS).
- R40 is the new parametrized cutover row; `focusPoint` has exactly one owner.
- The `x y` positional parse moves to utils and is shared with the still-legacy
  touch siblings, so a migrated command cannot drift from them.

`find` stays legacy: this unit owns its focus leg only, its `type` leg still
dispatches, and R35 waits on the Wave 5 `type` unit.

* test(focus): cover the owning interactor binders, lower the find ratchet

Review follow-ups on #1925.

P1: focus-runtime.test.ts bound a fake focusPoint, so deleting the interactor
call inside bindLocalFocusInteractor left focus a successful no-op with every
test green. Adds packages/contracts/src/focus-runtime.test.ts, which executes
both binders and asserts resolver context, positional (x, y) forwarding, the
structured missing-provider failure, and that an already-cancelled request
never resolves an interactor at all.

Two planted mutants confirm it bites: removing
`await interactor.focus(input.point.x, input.point.y)` and transposing its two
arguments each fail exactly the two forwarding tests, while the daemon-level
focus and find suites stay green — which is the gap the reviewer named.

Coverage: find.test.ts shrank to 1204 lines when its focus assertion moved off
the dispatch mock; the ratchet pin follows it down.

* test(focus): add live Linux focus coverage to the desktop replay

The Linux `focus` claim rested on the provider scenario at command-contract
level. The desktop replay runs on real Linux hardware in the Smoke lane, so it
now runs a coordinate focus and re-asserts the session survived it.

Coordinate, not selector: the step exists to prove the migrated `focusPoint`
path executes on real hardware, so it must not be able to fail on match
ambiguity or CI layout drift.

Reclassifies focus contract -> live in the Linux coverage manifest and updates
the two pinned counts. The manifest gate is two-sided — a live claim must name
a command the replay actually invokes — so the claim cannot drift from the file.
2026-08-21 11:34:22 +02:00
Michał Pierzchała 2a9a4ee80c test: add Linux platform command coverage manifest (#1921)
* test: add Linux platform command coverage manifest

* test: address platform coverage review feedback
2026-08-20 21:48:05 +02:00