4 Commits

Author	SHA1	Message	Date
Michał Pierzchała	81409f1a7c	refactor: migrate type to the request-bound device runtime (#1935) ... * refactor: migrate type to the request-bound device runtime Wave 5 unit 2 for #1739 (ADR 0019), find's last blocker. `type "text"` and `find <q> type "text"` now reach the device through one admitted, request-bound `typeText` operation instead of the `handleTypeCommand` interactor leaf and its dispatch-table arm. - New `TypeTextRuntimeOperations` contract riding the same `Interactor` seam as focus/screenshot/element-text; the operation returns the interactor's own closed `TypeTextBackendResult`, so Apple route evidence passes through and every other owner types blind, exactly as before. The iOS synthesized-type commit wait (#1676) is Apple-interactor-internal and moves nowhere. - The interaction backend's `typeText` member exists only when the `type` handler admitted and bound a runtime — no caller can fall back to legacy dispatch, so the command keeps exactly one execution path (R41). - `executeBoundTypeText` reproduces the retired leaf byte-for-byte: leading-ref rejection with the same hint, space-joined positionals, the 0-10000 delay bound, and only textEntryRoute surviving from the owner's result. Its parse pins moved from the dispatch-level tests into the daemon runtime test. - Exact-owner facts replace the capability bucket (apple sim+device, android all-but-simulator-row, harmonyos emulator+device, linux device, web device, vega unavailable, providers wherever their interactor is reachable) and `type` leaves HARMONYOS_SUPPORTED_COMMANDS / WEB_INTERACTION_COMMANDS. - The Linux desktop replay types a digit on real hardware; the coverage manifest promotes `type` contract -> live with the two-sided count pins. - Android/webdriver facts helpers extracted (androidTouchFact, interactorCell) to keep inspectFacts under the complexity gate. `find` stays legacy: both of its direct execution legs now share bound runtimes, so the atomic R35 cutover is next. * refactor(type): single-pass daemon routing, shared binder source, owner cell tests Review follow-ups on #1935. - The type handler now calls the bound executor directly: the interaction- runtime hop validated and formatted what executeBoundTypeText validates and formats again, so it is gone — no boundTypeText backend member, no second result rebuild. The ADR 0014 frame expiry moves to the handler. - New contracts/interactor-operation-binding.ts: one local resolver and one fail-closed provider resolver shared by the screenshot, focus, and type binders — three private copies retired, provider error text preserved. - provider-limrun/interaction-operations.ts: the interactor-backed interaction cells move out of the app-log owner (586 -> 563 lines, below its pre-unit size); text interaction is composed by that owner, not defined in it. - Every owner runtime test now pins the focusPoint/typeText fact cells and bound-operation presence for its exact kinds: apple, android (incl. the synthetic-simulator refusal), harmonyos, linux, web, vega (refusal + hint), webdriver (reachability-gated, incl. inactive session), limrun (live + recovery). The webdriver unsupported-capability row documents that interaction gates on interactor reachability, not capture declarations. - The Linux replay assertion is now change-sensitive: type "555" then wait for a 555 node — no calculator button carries that label, so the wait passes only if the keystrokes landed in the display; deleting the type step turns it red. * fix(replay): give the calculator focus before the Linux type assertion The change-sensitive wait exposed what the review predicted: the typed digits never landed, because `focus 100 100` clicks the DESKTOP and takes keyboard focus away from the calculator. The old broad assertion masked exactly this. The retries then wedged on a latent quirk: attempt-1 leaves the pointer at (100,100), and the next attempt's `xdotool mousemove --sync` to the same point waits for a motion event that never comes, so every retry dies at the focus step with a 10s timeout — which is why the lane reported step 7, not the failing wait. New tail: `focus 100 100` (R40 evidence + survival assert), then `click "label=1"` — a resolved press inside the window that restores keyboard focus, proves pointer input lands in the app, and moves the pointer off (100,100) so retries cannot trip the mousemove no-op hang — then `type "55"` and `wait "label=155 || text=155 || value=155"`. No button is labelled 155, so the wait passes only if the typed keystrokes reached the display. * refactor(type): delete the fallow SDK typeText surface, drop dead surface fields Thermo-nuclear review follow-ups (reviewed at 82fb8c2dc; the three-hop relay it names was already deleted in 3d44b6dea — these are the residuals). - typeTextCommand had zero production callers after the direct-executor routing: the daemon was its only consumer, and the released SDK types over the wire (executeCommand('type')), not through the embedded runtime catalog. Deleted: the command, its Options/Result types, the catalog registrations, the AgentDeviceBackend.typeText member, and every fixture/pin that kept the dead surface alive. R41's retirement claim now names typeTextCommand, so a revival fails the gate. One parse/compose owner remains: executeBoundTypeText. - TypeTextInput.options.surface and FocusPointInput.options.surface were dead clones — never set by a projector, never read by a binder. Removed both. - The Linux replay click disambiguates with role=button: the calculator tree carries [text] digit nodes beside the buttons, so a bare label=1 was an AMBIGUOUS_MATCH rejection on attempt-1 — which parked the pointer at (100,100) and made every retry hang in the xdotool no-op move, reporting as the step-7 focus timeout. Live-verified on iPhone 17 Pro at this head: coordinate focus -> bare type -> route synthesized-first-responder -> text observable in the captured tree. * fix(ci): lower the settle.test.ts pin, add pixel evidence to the Linux type wait - settle.test.ts shrank to 2359 lines when its dead typeText fixture member left; the ratchet pin follows it down (the history-backed gate caught the gap on CI while the local affected run had not re-selected the ratchet). - The Linux replay now screenshots the calculator right after the type step. The lane uploads test/screenshots/replays/*.png on pass and fail, so a wait-155 timeout becomes diagnosable from the artifact: display showing 155 means a tree-exposure gap; an empty display means the input never landed. The 20-ref divergence dump cannot show the entry node either way. * fix(replay): assert the Linux type through the computed result Run 32485981780's typed-state artifact settled both open questions with one image: the display shows "55" — the bound typeText keystrokes LAND on Linux CI — while the wait for that value timed out, so the calculator entry does not expose its text to selectors; and the display shows no leading "1", so the resolved click on button 1 missed its target entirely. Both discoveries leave the replay: the click dependency goes (a pre-existing click-coordinate defect is not this unit's evidence chain), and the assertion moves to where the tree can answer — the typed string is now a full calculation ("100+55=") whose `=` creates a history row, and the wait matches the computed 155. No button carries that label and the typed string never contains it, so the wait passes only if the keystrokes executed; deleting the type step turns it red. The typed-state screenshot stays as per-run pixel evidence either way. * fix(linux): guard the no-op --sync mousemove; assert typed digits via value= Two artifact PNGs decided this. Run 32485981780 shows "55" in the entry while the wait for it timed out on a mismatched target; run 32487868346 shows "100+55" — digits and + land, the = keystroke does not, and the retries died in the focus step again because the previous commit removed the pointer-moving click. - moveTo now probes `xdotool getmouselocation --shell` and skips the --sync move when the pointer already sits on the target: a no-op move emits no motion event and hangs until the action timeout, which is what reported every failed replay retry as its first coordinate step. A failed probe never blocks the move. The provider test pins both sequences, including the skip case. - The replay types digits only ("155") and waits on `value=155`: the AT-SPI dumper reads the entry's Text interface into the node's value and the selector engine matches it — the earlier "exposure gap" conclusion came from a pair that never tested the matching value. No button carries 155, so the wait passes only if the keystrokes executed. * fix(replay): keep Linux type at contract tier — GTK4 exposes no entry text Run 32490373693 closed the investigation: the typed-state artifact shows "155" in the calculator entry while the wait for value=155 timed out with no interactive filtering in play and a matcher that does compare node.value. The AT-SPI dumper's get_text_iface() route returns nothing for GTK4 gnome-calculator, so no tree-level assertion on typed text can hold on this lane today. The replay keeps the type step and uploads the typed-entry screenshot every run — live pixel evidence that the migrated typeText path lands keystrokes on real Linux hardware — and closes with the survival assertion. The manifest claim returns to the contract tier with the reason written at the entry, and the count pins follow. The GTK4 exposure defect joins the Linux input-defect chip; fixing PyGObject-vs-GTK4 blind through CI rounds is not a sane loop. The moveTo no-op guard from the previous commit stays: it is why this run finally reported the true failing step on every attempt instead of the step-7 hang.	2026-08-21 17:15:35 +02:00
Thiago Brezinski	a13a6832ee	feat: add selector-targeted drag gestures (#1567) ... * feat: add selector-targeted drag gestures * fix: address drag gesture review feedback * fix: satisfy drag review quality gates * fix(android): lower drag trajectories piecewise * test(replay): validate drag fixture selectors * fix(ios): ignore full-viewport chrome containers * test(drag): prove destination on live devices	2026-08-05 12:37:02 +02:00
Michał Pierzchała	76453add71	refactor: pnpm workspace + @agent-device/kernel pilot (#1490 W0) (#1494) ... * refactor: pnpm workspace + @agent-device/kernel pilot (#1490 W0) Extend the workspace with packages/* and move the kernel behind an enforced public API: packages/kernel with nine consumer-earned subpath exports (errors, device, snapshot, contracts, collections, rect, redaction, daemon-error, bounds — the last absorbed from utils as Rect vocabulary). Every kernel import repo-wide becomes the @agent-device/kernel/<sub> specifier; kernel tests move to src/__tests__/kernel/ and exercise the package surface. The root declares the package in devDependencies (workspace:*), tsdown bundles it (noExternal) so the published artifact and its runtime dependency manifest are unchanged. Gate rewiring in the same change, per the W0 brief: - R1 kernel-sink retires (physically subsumed); new R11 package-boundaries guards no-root-back-imports, relative tunnelling past exports maps, undeclared workspace deps, and non-exported subpaths, with runtime resolution pins via import.meta.resolve. - resolveImportEdges and mutation ownership follow workspace specifiers through exports maps, keeping R4 cycle checks, depgraph, and derived test ownership connected across the seam (kernel-errors still owns 495 tests). listSourceFiles includes packages/*/src. - kernel becomes an unranked zone; mutation registry, stryker mutate globs, and the mutation-affected workflow path filter move to packages/kernel/src/errors.ts. - check:affected gains packages/ ownership (manifests fail open); vitest and coverage include packages/*/src; fallow ignores packages/** (its resolver cannot follow workspace specifiers). - The affected-selector CI job installs dependencies: its closure now crosses workspace specifiers, and the R8 relative exception is unsafe for production src files (Node ESM does not realpath, so dual specifier/relative loads would instantiate modules twice). The R8 zero-dep set is pinned empty with that rationale. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix: address W0 review — mutation sandbox, exports-map resolution, tsc -b Review findings on #1494, all five: 1. contracts-schema-public.test.ts reads the kernel source at its packages/ path (fs access invisible to the codemod and typecheck). 2. Mutation lane: Stryker sandboxes the tree but pnpm's node_modules symlink resolves @agent-device/* back to the real repo, so mutants in the sandbox never load and vitest.related finds no tests. vitest.mutation.config.ts now aliases each EXPORTED specifier to its source (derived from exports maps, never a wildcard), keeping resolution inside the mutated tree. Validated: kernel-errors module runs end to end (dry run 3,984 tests, mutants killed, exit 0). 3. Layering/depgraph resolve workspace specifiers through the exports-derived map (workspaceSpecifierTargets) instead of reconstructing paths, so '.'-facade packages resolve; the positional fallback remains only for map-less fixtures (P0 pin). 4. Per-package project references implemented: packages/kernel is composite (emitDeclarationOnly -> dist-types, gitignored), the root references it, and typecheck becomes tsc -b — probed to catch type errors on both sides under TypeScript 7 native. 5. R11's relative-route exception now requires membership in an actual R8 zero-dep job closure (zeroDepClosureFiles walks entries), not mere scripts/ placement — closing the dual-instantiation bypass. Also from review discussion: daemon-error moves out of the kernel package to src/client/ — its consumers (cli, client facade) rehydrate wire DaemonErrors client-side; the daemon only produces them. Kernel drops to 8 exported subpaths before any of them ship. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * refactor: one exports-map reader for mutation alias and ownership Fallow flagged workspaceExportAliases (cognitive 15, CRAP 90). The manifest-reading logic already exists as workspaceSpecifierTargets in scripts/layering/package-boundaries.ts, so both the Stryker sandbox alias table and the mutation ownership walker now consume it instead of carrying near-clones. Behavior unchanged; mutation suite 45/45 and changed-code fallow green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix: composite kernel without a root references edge FreeRange runs plain `tsc -p tsconfig.json`, and a root `references` entry makes non-build-mode TypeScript demand the referenced project's built declarations (TS6305) — a standing "build first" tax on every plain -p consumer (fr, editors). Keep the per-package composite project and build it in typecheck (`tsc -b packages/kernel` before the root and examples/sdk passes), but drop the root references edge: root consumption resolves through exports to source, identical to runtime and to the bundler. Probed: plain -p green with no prebuilt output; kernel-side type errors still caught by its own build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix: R11 uses the layering parser; mutation config is a fallow entry Review blockers on #1494: - R11's private single-quote regex could miss a double-quoted or re-export route into packages/*/src. specifierSites now delegates to the layering model's parseImports (both quote styles, side-effect imports, re-exports, dynamic imports), with direct regressions for each formerly-invisible form. - vitest.mutation.config.ts becomes a declared fallow entry instead of a tolerated unused-file finding: the full-repo audit now reports it reachable (unused files 2 -> 1; the remainder predates this PR). FreeRange clean-checkout evidence: with packages/kernel/dist-types and every *.tsbuildinfo deleted, `pnpm check:freerange` reports 0 findings on this head — the TS6305 topology died with the root references edge in the previous commit; check:freerange has no build precondition. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-07-30 12:12:46 +02:00
Michał Pierzchała	f3e07ff236	test: interaction contract suite with registry-driven coverage gate (ADR 0011 Layer 3) (#1092) ... * fix: preserve the runner's non-hittable fallback marker through direct selector press The direct iOS selector press handler spread successText('Tapped <selector>') after the runner payload, clobbering the 'tapped via non-hittable coordinate fallback' message that directIosSelectorFallbackDetails keys on — so maestroNonHittableCoordinateFallbackUsed could never be true end-to-end (the existing unit test passes because it mocks dispatchCommand above this layer). Found by the ADR 0011 Layer-3 maestro-fallback contract scenario. Share the marker string as MAESTRO_NON_HITTABLE_FALLBACK_MESSAGE and keep it as the success message when the runner reports fallback usage. * test: interaction contract suite with registry-driven coverage gate (ADR 0011 Layer 3) test/integration/interaction-contract/ holds one scenario file per dispatch path, each with a sibling .coverage.ts manifest declaring which guarantee matrix cells it proves (scenario strings double as the vitest test titles). index.ts aggregates the manifests statically, and a new Layer-3 gate in src/contracts/__tests__/interaction-contract-coverage.test.ts fails when any enforced (runtime/runner/delegated) cell lacks a scenario or when a scenario claims a waived/inapplicable cell — coverage of the matrix is by construction in both directions. Path forcing is natural (no test-only env switch needed): selector/ref targets take the runtime path, a tapTarget backend takes the native-ref fast path, x/y takes the coordinate path, and simple-selector clicks on an iOS provider transcript take the direct runner path (the transcript itself proves which path ran via assertComplete). Fixtures are the permanent Bluesky shapes: closed drawer, drawer + visible twin, edge-grazing container, covered button, non-hittable cell. Refs #1081	2026-07-04 16:02:41 +02:00