mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
docs/improvement-audit-tracker
159 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Michał Pierzchała
|
a830ac8df2 |
feat: add Linux command evidence lane (#2017)
* feat: add Linux command evidence lane * fix: assert Linux find result shape * fix: read Linux find result envelope * fix: reset Linux calculator before diff * fix: release Linux session before reset * fix: guard Linux evidence session reset * fix: forward Linux evidence timeout * fix: tighten Linux evidence assertions * fix: preserve Linux replay session identity * fix: close Linux replay session before reset * fix: share Linux evidence daemon state * fix: keep Linux swipe evidence in bounds * fix: keep Linux artifact gap honest |
||
|
Michał Pierzchała
|
d97a628e38 |
fix(ci): make the two rg-based static checks actually run (#2006)
* fix(ci): make the two rg-based static checks actually run
ripgrep is never installed on ubuntu-latest, so both `rg` assertions in
the Lint & Format job failed with "command not found" (exit 127) on
every run. `if rg ...; then ... fi` cannot distinguish that from "no
matches" (exit 1) — both read as false, so each step silently passed
without its assertion ever executing. The DI-seams check had 7 live
violations it never reported.
Rewrite both against `grep`, which every runner ships, with match/
no-match/error exit codes handled explicitly so a broken scan fails
the lane instead of reading as a pass, plus a zero-tracked-files guard
so a renamed directory can't quietly go uncovered.
The DI-seam pattern also gets narrower to drop two classes of false
positive surfaced by actually running it: `typeof fetch` (fetchImpl?/
fetch? seams inject the one global with no module boundary vi.mock can
intercept; auth-session.ts/cloud-profile.ts/daemon-proxy.ts exercise
the seam directly in their unit tests, while CLI-level tests use
vi.stubGlobal('fetch', ...) where the seam isn't reachable — a
deliberate, exercised seam) and `typeof SOME_CONSTANT` in
SCREAMING_SNAKE_CASE (derives a literal union type from a constant,
e.g. interaction-touch-response.ts's dispatchPath field — not an
injectable seam at all).
Fixes #1976
* fix(ci): replace the DI-seam name-based allowlist with an explicit per-site one
Review on PR #2006 (#1976): the previous revision fixed the exit-code
handling but decided which `?: typeof X` matches to ban with a regex
that exempted matches by the *spelling* of the typeof target
(`typeof fetch` always passed, SCREAMING_SNAKE_CASE targets always
passed). That's a name-based semantic allowlist, not ownership: a new,
genuinely test-only `typeof fetch` seam anywhere in the tree would
have silently passed, while an equally legitimate seam under any
other name would still fail.
Add scripts/di-seams: a small, tested TypeScript checker that judges
each match against an explicit, typed, per-site allowlist
(scripts/di-seams/approved.ts) keyed by (file, field name, typeof
target) rather than by name. A triple is exempt only because it was
individually reviewed and named — never because of how it's spelled —
and the gate fails just as hard on a stale approval (one whose triple
no longer matches anything, e.g. after a rename) as on an unapproved
seam, so the list can't silently drift out of sync with the code it
describes.
Moves the DI-seams step in ci.yml to run after Setup toolchain (it's
no longer a toolchain-free text scan); the Swift trailing-comma check
stays where it was.
* fix(ci): register di-seams as a real gate and route it through the tmpdir wrapper
CI caught two things the local (dependency-free) run couldn't:
- oxfmt formatting on the two new files.
- scripts/node-test-tmpdir.test.ts's repo-wide audit: every package.json
script that invokes `node --test` directly must route through
scripts/node-test-tmpdir.ts, or a crash/timeout mid-run leaks its
scratch TMPDIR. check:di-seams now does.
- check:gate-manifest: a package.json script that runs `node --test`
must be covered by a registered CHECK_CATALOG gate, or the audit
reports the test suite as run by no lane. Registered 'di-seams' in
scripts/check-affected/{model,checks}.ts and wired the CI step
through run-gate like every other structural guard in this job,
instead of invoking pnpm directly.
Verified locally with node_modules installed: check:di-seams,
check:gate-manifest, check:gate-manifest:test, check:affected:test,
check:layering, check:fallow (scoped to the changed files), format,
lint, and typecheck all pass.
* fix(ci): close the multiline and duplicate-site gaps in the DI-seam scanner
Review round 2 on PR #2006 (#1976):
- findSeamMatches scanned line by line, so a declaration split across
lines (`field?:` on one line, `typeof X` on the next) was invisible.
Matching now runs against each file's whole source in one pass —
`\s` matches a real newline in JavaScript regexes with no extra flag
needed — with the line number derived from the match's character
offset.
- checkSeams keyed approval by (file, field, target) alone, so once
one occurrence of a triple was approved, any further occurrence of
that same triple anywhere in the file passed too. The key now
includes the line the match starts on, so an approval names one
specific declaration, not a recurring pattern. approved.ts expands
from 5 collapsed entries to the 7 exact sites this closes down to.
Added regression tests planting both gaps directly (a cross-line
declaration, and a second unreviewed fetchImpl?: typeof fetch at a
different line in an already-approved file) and verified both against
the real tree with injected violations, restored cleanly afterward.
Re-ran the full local gate suite (di-seams, gate-manifest, layering,
fallow, format, lint, typecheck) — all green.
* fix(ci): resync approved DI-seam line after merging main
Merging main (#2002) removed an unused import above the approved
dispatchPath?: typeof MAESTRO_COORDINATE_FALLBACK_PATH declaration in
interaction-touch-response.ts, shifting it from line 61 to line 60 —
exactly the location-specific-approval staleness the gate is designed
to catch, just triggered by an unrelated upstream edit rather than a
change in this PR. Updated the approved line to match.
* fix(ci): replace the DI-seam positional table with a code-local approval marker
Review round 3 on PR #2006 (#1976): CI proved the round-2 fix's core
assumption wrong within one push. Keying approval by (file, line,
field, target) made a line number the identity — an unrelated edit
anywhere earlier in a file shifts every approval below it, and that's
exactly what happened: merging main removed an unused import above
the approved dispatchPath declaration, and the gate rejected an
unchanged, already-reviewed line.
Detection is now AST-based (oxc-parser, the same tool
scripts/layering/*.ts already uses) instead of a source-text regex:
any `{ optional: true, typeAnnotation: TSTypeQuery }` node — a
property signature or a bare parameter — is a candidate, which finds
a multiline `field?:\n typeof X` declaration for free instead of
needing a special case for it.
Approval is a `// di-seam-approved: <reason>` comment immediately
above the declaration, matching this repo's own `//
fallow-ignore-next-line complexity` convention: the marker precedes
what it exempts. approved.ts (the external table) is deleted — there
is nothing left to keep in sync, since the approval travels with the
code it approves. A second, unmarked seam under the same field/target
elsewhere still fails; reordering unrelated code around an approved
declaration no longer touches it.
Added the marker to the 7 real approved sites (fetch-global
injection seams in auth-session.ts/cloud-profile.ts/daemon-proxy.ts;
the literal-type-derivation false positive in
interaction-touch-response.ts) and regression tests proving: a
cross-line declaration is still found, a second unmarked occurrence
of an approved field/target pair still fails, and an unrelated
insertion above an approved declaration no longer breaks it. Verified
against the real tree with an injected multi-line unrelated insertion
before an approved site — still green. Re-ran the full local gate
suite (di-seams, gate-manifest, layering, fallow, format, lint,
typecheck, auth-session unit tests) — all green.
* fix(ci): reject a di-seam-approved marker with no reason text
Review round 4 on PR #2006 (#1976): approvalReason() returned '' (not
null) for a bare `// di-seam-approved:` comment with nothing after
it, and checkSeams() only filtered out null, so an empty marker
silently approved a seam with zero justification — exactly the kind
of unreviewed bypass this gate exists to prevent.
approvalReason() now returns null when the joined reason text is
empty after trimming, so a bare or whitespace-only marker is treated
the same as no marker at all. Added tests for both the model-level
behavior and the end-to-end checkSeams() result, plus verified
against the real tree by injecting a bare-marker declaration and
confirming it's flagged, then restored cleanly.
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
Michał Pierzchała
|
02d548dfc9 |
ci: consolidate CI workflow from 15 jobs to 8 (#1996)
* ci: consolidate CI workflow from 15 jobs to 8 Merge single-gate ubuntu jobs into grouped jobs sharing one checkout and install: Lint & Format (plus the static text assertions), Repo Guards (layering/selector/wiring/maestro/mcp-metadata), Compatibility & Provenance (shared fetch-depth: 0 checkout), Typecheck & Package, and Integration Tests (absorbs the web smoke with step-scoped env). Every gate remains an independently named run-gate step; the gate manifest derives lane ownership structurally. Drop the Bun setup from FreeRange: @chenglou/freerange's bin is a plain Node script. It stays GitHub-owned; only the runtime requirement is retired. * ci: fold FreeRange into Repo Guards and skip no-op fixture release jobs FreeRange runs on plain Node now, so its gate joins Repo Guards as the last step instead of occupying its own worker for the slowest guard. The fixture release matrix filters to entries that will actually build, so a cached-fingerprint PR starts zero release runners. * ci: fold host XCTests into the macOS smoke lane and shard Coverage The macOS lane now builds one unit-test-flagged runner bundle that both the host XCTest run and the replay smoke consume, so the host lane no longer occupies its own macos-26 runner behind a separate queue. The host lane's file moves with it, and check:xctest-selection follows. Coverage shards across two runners via blob reports and merges them on a report job that evaluates thresholds once over the full suite and produces every coverage artifact. The tmpdir leak check runs per shard, since a leak lands on whichever runner executed the file. * ci: drop local shard-smoke artifacts from tracking * ci: enforce coverage thresholds only on the merged run A shard evaluates its own half-suite coverage, so the global gate fired per shard. Shards now report without gating; Coverage Report keeps the real thresholds over the full merged suite. * ci: include hidden files when uploading coverage blobs |
||
|
Michał Pierzchała
|
7f3e355426 |
fix(ios): preserve regular snapshot depth through structural wrappers (#1947)
* fix(ios): complete regular snapshot depth frontier * fix(ios): align depth frontier with visibility fold * fix(ios): exercise regular depth frontier in CI * fix(ios): cover visible-depth frontier through public snapshot * fix(ios): tolerate absent deep-link confirmation * test(ios): expose visible-depth fixture hierarchy * test(ios): wait for visible-depth fixture subtree * fix(ios): keep visible-depth fixture minimal * fix(ios): update snapshot hint fixtures * test(ios): avoid fixture label aggregation * test(ios): match fixture raw hierarchy * test(ios): prove visible-depth raw ancestry * test(ios): align depth smoke with AX hierarchy |
||
|
Michał Pierzchała
|
e5bfde3d13 |
diagnose(1874): instrument the synthesized commit wait and add a dispatchable stall loop (#1941)
* diagnose(1874): instrument synthesized commit wait and add stall loop workflow * diagnose(1874): fix empty-array expansion under set -u; raise default iterations * diagnose(1874): add arm64 matrix leg to isolate the Rosetta factor * ci: build the iOS runner for the native arm64 slice A generic simulator destination leaves the active arch undefined; Xcode 26.6 defaults it to x86_64, running the whole runner under Rosetta on arm64 hosts. Pin ARCHS=arm64 across every lane that builds the iOS runner and bump the derived-data cache suffixes. Measured ~30% faster commits on identical CI hardware; delivery-throttle episodes still occur but start from a lower base. * diagnose(1874): keep commit-wait cadence evidence value-free The per-poll trace logged the observed field's contents (prefix(40)) on the shipped type path; that value is user content and runner.log persists. Log lengths and the expected-prefix walk instead, allowlist every string-interpolating NSLog format in the module behind a source-scan guard, and pin commonPrefixLength in the host-lane policy tests. * diagnose(1874): narrow the log-format match for typecheck * diagnose(1874): route cadence evidence through a typed value-free boundary logCommitCadence accepts Int lengths and a timestamp only, so observed field contents are unrepresentable at the poll call site; its emitted line is pinned by a sentinel-secret test in the host-lane policy tests. The source guard becomes structural — boundary present, poll path logs through it, no raw NSLog in the observe closure — instead of parsing Swift format strings. #1874 is reopened as the removal-tracking thread for this temporary instrumentation. |
||
|
Michał Pierzchała
|
991c08561b |
fix(ios): enforce regular snapshot clip invariant (#1946)
* fix(ios): enforce regular snapshot clip invariant * fix(ios): restore typed snapshot failure construction * fix(ios): linearize snapshot clip validation * fix(ios): propagate snapshot presentation errors * fix(snapshot): clarify presentation failure recovery |
||
|
Michał Pierzchała
|
17da776350 |
feat(ios): add snapshot backend conformance (#1930)
* feat(ios): add snapshot backend conformance * fix(ios): load built SDK at live runtime * test(client): isolate snapshot forwarding regression * refactor(snapshot): keep backend capability metadata internal * fix(test): merge backend conformance imports * fix(snapshot): keep backend forcing internal * refactor(snapshot): isolate backend capability fixtures * refactor(snapshot): keep capability governance internal * fix(ios): align snapshot actionability contract |
||
|
Michał Pierzchała
|
30de1597d3 |
ci: attribute native package size and trim Apple runner (#1934)
* ci: attribute npm package size by shipped component * refactor: modularize size reporting and trim Apple runner * ci: preserve size reporter modules across base checkout |
||
|
Michał Pierzchała
|
07023eb202 | fix(ios): separate snapshot actionability from occlusion (#1933) | ||
|
Michał Pierzchała
|
d57aa69777 |
test: add macOS platform command coverage manifest (#1922)
* test: add macOS platform command coverage manifest * fix: remove unused macOS coverage type exports * test: route macOS coverage away from iOS lane * fix: account for host-dependent macOS audio capability * fix: run macOS coverage manifest in CI |
||
|
Michał Pierzchała
|
af96c6608d | feat(ios): publish effective snapshot geometry (#1931) | ||
|
Michał Pierzchała
|
73db7be2ff |
feat(ios): move the regular-projection clip fold into snapshot presentation (#1797) (#1929)
* feat(ios): move the regular-projection clip fold into snapshot presentation Both iOS snapshot backends carried their own copy of the visibility fold: the tree walker and the private-AX serializer each computed viewport-and-scroll-clip intersection, ancestor projection, hidden-content hints, and collapsed depth during acquisition. Hand-synchronized copies of that interpretation are what produced the scroll-overflow leak class (#1784), and C1 (fact-availability neutrality) could not hold while acquisition decided what a screen shows. Acquisition backends are now fact serializers: every traversed node is emitted at raw traversal depth with its reported frame, and SnapshotAcquisition carries the viewport. presentRegular runs the one clip fold for every backend -- viewport ∩ scroll clip, the ancestor cursor (an out-of-clip Cell or scroll container hides its clamped descendants), the sub-pixel decoration rule, scroll hints booked onto anchors, reparenting with collapsed depth -- and narrows the emitted hittable to the clip: nothing outside its clip, and nothing without geometry, is ever hittable, whatever the backend reported. Platform differences are a SnapshotFoldPolicy input to the shared algorithm (iOS cursor-projected; macOS/tvOS plain viewport), never a backend exception. The private-AX backend collapses to ONE serializer for both projections, and the flat filter-decision family dies with the acquisition gates it fed. Three intentional edge deltas, each toward one backend-neutral rule: sub-pixel content-free decorations now drop on every backend (was private-AX only); labeled offscreen Application/Window carriers survive on every backend (was tree only), never hittable; query-sweep regular without -i is viewport-folded. Declared acquisition residues: the traversal-depth budget cut, the sweep's frameless-element drop, the private-AX bridge's device-side cap. Refs #1797 (migration step 3, clip-fold delta). * refactor(ios): isolate snapshot visibility fold |
||
|
Michał Pierzchała
|
2a9a4ee80c |
test: add Linux platform command coverage manifest (#1921)
* test: add Linux platform command coverage manifest * test: address platform coverage review feedback |
||
|
Michał Pierzchała
|
4137e4275e |
feat(ios): split raw and regular snapshot projections behind one capture hint (#1926)
The private-AX backend interpreted `--raw` as the regular projection: it folded the viewport and scroll clips and dropped sub-pixel decorations before returning, so a raw capture that recovered onto it answered with viewport-pruned nodes labeled raw (#1797 D4). Nothing related the two copies of that decision. Presentation now exposes the two projections it always implied. `presentRegular` folds visibility, eligibility, scope and scroll hints; `presentRaw` is the acquired tree, normalized, narrowed only by a scope or depth the request asked for. Acquisition reads one derived `CaptureHint` instead of the request itself, so what a capture may skip is stated once, beside the reason skipping it keeps the projection complete: scope and its relative depth never narrow, raw depth does (raw depth is traversal depth), and the raw projection never carries `interactiveOnly` — `--raw -i` is the acquired tree. Two structural rules replace the hand-synchronized ones. The raw plan is derived from `SnapshotBackendKind.supportsRawProjection`, so the query sweep — an interactive element query with no hierarchy to return — cannot be planned for a raw request. And presentation compares the requested projection with the hint the acquisition was captured under, dropping that tier with a structured `IOS_SNAPSHOT_PROJECTION_MISMATCH` failure rather than presenting it under the requested label. Declared residue: a regular `--depth` request still cuts the traversal at that depth while regular presentation emits collapsed depth, so a node that would present within the limit can be dropped. The cut is what keeps `--depth 1` probes cheap; making it complete is the open visible-depth frontier obligation. Refs #1797 (migration step 3, raw-projection delta). |
||
|
Michał Pierzchała
|
06d27de4d0 |
test(gesture): assert pan duration in the iOS gesture-lab replay (#1901)
* test(gesture): assert pan duration in the iOS gesture-lab replay (#1584) The only replay exercising the `gesture pan` command class that regressed in #1562 asserted a counter, which stays green even if the requested duration collapses — nothing in CI could catch the regression coming back. Record an observed-duration bucket from a single-pointer Gesture.Pan's begin/end timestamps in GestureLab.tsx (iOS-only, so Android's raw-touch transform handling in the same shared component is untouched), render it as plain text, and assert it with a one-line wait in gesture-lab.ad. No runner protocol changes needed. * style: fix oxfmt line-wrap in GestureLab.tsx * ci(ios): run the pan-duration canary automatically on every PR gesture-lab.ad (and its new duration assertion) only runs under full:fixture-replays, which is currently dispatch-only in replays-manual.yml — the PR-triggered ios.yml lane runs the smoke tier, and replays-nightly.yml no longer carries device replays at all (#1781 A1). So the #1584 guard could not actually catch a regression automatically. Split the duration check into its own minimal, isolated replay (gesture-pan-duration.ad) and run it as a smoke-tier step in ios.yml, so it's cheap and doesn't depend on gesture-lab.ad's multi-touch commands, which stay full-tier only. * test: require pan recognition in duration canary |
||
|
Michał Pierzchała
|
40e4b0dd3e |
docs(agents): restore and enforce progressive disclosure (#1888)
* docs(agents): restore and enforce progressive disclosure * test(maestro): pin typed selector fallback signal * docs(agents): address progressive disclosure review * docs(agents): restore orphaned traps and close guidance-gate bypasses - AGENTS.md: skills carry a minimal start/routing card; command semantics stay in versioned CLI help (the skills contract enumerates two skills by hand, so prose retains ownership for the rest) - testing.md: restore the two local-only XCTest snags CI never hits (unsigned-bundle policy refusal signature + first-run automation permission) - scripts/gate/routing.ts: record GitHub's 300-changed-file path-filter limit at the paths-ignore assertion it bounds - agent-guidance-contract.test.ts: recurse docs/agents so nested guidance cannot evade the byte budgets while the gate stays green |
||
|
Michał Pierzchała
|
b4331815e2 |
test: add web platform command coverage manifest (#1902)
* test: add web platform command coverage manifest * fix: preserve web coverage report on cleanup failure |
||
|
Michał Pierzchała
|
e65443d774 |
ci: bound the Linux apt install so a stalled mirror fails fast (#1887)
Unbounded, the desktop-dependency install could not fail, only stall. On 2026-08-19 a slow package mirror held apt past the job's 30-minute budget on four main-branch runs and several unrelated PRs, cancelling each job before Setup toolchain, Xvfb/D-Bus, or the replay smoke test ran — a red check on branches that never executed a line of project code. timeout-minutes: 6 turns that into a named step failure in six minutes instead of a cancelled job at thirty; a healthy install takes about a minute. The apt options cover the transient cases without a retry loop layered on top of them: socket timeouts bound a mirror that connects and then goes quiet, Acquire::Retries absorbs a blip, and DPkg::Lock::Timeout bounds the runner's own unattended-upgrades timer, which stalls identically and is a plausible alternate cause of the same symptom. Tradeoff: a fast transient failure that apt's own retries miss now fails the job rather than self-healing, traded against carrying a bash retry loop in CI. |
||
|
Michał Pierzchała
|
9ce1ef7587 |
feat(snapshot): move scope into presentation (#1855)
* feat(snapshot): move scope into presentation Use one preorder label/identifier/value policy across Swift and TypeScript, keep scoped iOS acquisition conservative, and remove the daemon's second scope pass. Non-vacuity: label-only matching failed identifier/value parity fixtures; Android pass-through failed its boundary test; disconnecting Swift applyScope produced eight scope/depth/projection failures. * fix(snapshot): select scopes with presented content * docs(snapshot): describe presentation-owned scope * docs(snapshot): record contribution-aware scope * refactor(snapshot): drop unrelated provider churn * fix(snapshot): accept healthy empty scoped capture * refactor(snapshot): isolate empty-scope admission * fix(snapshot): align scope ownership across runtimes * test(snapshot): pin post-wire scope owner * test(snapshot): retain find test shrink |
||
|
Michał Pierzchała
|
80b4769230 |
test(fuzz): structured CLI/Maestro generators that reach command validation and assert error codes (#1781 B2) (#1866)
* test(fuzz): structured CLI/Maestro generators that reach command validation and assert error codes (#1781 B2) * test(fuzz): pin the rediscovered #1433 excess-positional case and keep numeric flag samples inside their range * style: apply oxfmt to the new fuzz modules * perf(fuzz): derive the CLI validation surface lazily so unrelated harness paths keep their startup * test(fuzz): resolve validation generators in the run path so corpus replay keeps its small module graph * test(fuzz): weight the CLI budget toward command validation, pin the finite classes as seeds, guard lazy surface derivation * docs(testing): describe the validation lane's layer split, seed-pinned classes, and PR-time gates * refactor(fuzz): split the validation generator into CLI and Maestro modules, mirrored in tests * refactor(fuzz): collapse the flag-shaped mutation classes and seed literals, derive class coverage from declarations * fix(fuzz): hash every case-generation module in configHash, guarded by an import-closure test * test(fuzz): assert CLI command and flag-key coverage against the registry, and close the six gaps it found |
||
|
Michał Pierzchała
|
393eb30a28 |
ci: give check:affected real Apple ownership rules and route ios.yml on them (#1781 A9-2) (#1857)
* ci: give check:affected real Apple ownership rules and route ios.yml on them (#1781 A9-2) Device-lane ownership by platform family in the affected selector (scripts/check-affected/device-lanes.ts): a TypeScript-only Apple change now carries replay-ios/replay-ios-device/replay-macos in a narrow plan, other families own only their own lanes, shared runtime surface owns every lane, unit tests own none. Golden tables (contracts/fixtures) own the parity unit test and both runner builds instead of failing open. ios.yml pull_request paths-ignore is routed on that ownership; the gate manifest asserts the list against the selector over every tracked path both ways (scripts/gate/routing.ts, ROUTED_LANES). push to main is unfiltered. Path coverage exempts declared manual-only checks the way owned does. * ci: tighten routing assertion shape (fallow: unused exports, complexity) * ci: name parked checks in check:affected --run skips * ci: bound the routed-lane exemption to sibling workflows (review of #1857) The exact-name .github exemption was unbounded: naming the lane's own setup-apple-runner-build or boot-ios-test-simulator action skipped the lane that runs them and the manifest stayed green. Lane now carries the transitive composite-action closure plus its own workflow file (Lane.uses, same walk declaredGates does), and the exemption refuses anything in it. Also: an unowned path under an ignored root (a non-TS fixture under a family root) asked for the ignore entry to be removed, which would un-route every sibling in that tree; it now asks for a selector owner. Both cases pinned, both proven red against the pre-fix code. Documents GitHub's 300-changed-file path-filter limit in docs/agents/testing.md. * ci: close the routed-lane exemption over composite-action support files Lane.uses recorded only each composite action's action.yml, so a support file the descriptor executes was exemptible as if it were an unrelated sibling workflow: ios.yml uses setup-fixture-app, whose action.yml runs "$GITHUB_ACTION_PATH/fetch-artifact.sh", and that script runs its siblings resolve-artifact-name.sh and trusted-artifact.mjs — references that exist only inside shell, one level past anything YAML parsing sees. The closure unit is the action's directory now. It needs no shell model and cannot miss a file however deep the reference chain runs; the coarseness is harmless because a file in an action's own directory belongs to that action. All three files pinned, red against the descriptor-only closure. |
||
|
Michał Pierzchała
|
d07b837621 |
test: classify the runner XCTests — pure decisions to a macOS host lane, simulator semantics gated os(iOS) (#1781 A7) (#1861)
Every declared AgentDeviceRunnerUITests method now belongs to a lane, and the #if guard is the classification: AGENT_DEVICE_RUNNER_UNIT_TESTS alone means a pure runner decision (runs on the macOS host on every PR — ci.yml's existing compile job now executes the bundle it builds), '&& os(iOS)' means runner/XCTest semantics (simulator lanes only). check:xctest-selection evaluates the guards per platform, derives each lane's reach, and fails on a flagged identifier that is undeclared or uncompiled on that lane, on a declared test no lane reaches (found the two tvOS-only tests, dark since birth — widened to os(tvOS) || os(macOS)), and on testCommand reaching any lane. The host and nightly lanes assert executed == derived reach, so a missing -D flag or a guard that compiles a file out reads red, not as a smaller green. One duplicate test deleted (sparse-verdict assertions folded into its twin). |
||
|
Michał Pierzchała
|
ee13203a16 |
feat(ios): unify snapshot eligibility (#1850)
Make iOS regular snapshot eligibility one backend-neutral presentation rule. Acquire tree nodes conservatively, preserve interactive scroll containers, normalize surviving hierarchy, and keep raw membership plus daemon publication policy unchanged. Part of #1797. - iOS and macOS unit-enabled runner builds - 2 focused XCTest cases - 3 production-path publication tests - live Settings snapshots: 73 regular nodes and 167 raw nodes, both healthy tree captures |
||
|
Michał Pierzchała
|
f03c0309a1 |
fix: derive iOS transition snapshots from visible presentation (#1831)
* fix: project iOS transition semantics * fix: derive iOS transition semantics from visible state * fix: preserve iOS presentation context for scoped snapshots * fix: confirm broad iOS transition settlement * ci: run coordinate input regression on pull requests * test: mock migrated snapshot capture seam * fix: confirm transitions across snapshot backends * fix: arm transition confirmation after first capture * fix: settle against immutable action baseline |
||
|
Michał Pierzchała
|
423927fdd8 |
chore(mutation): shrink to report-only — drop the ratchet, baseline and graduation (#1457, #1781) (#1828)
* chore(mutation): shrink the lane to report-only (#1457, #1781 wave 2) The mutation harness's two real catches (#1474, #1475) both came from humans reading the weekly score report. The ratchet half never operated: the baseline was committed exactly twice ( |
||
|
Michał Pierzchała
|
9d6154eecb |
ci: park perf-nightly to dispatch and stop the coverage-gate cascade double-red (#1781 A3, A5) (#1822)
A3: perf-nightly writes a report and compares nothing, so it structurally cannot catch a regression. iOS wall-clock medians swing up to +122% night-to-night at n=5 (a comparator would print noise), no doc/issue reads the report, and the iOS job holds a macOS runner ~22min nightly. Parked to workflow_dispatch following the #1781 A1 pattern (replays-manual.yml); it declares no gate-manifest check, so no declarations.ts change is needed. `pnpm perf` / scripts/perf are untouched. A5: the "Enforce changed-line coverage gate" step ran `if: always()`, so when the preceding "Run coverage" step failed, lcov.info was never written and this step failed too with "no lcov report" -- a cascade double-red, not a coverage verdict. 16 of the last 17 red instances (60d) were this cascade; the step now runs only when Run coverage succeeded. |
||
|
Michał Pierzchała
|
8300fa131e |
refactor(ios): establish snapshot presentation seam (#1845)
Introduce RawAXNode and PresentedNode so acquisition backends can no longer construct the wire-facing snapshot shape directly. Preserve current output while #1797 moves semantics behind the seam. Non-vacuity: setting PresentedNode.label to nil made testSnapshotPresentationPreservesCurrentWireShape execute once and fail on the missing label field; restoring the production mapping made the same focused XCTest pass. |
||
|
Michał Pierzchała
|
ef6ec2995b |
chore(layering): document R12/R18/R19, retire R8, make R9 shrink mandatory (#1781 A6) (#1825)
* chore(layering): document R12/R18/R19, retire R8, make R9 shrink mandatory (#1781 A6) The A6 review kept `check:layering` in full (15/15 planted violations fired, no other enforcer exists) and left four follow-throughs. R12 bin-alias-fast-path, R18 contracts-implementation-authority and R19 selector-pipeline-ownership were live rules with no ADR or CONTEXT anchor — they now carry one each, in the same list as R7/R9/R10/R13. R8 zero-dep-job-closure is retired: no CI job sets `install-deps: false` and ci.yml records why each keeps it enabled, so the invariant has no subjects. R11's relative-into-packages exception existed only because a zero-dep closure cannot coexist with specifier loads, so it retires with R8; the route is now closed to every caller. R1 was retired the same way at #1490. R9 was growth-only and merely suggested lowering the ceiling, which is headroom the next change spends without a number moving. It is now an equality pin like R6 and the R10 R7 counts, and the committed baseline drops 47 -> 46 (daemon-server ceiling 17 -> 16) to match the measurement. ADR 0019 §6 now says each runtime-command-cutover row is deleted when that command's migration is declared closed. * chore(layering): rename R9 to type-cycle-size now that it fails both ways (#1781 A6) |
||
|
Michał Pierzchała
|
4b44c1c53a |
chore(test): remove the contention retry and shrink the subprocess-stub project (#1781 A4) (#1827)
The enumerated single-retry policy (#1419) has fired zero times since it landed on 2026-07-29: 0 of 234 sampled Coverage-job lane envelopes (2026-08-11 to 2026-08-18) have retryCount > 0, and none of 17 recent failed runs was retried (5 refused "outside the enumerated retry list", 4 refused "unhandled error"). All three trackers its entries pointed at (#1098, #1414, #1419) are closed. It cost ~1,454 LOC, a per-run secret marker threaded through a setup file on every Vitest project, and a standing obligation for every future gate reporter to call the blocker bus. Delete the scripts, tests and fixtures, the check:contention-retry script and gate, the envelope artifact upload, and the runner-timeout setup file; test:coverage:ci is a plain `vitest run --coverage` again. lane-envelope.ts stays: the mutation, fuzz and concurrency-torture lanes build their envelopes from it. run-blocker-bus.ts goes: its only consumer was the retry's failure sink, and its only publisher already fails the run by setting process.exitCode. Keep the subprocess-stub project for the three files that really spawn (client-metro, fuzz harness, fuzz corpus-replay) and drop the three that run in 31/212/277ms in CI, which cannot contend for anything. The list is now a plain array in vitest.config.ts with the reason at each entry. Membership and the project's kill criterion live in #1823. Because test:coverage:ci is a bare vitest run, the gate manifest reads its projects directly, so OPAQUE_RUNNERS no longer needs it and an unrun Vitest project becomes unrepresentable rather than detected; the audit test now constructs that state by project-scoping the script. |
||
|
Michał Pierzchała
|
f843dc2df1 |
fix(scroll): keep saturated scroll gestures out of the status bar; gate Android replays from android/emulator (#1781 A1) (#1820)
* fix(scroll): keep saturated scroll gestures out of the status bar; gate Android replays from android/emulator (#1781 A1) `pnpm gate replay-android` failed 4/8 whenever it ran after the full-tier Android E2E (replays-nightly run 32107665052, job 95620294899): 05-app-lifecycle, 06-swipe-gestures and both fixture replays diverged under "A system surface covers the app". The E2E was not the cause. Reproduced on a pixel_7 / API 36 AVD with the same cutout geometry CI's `avdmanager --device pixel_7` produces (status bar 136px, not the 63px of a plain 1080x2400 skin): - `03-scroll-discovery.ad` runs `scroll up 3`. The scroll planner clamps travel to the viewport minus a 5% band, so the touch-down landed at y=120 — inside the 136px status bar — and pulled the notification shade instead of scrolling. On API 36 the app window is edge-to-edge, so the reported viewport starts at y=0 and includes that bar. - The shade then covered every replay until `04`'s `back` closed it. Native readdir order on the runner (03, 05, 06, fixture/02, fixture/01, 04, 01, 02) put four files in that window; the last green run (2026-07-30) had 04 right after 03, so the pull was masked. Fix in the product, not the lane: DEFAULT_EDGE_PADDING_FRACTION 0.05 -> 0.1 in the TS scroll planner and its Swift port. Every real Pixel has a cutout (5.7% of a Pixel 7's height) and an iPhone's Dynamic Island status bar is 6.9%, so any saturated `scroll up` opened the shade / Notification Center for real agents too. Parity vectors updated in both suites plus a Pixel 7 regression vector (1080x2400, amount 3 -> touch-down y=240 > 136). Second contamination the same order exposed once the shade was gone: `fixture/02-selector-routes-covered-diagnosis.ad` is a #1715 reproduction recipe that FAILS BY DESIGN at step 9 (covered-target refusal) and leaves the device in landscape, yet the gate enumerated `test/integration/replays/android` recursively. iOS keeps gate replays in `replays/ios/simulator` and fixture recipes in `replays/ios/fixture`; Android now mirrors that: the six Settings replays move to `replays/android/emulator`, `test:replay:android` points there, and `fixture/` stays E2E-owned (`full:fixture-replays` already runs 01 by path). android.yml and the workflow-evidence fixture follow the path; the replay-compat manifest keeps the historical paths it pins at released tags. Verified live (Pixel 7 geometry, API 36, --retries 0): control run at main head in CI order reproduces exactly CI's 4/8; with the fix, `pnpm gate replay-android` 6/6 in both native and CI order, and `03` leaves Settings on screen (scroll up 3 now touches down at y=240). * test(scroll): drive the TS and Swift scroll-plan parity vectors from one table (#1820 review) The two suites hand-mirrored the same vectors and #1820 had to edit both by hand — the drift class the repo already closes for the tap-point rule via contracts/fixtures/tap-point-policy.json. The scroll vectors (plus both planner constants, pinned behaviourally on a 1000px axis) now live in contracts/fixtures/scroll-gesture.json; scroll-gesture.test.ts and RunnerTests+ScrollGesture.swift iterate it. Verified: vitest 10/10; the four XCTests run on an iOS 26.2 simulator with the unit flag on (Executed 4 tests, 0 failures). Also: test/ci/android-workflow-evidence.json says what it guards. Follow-up for content-safe viewport bounds + discovery order: #1821. |
||
|
Michał Pierzchała
|
142d156338 |
ci(ios): run the full XCTest suite nightly and check the PR test list (#1781 A7) (#1789)
* ci(ios): run the full XCTest suite nightly and check the PR test list (#1781 A7) * fix(ci): skip the runner server entry point in the nightly and validate both test flags * docs(ci): restate the nightly lane cost and timeout honestly * docs(ci): stop quoting XCTest counts that drift between commits * ci(ios): tighten the nightly timeout to the measured suite duration |
||
|
Michał Pierzchała
|
8b0560a51d | fix: hide clamped descendants of offscreen iOS rows (#1811) | ||
|
Michał Pierzchała
|
ccf64f6797 |
ci: move parked device replay suites to a dispatch-only workflow (#1781 A1) (#1794)
* ci: move parked device replay suites to a dispatch-only workflow (#1781 A1) Both full-tier device jobs have failed every scheduled run since 2026-07-24: the Android suite inside full-tier scenarios that had never executed end to end, the iOS suite on varying steps. They move to .github/workflows/replays-manual.yml, which has no `schedule:`, so the schedule stops emitting a guaranteed failure while the suites stay runnable on demand. A job-level `if: github.event_name == 'workflow_dispatch'` would have looked the same and lied: `workflowLanes()` decides `qualifying` per workflow FILE and never reads job-level `if:`, so the manifest kept reporting replay-android, replay-ios, and replay-ios-device as scheduled-lane owners — the silent-owner-loss failure the manifest exists to catch. A separate file is what the file-level model already reads correctly. Those three checks now have no pull_request/schedule owner, so they are declared as MANUAL_ONLY_OWNERS rather than folded into UNPROVABLE_OWNERS, whose claim ("it runs, this loader cannot see it") is no longer true for replay-android. check:gate-manifest drops from 48 to 46 wired checks and names the three on every run. Two tests pin it: a dispatch-only lane is non-qualifying however many gates it declares, and every manual-only declaration must name a registered check that no qualifying lane owns, so a re-scheduled lane cannot keep a stale exemption. * ci: attest manual-only checks against their dispatch lane (#1781 A1) Review P1: MANUAL_ONLY_OWNERS was a negative allowlist — it proved each entry named a registered check no qualifying lane owned, but nothing tied the entry to a lane that can still run it. Deleting a parked job, or its run-gate step, would have left the manifest green and still printing the check as manual-only: parked coverage silently becoming deleted coverage. Each entry now names its dispatch lane, and a new 'manual-only' audit assertion resolves that name against the derived model: the lane must exist, must still be dispatch-only, and must still declare the gate. replay-android carries an explicit `opaque` flag because its gate sits inside the third-party emulator action's `script:` (#1429), so the job's existence is the whole attestation the model can make — and the flag says so rather than letting an unreadable lane look like a declaring one. Four regressions pin both directions: deleting a declaration reports the check as unowned; deleting the parked job fails with 'no workflow defines'; re-scheduling the lane fails until the entry is dropped; and a parked lane that loses its run-gate step fails unless the entry is opaque. * ci: make manual-only mean dispatch-only, not merely non-qualifying (#1781 A1) Review follow-up: the attestation checked `qualifying === false`, which is true of any lane that is not pull_request/schedule. Swapping `workflow_dispatch` for `push` in replays-manual.yml would have kept the audit green and the checks printed as manual-only, while the runs nobody starts by hand quietly started themselves on every push. The lane model now keeps the trigger names instead of collapsing them into that one bit, and the manual-only assertion requires `workflow_dispatch` and nothing else. Three planted regressions cover the gap the review named: a parked lane re-triggered by `push` fails, a parked lane with no trigger at all fails, and the loader test pins that trigger kinds survive into the model (a push lane reads `[push]`, the nightly reads `[schedule, workflow_dispatch]`). |
||
|
Michał Pierzchała
|
f45228ae71 |
ci: skip device lanes for root-level docs-only changes (#1781 A9) (#1791)
* ci: skip device lanes for root-level docs-only changes (#1781 A9) Add AGENTS.md, CHANGELOG.md, CONTEXT.md, CONTRIBUTING.md, LICENSE, and SECURITY.md to the pull_request paths-ignore block in ios.yml, android.yml, linux.yml, macos.yml, ci.yml, and size.yml. These root-level docs files were the only gap left after docs/**, website/**, and README.md — PRs #1568 (SECURITY.md only), #1697 (CONTEXT.md + docs/adr only), and #1722 (AGENTS.md + docs/) each still triggered a full 9-15 min macOS iOS run despite touching only prose. Why each file is safe to ignore for every one of these six workflows: - None of the four device workflows (ios/android/linux/macos) or their composite actions read any of these six files at runtime; the only hits from `grep -rln` across scripts/, src/, test/, and .github/actions/ are prose comments pointing humans at CONTEXT.md or AGENTS.md sections (e.g. scripts/layering/check.ts, scripts/wire-compat/run.ts, src/mcp/tool-ref-pins.ts) — never an `fs.readFileSync`/`readFile` of the file itself. - The check-affected selector (scripts/check-affected/model.ts) already classifies all six as pure docs: `isDocs()` matches any `.md` file plus the literal `LICENSE`, and `docsOwnership()` only special-cases `website/docs/docs/commands.md` (unrelated). So these files already select zero checks — they only ever produced `docsOnlyPaths` entries, never `SelectionReason`s. - Because they select zero checks, the gate-manifest's path-coverage category derivation (`scripts/gate/model.ts` `categories()`, which iterates `plan.reasons`) never records a category for them, so ci.yml has nothing check-manifest-only that these six files would need to keep reachable. `pnpm check:gate-manifest` and `pnpm check:gate-manifest:test` both stay green after the change (48 checks / 33 lanes, 28/28 gate tests passing). - size.yml's bundle-size job (scripts/size-report.mjs) measures the `pnpm build` dist output and startup timing only — no reference to any of these six files. (npm packs LICENSE/README.md into the publishable tarball, but that's a `pnpm check:package` node-22.12 concern in ci.yml's packaged-cli job, which is driven by `dist` contents and `package.json`, not by LICENSE/README prose — already evidenced by README.md being ignored here since before this change.) Scope disclosure: `mutation-affected.yml` uses a `paths:` allowlist (not paths-ignore) so it's structurally unaffected; `test-app-build-cache.yml` has no path filter at all. Neither was touched. actionlint and `pnpm check:gate-manifest`/`:test` pass on the changed workflows. * test: pin root-doc paths-ignore entries with a regression test Addresses review feedback on #1791 from thymikee: the docs-only classifier for AGENTS.md/CHANGELOG.md/CONTEXT.md/CONTRIBUTING.md/ LICENSE/SECURITY.md across ios.yml/android.yml/linux.yml/macos.yml/ ci.yml/size.yml had no regression pin. Neither check:gate-manifest (only proves a *registered check* is reachable) nor actionlint (only validates YAML shape) nor generic Markdown coverage would catch a single dropped entry — e.g. LICENSE reappearing in one workflow's paths-ignore list but not another's would silently put a full 9-15 min device run back on prose-only PRs. test/ci/root-docs-paths-ignore.test.ts parses the six real workflow files and asserts, using the same matchesGlob the gate-manifest model uses to decide lane triggering, that each of the six root docs is ignored by each workflow's pull_request paths-ignore. Registered in vitest.config.ts's unit-core project next to its sibling upload-agent-device-artifacts.test.ts (parse-only, no device/subprocess lane needed). Verified red on main (all 36 file x doc assertions fail — confirmed via a throwaway script reading `git show main:.github/workflows/*.yml`) and green on this branch (6/6). Full unit-core project (873 files / 6641 tests) still passes; check:gate-manifest and check:gate-manifest:test unchanged (48 checks / 33 lanes, 28/28). |
||
|
Michał Pierzchała
|
0d3b7413c5 |
fix: prevent private AX subtree leaks at source (#1807)
* fix: prevent private AX subtree leaks at source * fix: preserve values in settle signals * fix: normalize settle signal semantics |
||
|
Michał Pierzchała
|
302019563c |
ci: drop duplicate nightly-macos replay job (#1781 A2) (#1787)
The nightly-macos job in .github/workflows/replays-nightly.yml is byte-identical in substance to smoke-macos in .github/workflows/macos.yml: same setup-apple-runner-build call, same macos-helper gate, same replay-macos gate invoked with --retries 2 --report-junit test/artifacts/replays-macos.junit.xml. macos.yml already runs this gate on every pull_request and every push to main, so the nightly copy adds no additional information — 0/40 nightly runs have ever failed independently of the PR/push gate. Verified: check:gate-manifest and check:gate-manifest:test still pass against the trimmed workflow (no gate lost its CI owner), and the full check:tooling suite is green. |
||
|
Michał Pierzchała
|
8b698e8efc |
fix: stabilize private AX settle snapshots (#1784)
* fix: stabilize private AX settle snapshots * fix: harden private AX settling |
||
|
Michał Pierzchała
|
75a4817789 | fix: install dependencies for iOS runner releases (#1785) | ||
|
Michał Pierzchała
|
9c22467832 |
refactor(ci): make gate ownership structural (#1429) (#1753)
* test(ci): prove every registered gate is owned and reachable (#1429) A check that silently stops running looks exactly like a green build. Two suites had already stopped: `check:tmpdir-leaks` (with its model tests) and `test:fixture-cache` are real package scripts that no workflow ran, reachable only through the `check:unit` aggregate CI never invokes. `CHECK_CATALOG` becomes the registry of every check and `pnpm gate <id>` the only way CI runs one, so finding what a lane runs is a scan for `pnpm gate` rather than an attempt to interpret shell. `pnpm check:gate-manifest` then asserts against the real workflows that every registered check is run by some qualifying lane (per unit, not per script name), that every check the real selector activates for a path is run by a lane that path would start (#1420's class), and that every Vitest project and suite script belongs to a check. The wiring that keeps those honest is asserted too: a gate id must name a registered check, an `if:` must be ruled on in GATE_CONDITIONS so `if: false` unowns what it guards, an action declared to run a gate is proven to, and a job whose steps the loader cannot open fails closed. It deliberately does not try to prove CI runs project code only through `pnpm gate`. Whether a shell block executes project code is not decidable from its text, so shell this model does not recognise earns no ownership credit — the failure direction is a check reported unowned, never one waved through. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * test(ci): update the two suites that assert on rewired workflow text `scripts/mutation/workflow.test.ts` and `test/ci/trusted-fixture-artifact.test.mjs` read the workflow and action files and assert on their command text, so routing those steps through `pnpm gate <id>` moved what they were matching. They are the two suites the manifest cannot help with: it proves a gate is still run, not that a test asserting on how CI spells a command was updated with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * fix(ci): credit gates by execution shape, and keep every guard Three ways the manifest could report a gate as owned when it does not run. 1. Crediting was a substring scan over `run:`, which #1429 explicitly rules out — "do not infer reachability from a command name merely appearing in workflow text". `false && pnpm gate x`, a gate inside `if false; then … fi`, one named in a heredoc, and `echo pnpm gate x` all credited it. There is a live instance: conformance-regenerate.yml's "Fail if regeneration changed anything" step names `pnpm gate maestro-regenerate` inside an error message telling a human to run it, and that credited the gate. A gate now counts only as the first command segment of a line, and a body carrying shell structure earns nothing. Reachability inside a script is not decidable, so this does not try: unrecognised shape means no credit and the check reports unowned. `VAR=$(pnpm gate x …)` is read, since the assignment form is unambiguous and the gate runs. 2. Job-level `if:` was not modelled at all, though six live jobs carry one, so a job that cannot run still credited every gate inside it. Two conditions on the mutation lanes are now declared. 3. A caller's `if:` REPLACED the guard on a nested composite-action step (`guard[0] ?? step.condition`), so an outer `always()` erased an inner `if: false`. Steps carry every guard between the lane and the step. Also corrects two source comments that still claimed project code run outside the runner fails the manifest. It does not: such a step earns no credit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * ci: add the run-gate action that names a gate structurally The seam the ownership proof will read instead of shell. A lane says which gate it runs in `with.gate`, a typed input the manifest reads straight out of the YAML and validates against CHECK_CATALOG. Nothing here is wired yet — the ~60 call sites and the model change follow. Added first so the target of that conversion is reviewable on its own. `args` cannot select which gate runs; it is appended after the id, so the worst a wrong value does is fail the gate it already named. There is no `|| true` and no output capture: the gate's exit code is the step's exit code, so a gate cannot run without being able to fail its lane. Part of #1429. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * merge: main (#1770) and route its three new steps through the runner #1770 landed the orphan-check fix on main, wiring `check:tmpdir-leaks`, `check:tmpdir-leaks:test` and `test:fixture-cache` into Coverage, Layering Guard and Integration Tests. This branch had wired the same three through `pnpm gate`, so the merge produced two steps per check rather than a conflict — each check ran twice. Kept main's steps, with the placement and reasoning reviewed on #1770, and changed only their `run:` line to the canonical runner. Dropped this branch's duplicates. Net effect on CI is unchanged: the same three checks, in the same three lanes, once each. Gate manifest green after the merge: 47 checks wired across 33 lanes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * fix(ci): address review — suite detection, freerange, glob, vacuous skip-list Six review findings plus the mutation blocker. [bug] `registered` was shape-only, so a `test:*` script running `node src/bin.ts test <dir>` resolved to a `script:` leaf and was invisible. Four `test:replay:*` scripts were owned only because someone hand-registered them; `test:replay:android` was neither registered nor reported while the nightly ran the same six .ad files by inlining them. A `test:*` script is now a suite by name. `replay-android` is registered, and the nightly runs the script instead of re-listing its files so the two cannot drift. The nightly invokes it inside `reactivecircus/android-emulator-runner`'s `script:` input — shell handed to a third-party action this loader does not read — so the suite executes but cannot be credited. Recorded in UNPROVABLE_OWNERS with that exact reason rather than assumed. The fixed detector also found a second orphan the review did not name: `test:integration:progress`. That one is a reporter whose `--check` sibling is the registered gate, so it is declared in REPORTING_SCRIPTS — a declaration that itself fails when inert. [bug] `freerange` defaulted to localRunnable, so fail-open ran `fr` (a Bun binary) on the pre-push path. Now false. [suggestion] The `--run` skip-list asserted `build:android-snapshot-helper`, a name `android-helpers` no longer uses, so it could not fail. Derived from the catalog instead. [suggestion] `matchesGlob` joined `**` splits with `.*`, making the adjacent slash mandatory — GitHub's `**` matches zero directories, so `src/**/*.test.ts` did not match `src/a.test.ts`. Pinned against `packages/*/src/**/*.test.ts`. [suggestion] Deleted the unwired `run-gate` action. It had no callers, was absent from GATE_ACTIONS, and its comment described a system that had not shipped. It returns with the rewiring, not before. [suggestion] Collapsed the module headers that narrated discarded designs. Mutation: `daemon entrypoint publishes HTTP metadata and cleans up on shutdown` is the only test here that spawns a real daemon process. It takes ~1.1s alone but exceeds Vitest's 5s default inside Stryker's dry run, which aborts the sweep before a single mutant runs. Given 30s. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * fix(mutation): order sandbox aliases longest-first so subpaths resolve Every shard of the mutation sweep aborted in Stryker's dry run with: Cannot find package '@agent-device/selectors/engine' imported from .tmp/stryker/sandbox-*/src/core/selector-pipeline.ts The alias was generated correctly; it just never won. Vite matches a STRING alias by prefix and takes the first hit, and `workspaceSpecifierTargets` emitted the bare `@agent-device/selectors` ahead of the subpath entries. The bare entry therefore captured `@agent-device/selectors/engine` and rewrote it to `…/src/index.ts/engine`, which does not exist; Node fell back to real package resolution, could not find the subpath inside the sandbox, and the dry run failed before a single mutant ran — so the shard uploaded an empty envelope instead of a report and the ratchet failed for want of one. Sorting longest specifier first makes the most specific alias win: @agent-device/selectors/engine -> packages/selectors/src/engine.ts @agent-device/selectors/ast -> packages/selectors/src/ast.ts @agent-device/selectors -> packages/selectors/src/index.ts `/ast` never tripped this because nothing in a related test set imported it; `selector-pipeline.ts` introduced the first subpath import that mattered (#1744), so the mutation lane has been unable to run since that landed. Any PR touching `scripts/mutation/**` — which fails open into the full sweep — would have hit it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * refactor: derive gate ownership from workflow structure * fix: run gates without optional arguments * fix: resolve mutation workspace subpaths exactly --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
3eda37b0e0 |
fix(ci): run the three checks no workflow was running (#1770)
`check:tmpdir-leaks`, `check:tmpdir-leaks:test` and `test:fixture-cache` are real package scripts with real assertions that no CI lane executed. All three are reachable only through `check:unit`, an aggregate no workflow invokes, so a regression in any of them could not fail a PR — and a check that silently stops running looks exactly like a green build. Verified by transitive script reachability over every workflow and composite action on main: zero references to any of the three anywhere in .github. Placement: - `check:tmpdir-leaks` joins Coverage, the lane whose instrumented suite is what would leak a run directory. - `check:tmpdir-leaks:test` joins Layering Guard rather than sitting next to the check it covers. vitest-tmpdir-global-setup.test.ts proves the lifecycle by spawning a real nested `vitest run`, and Coverage already loses runs to worker-fork teardown errors; starting a nested Vitest beside the full instrumented suite is a contention risk with nothing to gain. - `test:fixture-cache` joins Integration Tests, beside the fixture-app fallback smoke that covers the same artifact contract. All three pass at this commit, so this wires green gates rather than red ones. `check:tmpdir-leaks` was also shown non-vacuous: planting an abandoned agent-device-test-run-* directory makes it fail and name the directory. Part of #1429, which stays open for the ownership proof that would have caught this class automatically. Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
05a1d76f2e |
test: add daemon RPC wire-surface compatibility gate (#1717)
* test: gate daemon RPC wire compatibility against the last released tag (#1432) ADR 0006 fixes exactly when DAEMON_RPC_PROTOCOL_VERSION must be bumped, and nothing checked that it was. The runtime guard (readRemoteDaemonHealth) refuses a mismatched peer, but only fires when someone remembered the bump — a wire change that skipped it left both sides advertising protocol 2 while parsing different payloads, which is the failure ADR 0006 exists to prevent. Local daemons cannot skew (isReusableDaemonInfo takes over on any package version mismatch). Cross-machine is skewed by design — proxy, cloud/limrun, a remote macOS host — and ADR 0006 explicitly rules package version out as the compatibility gate there, so the one boundary where skew is intended was the one boundary with no gate. test/wire-compat/surface.ts declares the wire surface grouped by the ADR bullet each group serves, quoting it, with an `uncovered` note where a bullet is only partly digestible (the /health and /rpc literals inside http-server.ts stay reviewer-owned: a moved route 404s at connect time rather than misparsing). ledger.json records what each declaration hashes to, at which protocol version. Two gates, split for the same reason the replay-compat corpus splits: - unit-core holds the ledger to its source and prints the digest to paste; - Released-Surface Compatibility reads the ledger at the last RELEASED tag and requires the drift since then to carry a bump or a compatibleChanges ack. From one commit a bumped ledger and an unbumped one are both just an edited file, so only a released baseline can tell them apart. Acks are keyed by the digest they cover, so one "added an optional field" cannot launder later changes. Digests ignore comments and formatting; the manifest's closure is derived from the AST, so a field typed by an unlisted sibling fails rather than sitting outside the gate. CI cost: one added job (checkout + toolchain + two node scripts, ~1 min), mirroring the existing full-history replay-compat job. * test: close wire-surface overclaim and make the closure fail closed (#1432) Addresses both review P1s on #1717. P1 — the manifest materially overclaimed ADR 0006 coverage. It quoted all four bullets while digesting only the payload TYPES, so the producer and consumer seams could break a skewed peer without moving a listed digest. Now listed on both sides of every boundary: JSON-RPC method sets and the projections that turn each method's params into a DaemonRequest, createRpcError/sendJson/ writeRpcResponseEnvelope, resolveToken and the auth-hook types, upload preflight/finalize/308 handlers and the resumable ticket shape, artifact route and download/inventory framing, REST error mapping, and the client's own payload builder, lease-method mapping, response parser and error projection. 57 -> 117 declarations. What stays out is now named rather than implied: createDaemonHttpServer's dispatch wiring and the /health and /rpc literals inside it. Everything it dispatches WITH is digested individually, and a moved route 404s at connect time rather than misparsing — the loud failure, not the silent one. P1 — imported and re-exported payload shapes escaped the closure. declarationHomes() scanned only the manifest's own files and the walk continued silently when a name could not be placed, so a listed type could gain foo?: ImportedShape from a new module and stay green. Resolution is now explicit and fails closed: relative imports, workspace specifiers (through the owning package's own exports map, so a re-pointed export cannot drop a type), and facade re-export chains. Every referenced name must land on a listed declaration, a waiver with a written reason, a declared external module, or the TS/Node global set. Fixed two extractor blind spots the walk exposed: a declaration's own generic parameters and `as const` were being reported as references. Planted-red proofs (wire-mutations.test.ts): 13 cases independently mutate method naming, response serialization, response parsing, auth projection, upload ticket shape, 308 framing, artifact framing, REST error mapping, and progress framing, each asserting the digest moves; 3 probes prove the closure really reaches across a package boundary, a facade re-export, and a plain relative import. Mutations apply inside the declaration's own span — a whole-file replace silently hit a sibling sharing the substring, which is how the first draft of one case passed vacuously. The largest waiver pair (InternalRequestOptions, CommandFlags) rests on ADR 0006's own additive rule: they reach the peer inside DaemonRequest's untyped flags/input bags, and the decision says a new flag needs no bump. Digesting them would fire the gate on every new CLI flag and train reviewers to rubber-stamp acks. * test: list the consumer half of the auxiliary HTTP boundaries (#1432) Addresses the remaining review P1 on #1717. The manifest claimed both sides of response/upload/artifact framing while listing nothing from upload-client.ts, daemon-artifacts.ts, or the health consumer in daemon-client-transport.ts, so those parsers could narrow without moving a listed digest or protocol 2. Now listed (117 -> 141 declarations): - /health consumer: RemoteDaemonHealth, readHealthPayload, readDaemonHttpHealth, readRemoteDaemonHealth. This is the sharpest of the three — narrowing the reader or the comparison disables the very refusal ADR 0006 exists to guarantee, and nothing else in the repo would notice. - /upload consumer: UploadResponse, UploadPreflightResponse, UploadPreflightResult, parseUploadPreflightResult, requestUploadPreflight, uploadDirectArtifact, tryDirectUploadWithResume, shouldRetryDirectUpload, finalizeDirectUpload, uploadLegacyArtifact, ARTIFACT_HASH_ALGORITHM, isStringRecord, and PreparedUploadArtifact — whose sha256/sizeBytes/fileName/artifactType/ contentType fields ARE the preflight body the daemon parses. - /artifacts/* consumer: DaemonArtifactEndpoint, buildDaemonArtifactUrl, isRemoteDaemon, DownloadRemoteArtifactParams, downloadRemoteArtifact, materializeRemoteArtifacts, resolveMaterializedArtifactPath. Running the closure fail-closed over the new files surfaced three more stops, each decided rather than skipped: PreparedUploadArtifact listed (it is payload), UploadProgressSink waived (client-local rendering, never leaves the process), and src/daemon/types.ts#DaemonArtifact waived as a re-export alias of the listed kernel type, matching its DaemonRequest/DaemonResponse siblings. 10 more planted-red mutations cover the new seams: health version-read and mismatch-refusal defeated, RemoteDaemonHealth field dropped, preflight parser narrowed, preflight/legacy response shapes narrowed, finalize body key renamed, ticket field renamed, artifact tenant header dropped, artifact URL moved. A fourth closure probe proves the upload-consumer files are genuinely reached by the walk rather than merely listed. 22 -> 33 tests. The README now states the coverage as a producer/consumer table per boundary, so the claim is checkable at a glance instead of asserted in prose. * test: list the client half of the resumable 308 contract (#1432) Addresses the third review P1 on #1717. Listing the daemon's handleResumableUpload proved it still PRODUCES 308; nothing proved the client still CONSUMES the released one. src/remote/upload-stream.ts owns that half and was entirely outside the manifest, so a newer client could stop accepting `upload-offset`, change how it reads `Range: bytes=0-N`, or emit a different resumed `Content-Range` without moving one of the 141 listed digests. Now listed (141 -> 151): UploadStreamResponse, streamFileToHttpRequest, streamFileToHttpRequestAttempt, buildUploadRequestHeaders, isUploadResumeStatus, isUploadRedirectStatus, parseUploadResumeOffset, parseNonNegativeIntegerHeader, firstHeaderValue, MAX_UPLOAD_REDIRECTS. streamFileToHttpRequestAttempt is listed despite its size, unlike createDaemonHttpServer which stays in `uncovered`. The distinction is stated at the declaration: the HTTP server only dispatches to handlers that are each digested, while the attempt loop IS the resume state machine — it decides whether a 308 continues the upload and what the next request carries, so its sequencing alone can break a released daemon while every helper keeps its digest. 6 new planted-red mutations prove the client half moves the ledger: a dropped `upload-offset` fallback, narrowed Range parsing, a changed resumed Content-Range, 308 no longer treated as continue, a narrowed UploadStreamResponse, and dropped header-value coercion. 33 -> 39 tests. Closure fail-closed surfaced two more stops: UploadStreamProgressOptions waived (local byte-progress rendering) and URL/URLSearchParams added to the global set. README now carries a `/upload` resume row in the producer/consumer table, and names the pattern behind three rounds of review: the coverage sentence kept getting written ahead of the coverage, so the table and the `uncovered` notes are the claims to trust — they are checkable against surface.ts, prose is not. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
cdc754e6ed |
perf: speed up iOS agent recovery and streamline CLI guidance (#1700)
* Avoid interactive children in parent taps * docs: streamline no-skill CLI help * perf: recover faster from sparse iOS trees * fix: preserve selector context for blocked parent taps * fix: preserve coordinate text-entry focus * fix: preserve thin parent touch targets * fix: fail closed for unscoped iOS typing * test: isolate replay lock fixture * test: share node integration process |
||
|
Michał Pierzchała
|
e18a183ac9 |
fix: harden artifact ingestion boundaries (#1692)
* fix: harden artifact ingestion boundaries * fix: bound archive inspection and upload expiry * fix: preserve upload preflight expiry |
||
|
Michał Pierzchała
|
6c0fcb64a1 |
fix: reject distinct ambiguous mutation targets (#1667)
* fix: reject distinct ambiguous mutation targets * fix(ios): scope the raw-match rejection to mutating dispatches `RunnerTests+Interaction.findElement` applied the new fail-closed classification to `querySelector` as well as press/type, because the read call site takes the default `allowNonHittableFallback: false`. With one visible/hittable match and one non-hittable same-selector duplicate the query started returning AMBIGUOUS_MATCH where it previously selected the hittable element, and `queryDirectIosSelectorOrFallback` preserves that error for read callers — so `get`, `is`, and `wait` surfaced an error instead of their prior answer. `classifyDirectSelectorCandidates` now takes a `rawMatchPolicy`. Mutations keep `.rejectDistinctMatches` (the default, so no mutation call site changes); `queryElement` passes `.preferHittableMatch`, restoring the prior read rule: prefer the single hittable match, ambiguous only when hittable matches compete, and never adopt the Maestro coordinate fallback. The Maestro expected-point path is untouched. Covers the one-hittable + one-non-hittable read, competing hittable reads, and the non-hittable-only read. ADR 0011's amendment now states the scope. * test(ios): execute selector read ambiguity regression --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
04c33f9e1e |
feat(ios): expose AX custom actions on merged accessibility elements (#1665)
* feat(ios): expose AX custom actions on merged accessibility elements
Apps that merge a card into one accessibility element for VoiceOver (React
Native's `accessible` prop) publish the card's real affordances as
UIAccessibilityCustomActions rather than as child elements. Our snapshot showed
only the merged node, so an agent looking for a feed card's options control had
nothing to aim at and fell back to coordinate guessing.
`snapshot --actions` now names them:
@e8 [link] "feedItem-by-whiskers.test" actions: ["Reply", "Repost", "Open post options menu"]
Opt-in, because the AX server cannot serve custom actions in a bulk tree
request: adding the attribute makes testmanagerd's reply decoder reject the
nested arrays a custom action serializes into, drop the reply, and time the
request out (~65s vs ~110ms). Only per-element reads answer, at ~100ms each, so
the runner reads at most 12 labelled childless nodes and stops at the
capture-plan deadline. The request pins the private-AX backend, since no other
backend can read the attribute, and reports that as its own `requested-backend`
verdict so a deliberate pin never renders as a degradation warning.
Invocation is not shipped: the actions are readable but not invocable from the
runner. RunnerAXSnapshotBridge.h records the five APIs that were tried.
* fix(ios): disclose a capped custom-action pass, and read on-screen elements first
Two gaps in the first cut.
An element the bounded pass never reached rendered identically to one with no
custom actions, so a capped capture silently taught the reader that later feed
cards have no affordances — the exact mis-inference this feature exists to
prevent. The runner already counted reads against candidates; it now carries
both to the response, and the verdict renders one response-level line when the
pass was incomplete. A complete pass stays silent, and "never asked" stays
distinguishable from "read none" (the key is absent, not (0, 0)).
The obvious remedy for a capped pass would be a scoped re-run, but scope is
applied when the Swift walk builds nodes, long after the read pass, so it does
not redirect the budget at all — measured: reads=12 candidates=18 with and
without --scope. Rather than print a remedy that does nothing, the read pass now
orders candidates on-screen first. That makes the budget land on elements an
agent can act on, and makes the disclosed remedy true: scrolling changes the
on-screen set, so a re-run reads elements the previous pass could not.
Also states plainly, in the flag help and the tool/SDK field description, that
the names are for planning: nothing invokes them, so the affordance is reached
through the element's detail screen, the same control exposed elsewhere, or
coordinates.
* test(snapshot): pin the custom-action coverage pair in the verdict shape assertion
* chore(scripts): classify the --actions flag in the integration progress model
The completeness gate flagged snapshotCustomActions as unclassified, which is
what it is for. It gets its own bucket rather than joining the provider-scenario
table: the values come from the private AX client inside the runner process, and
the fake runner derives its behavior from fixture tables that cannot fabricate
custom actions, so there is no provider-backed scenario to claim. The owning
coverage is named instead — runner XCTest unit, snapshot-lines, snapshot-quality.
* fix(ios): fail closed on unserviceable --actions, bound each read, cap output, and count actions in identity
Four review findings.
1. `--actions` with `--raw`, or on any target that is not an iOS simulator, used
to succeed and return nodes with no actions — a requested capability silently
no-opped, indistinguishable from "this screen has none". Both now fail closed.
The raw pairing is rejected at the shared request seam (INVALID_ARGS) so CLI,
Node client and MCP answer alike before any device work; the platform case is
rejected once the session device is resolved (UNSUPPORTED_OPERATION), naming the
resolved target. `diff --actions` was already rejected as an unsupported flag.
2. The per-element AX read had no timeout, so one wedged element could consume
the whole capture budget. Each read now runs off-thread behind a 1s wait. A
timed-out element counts as unread, never as "read, and it has no actions", so
the existing partial-pass disclosure already covers it.
3. The element budget bounded element count only; one element could still return
an unbounded list of unbounded names. Capped at 8 names of 80 characters, and
clipped elements are counted into the coverage so a truncated list is disclosed
rather than silently presented as complete.
4. Action names were rendered unescaped, and no comparison key read them. Names
now get the same escaping as text previews plus control-character folding, so an
app-authored name cannot split or corrupt a line. `actions` joins the diff
comparable key, the unchanged-comparison projection, and — the sharper bug —
the snapshot presentation key, without which `snapshot` followed by
`snapshot --actions` on a still screen answered "unchanged" and never delivered
the actions that were explicitly requested.
* fix(ios): contain a hung custom-action read instead of accumulating orphans
The 1s read deadline frees the caller, but the underlying AX call is a
synchronous XPC round trip that cannot be cancelled — it keeps running. On a
global concurrent queue that meant repeated `snapshot --actions` against a
wedged element piled up orphaned reads, all using the shared XCAXClient
concurrently. The deadline was containment for the capture, not for the runner.
Since the call cannot be cancelled, contain it instead:
- every read runs on one dedicated serial queue, so a wedged call can never be
joined by a second concurrent user of the shared client;
- a single-flight guard refuses to dispatch at all while an abandoned read is
still outstanding, so a repeated capture adds no work — the dispatch counter
stands still;
- the read pass stops at that point rather than paying a deadline per element
on reads that would all be refused, and reports `blocked` so the capture stays
honest. That gets its own line, because the partial-pass remedy (scroll and
re-run) cannot clear a hang and would send the reader in circles.
Recovery needs no reset: when the hung call finally returns, in-flight drops to
zero and reads resume.
The regression drives a fake AX client that never returns, and asserts the three
things the fix exists for — exactly one in-flight read with no further
dispatches across repeated captures, immediate returns with the skip disclosed
instead of the scroll remedy, and reads working again once the wedge clears.
* ci(ios): execute the custom-action runner regressions instead of only compiling them
The iOS workflow runs a targeted -only-testing list, so a runner test that is
not named there is compiled by the build step and then never executed. All seven
custom-action tests were in that gap — including the containment regression,
which is the only executable proof that a hung AX read cannot accumulate
orphaned in-flight reads.
Red/green against the containment regression, with the fix reverted to its
pre-fix concurrency behavior (global concurrent queue, no single-flight guard,
no blocked exit):
RED in-flight 6 (want 1), dispatches 6 (want 1), each repeat paid the full
1.004s deadline (want <0.2s), blocked=false (want true), and the
in-flight drain never completed — "Exceeded timeout of 5 seconds".
GREEN 7/7 pass, containment regression in 1.02s.
|
||
|
Michał Pierzchała
|
5f90d908e2 |
fix(ios): wait for hidden-keyboard synthesized text to commit before responding (#1676)
* fix(test): wait for typed text to settle in the hidden-keyboard runner test
testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHidden read textField.value
in one shot right after executeTypeCommand returned. The simulator commits
synthesized keystrokes after the command responds, so on a loaded CI machine
the read landed mid-word — observed failures reported ("h") and
("hardware-ke"). It failed on 3 of 5 runs of a branch carrying zero Swift
changes and passed on re-run.
Poll the value until it holds the expected text (10s ceiling) and assert on
the last value read, so a real regression still fails with what the field
actually held. Both reads in the test use the same helper; the assertions are
unchanged.
* Revert "fix(test): wait for typed text to settle in the hidden-keyboard runner test"
This reverts commit
|
||
|
Michał Pierzchała
|
ac52281448 |
fix(test): deterministic temp-dir cleanup across node --test lanes (#1661)
* fix(test): deterministic temp-dir cleanup across node --test lanes node --test has no global setup/teardown hook, so unlike Vitest (#1593) every node --test package.json script (maestro:conformance, mutation:test, check:affected:test, check:coverage-changed:test, check:layering, depgraph:test, check:tmpdir-leaks:test, check:contention-retry, test:fixture-cache, test:smoke(:web), test:integration:node, test:concurrency-torture) still created scratch directories against the real, unredirected os.tmpdir(), with cleanup only as reliable as each call site's own try/finally — which a crash, OOM, or timeout kill bypasses entirely. Add scripts/node-test-tmpdir.ts: it wraps the whole `node --test` invocation as a child process, redirecting TMPDIR to one disposable, pid-tagged directory (shared root/prefix with the Vitest lane) and removing it from the process 'exit' event, which fires on normal completion, a thrown error, or a forwarded SIGINT/SIGTERM alike. Every node --test script now runs through it. check-tmpdir-leaks.ts already scans by root/prefix, so it covers both mechanisms with no changes to its detection logic. Verified: a node --test process that mkdtemp's then gets SIGKILL'd leaves a directory behind unwrapped; wrapped and SIGTERM'd, TMPDIR is redirected and the directory is gone with no orphaned processes. All 13 wrapped lanes and the full Vitest suite (5,591 tests) pass with zero residual agent-device-test-run-* directories after the run. Fixes #1595 * test(tmpdir): ratchet every node --test script through the wrapper The 13 lanes wrapped in package.json were a one-time hand sweep with nothing enforcing the pattern going forward — a 14th node --test script added later without scripts/node-test-tmpdir.ts would silently reopen #1595 for that one lane. Add a structural check to scripts/node-test-tmpdir.test.ts (now part of check:tmpdir-leaks:test) that reads package.json and fails if any script invokes `node ... --test` without routing through the wrapper. Dumb string matching over the scripts map, no shell parsing, with an explicit (currently empty) NODE_TEST_WRAPPER_BYPASS_ALLOWLIST for any lane that must legitimately bypass it. Verified it both passes on the current package.json and fails when a synthetic unwrapped `node --test` script is added. * fix(test): preserve the Swift cache and close the raw node --test bypasses Review on #1661 found two gaps: 1. The wrapper only overrode TMPDIR, so it discarded and forced a recompile of the durable Swift compiler cache every run instead of mirroring vitest-tmpdir-global-setup.ts's carve-out for it. Read os.tmpdir() before the child's TMPDIR redirect takes effect and set AGENT_DEVICE_SWIFT_CACHE_DIR from that (only when unset), same as the Vitest lane — the two now share one durable cache instead of each discarding and recompiling their own. Added a probe assertion (scripts/node-test-tmpdir.test.ts) that fails without the fix and passes with it (verified both ways). 2. docs/agents/testing.md documented raw `node --test` commands for the iOS smoke files, and the android/ios/conformance-regenerate/nightly workflows invoked `node --test` directly outside package.json. Routed all of them through scripts/node-test-tmpdir.ts so the documented local commands and CI lanes get the same crash/timeout-safe cleanup the package.json scripts already have. |
||
|
Michał Pierzchała
|
3d2a9a05e8 |
ci: remove package smoke workflow (#1624)
* ci: remove package smoke workflow * ci: align affected package check * ci: verify packaged tarball before publish |
||
|
Michał Pierzchała
|
a67c72c211 |
fix(ios): pin tap-outcome corroboration probes to the baseline's backend (#1634)
* fix(ios): pin tap-outcome corroboration probes to the baseline's backend The recorded-failure screens are exactly where the capture plan flips between XCTest and private-AX (the penalty boundary), so #1605's same-backend requirement failed closed right where XCTest tap false negatives actually happen: the baseline was captured via private-AX under penalty, the probe came back via tree, and a landed tap surfaced as XCTEST_RECORDED_FAILURE. In the AppControlBench bsky-16 run this fired four times, each sending the model into a re-observe/retry spiral. The comparison stays same-backend by design (backends are not comparable views of a screen); instead the probe is now CAPTURED the way its baseline was: a new internal preferredBackend option (never CLI-exposed) threads daemon -> runner, and a private-AX-preferred capture takes the exact penalized route — privateAX-first plan, 'deferred' verdict, no degradation warning, no settle budget reset. Live-verified on the deterministic repro (Bluesky drawer-menu press under penalty, seeded bench feed): errored with the backend-mismatch diagnostic before, corroborates as landed after, with no mismatch phase in the request diagnostics. Daemon tests cover pinned and unpinned baselines end to end through the dispatch context; the Swift plan gate is a pure function with an executed in-bundle test (added to the ios.yml regression list). * style: oxfmt * fix: exclude raw baselines from corroboration and prove the pin end to end (review) Raw baselines could not be pinned: the raw diagnostic plan keeps tree-first error propagation by contract and is never rerouted by the penalty or the preferred backend, so preserving 'raw: true' on the probe recreated exactly the backend-mismatch false failure this PR removes. Corroboration now declines raw baselines up front (they are diagnostics, not evidence baselines) with a regression pinning that no probe capture is dispatched at all. The wire is now regression-proven at every hop: a dispatch-level test drives dispatchCommand with the context flag and asserts the emitted RunnerCommand carries preferredBackend (red if handleSnapshotCommand or the interactor stops forwarding); the injected-transport test asserts the interactor's snapshot payload both ways; and a runner unit test decodes the wire JSON, projects it through the extracted snapshotOptions(from:), and composes it with the plan rule — pinned regular plan defers to privateAX-first, RAW plan stays untouched. Executed on-simulator; added to the ios.yml regression list. |
||
|
Michał Pierzchała
|
4a3ed1e8c3 |
feat(ios): extend depth-capped private-AX captures via element-rooted requests (#1627)
* feat(ios): extend depth-capped private-AX captures via element-rooted requests The AX server's depth limit is per-request (kAXErrorIllegalArgument above a tree-size-dependent value), so a capture capped at depth 56 on Bluesky-class React Native trees returned chains of unlabeled [other] containers and hid every actionable control below the cap — agents fell back to screenshot-and-coordinate guessing. After a capped serialization, the bridge now re-issues the same snapshot request rooted at each deepest-level childless node's live accessibility element, splicing the returned children in and chaining further while capped subtrees remain — bounded by a call limit, the shared node budget, and the capture-plan deadline. The AX server counts node levels, not edges: a maxDepth=56 request emits nodes to depth 55, so the frontier is the deepest observed level, kept only when it sits at the cap; a tree that ends naturally above the cap yields no frontiers and costs nothing. A capture whose frontier extension drained every capped node no longer reports itself depth-limited — the re-run hint it used to trigger could not add anything. Explicit --depth requests stay exact captures with no extension. Live on the seeded Bluesky bench feed: 8 extension calls (~106ms each) turn the 117-node all-[other] tree into a 182-node tree carrying post text, testID links, and every feed control (Reply/Repost/Like/options); steady-state capture 0.52s -> 1.37s. Observation-only captures paying the extension needlessly is #1626. * fix: count missed frontiers and tighten deep-extension shape (review) The completeness verdict inverted in the failure paths: a frontier whose live element vanished or whose re-rooted request failed was silently dropped, so an all-miss extension reported pendingFrontiers=0 and the capture presented as complete while whole subtrees were missing. Missed frontiers are now counted, logged, and keep the depth-limited verdict — the pure decision lives in privateAXDepthLimited with in-bundle tests. The truncation hint no longer advertises --depth (an explicit --depth capture disables extension, so following it returned strictly less than the capture that produced the hint); the honest remedy is a plain re-run with a fresh extension budget. Shape: candidates collected only at the cap boundary and not at all when extension is disabled (exact --depth captures pay zero bookkeeping); the zero-caller 3-arg overload is gone; the response shape's keys are shared constants; the unsupported-selector diagnostic is restored; the exact-depth policy is hoisted to one named local. * test: execute the deep-extension miss-path contract in CI (review) The depth-limited regression compiled but never ran (absent from ios.yml's -only-testing list), and the pure Swift consumer test was vacuous against the Objective-C producer — a literal missedFrontiers proves nothing about the increments. RunnerAXSnapshotFrontier and extendSnapshotFrontiers move to the header as the executed contract seam, and testDeepExtensionCountsMissedFrontiers drives both real miss paths with fabricated snapshots: a nil accessibilityElement (explicit nil property — bare NSObject resolves the key through a UIKit category and takes the call path instead) counts missed without consuming a call; a resolving element whose client cannot serve the re-rooted request consumes a call AND counts missed. Both tests join the executed ios.yml list. Red-before verified on-simulator: with both increments stripped the producer test fails at the counter assertion (0 != 2); restored, both tests pass. |