Commit Graph

12 Commits

Author SHA1 Message Date
Michał Pierzchała b15121ffc8 test(ios): establish snapshot convergence baselines and permanent evidence (#2204)
* test(ios): add snapshot convergence evidence harness

* fix(ios): satisfy benchmark CI guards

* fix(ios): constrain benchmark proxy routes

* fix(ios-benchmark): enforce cell admission evidence

* fix(ios-benchmark): protect benchmark state ownership

* fix(ios-benchmark): use proxy port flag

* fix(ios-benchmark): let proxy choose an ephemeral port

* test(ios-benchmark): keep CLI process seam local

* fix(ios-benchmark): parse proxy startup envelope

* fix(ios-benchmark): bind proxy lease to simulator

* fix(ios-benchmark): keep fresh proxy CLI sessions isolated

* fix(ios-benchmark): preserve async timeout evidence

* docs(ios-benchmark): retain exact-head evidence

* test(ios): reveal offscreen alert fixture controls

* test(ios): reset alert between relaunch samples

* test(ios): admit native alert snapshots

* docs(ios): publish snapshot convergence corpus

* chore(ios): format benchmark evidence

* fix(ios-benchmark): admit proxy fixture anchors

* docs(ios): republish exact-head benchmark corpus

* fix(size): make publish asset evidence hermetic

* style(size): format package evidence test

* test(size): update publish preparation contracts

* fix: retire stale utils layering zone

* test: pin shared publish asset owner

* test: verify preserved size reporter closure

* fix: move mutation ownership to snapshot module

* test(ios): add snapshot convergence evidence harness

* fix(ios): satisfy benchmark CI guards

* fix(ios): constrain benchmark proxy routes

* fix(ios-benchmark): enforce cell admission evidence

* fix(ios-benchmark): protect benchmark state ownership

* fix(ios-benchmark): use proxy port flag

* fix(ios-benchmark): let proxy choose an ephemeral port

* test(ios-benchmark): keep CLI process seam local

* fix(ios-benchmark): parse proxy startup envelope

* fix(ios-benchmark): bind proxy lease to simulator

* fix(ios-benchmark): keep fresh proxy CLI sessions isolated

* fix(ios-benchmark): preserve async timeout evidence

* docs(ios-benchmark): retain exact-head evidence

* test(ios): reveal offscreen alert fixture controls

* test(ios): reset alert between relaunch samples

* test(ios): admit native alert snapshots

* docs(ios): publish snapshot convergence corpus

* chore(ios): format benchmark evidence

* fix(ios-benchmark): admit proxy fixture anchors

* docs(ios): republish exact-head benchmark corpus

* fix(size): make publish asset evidence hermetic

* test(size): update publish preparation contracts

* fix: keep git-state gates out of mutation sandboxes
2026-09-01 21:39:05 +02:00
Michał Pierzchała 42dc9adb5d fix: address security scanner findings (#2182)
* fix: address security scanner findings

* fix: close image-size parser review gap

* test: prove zero-length image box regressions

* fix: keep fixture fingerprint output machine-readable

* test: align fixture fallback with fingerprint owner
2026-08-31 20:38:44 +02:00
Michał Pierzchała af6f12e391 chore: adopt shared oxlint config (#2115)
* chore: adopt shared oxlint config

* fix: preserve project lint boundaries

* fix: remove redundant oxlint config
2026-08-28 11:42:58 +02:00
Michał Pierzchała 539e848e0c fix(ci): stop ten artifact uploads discarding their hidden paths (#2091)
* fix(ci): stop ten artifact uploads discarding their hidden paths

`actions/upload-artifact` has excluded hidden files and directories by default
since v4.4 (this repository pins v4.6.2), and most diagnostics here are written
under `.tmp`. Ten upload steps across seven files therefore uploaded nothing
from those paths: macos.yml's xcresult bundle, both mutation lanes' reports and
shards, replays-nightly's fuzz output, xctest-nightly's results,
test-app-build-cache's fixture tarball, and 1874-diagnose's per-iteration logs.

Most fail silently, since they pair the omission with `if-no-files-found: warn`
or `ignore`. test-app-build-cache sets `error`, so that one does not.

A structural guard rather than a shared upload wrapper: the wrapper would be a
shallow mirror of the action's options over artifacts with different owners,
while the policy question — a hidden path needs the flag — is one rule that
belongs in one place. Each workflow still declares its own artifact.

test/ci/upload-artifact-hidden-paths.test.ts holds it across every workflow and
composite action, and is red if any single flag is dropped.

* test(ci): scan every YAML shape GitHub accepts, not just top-level *.yml

The guard read `.github/workflows/*.yml` and assumed local actions live one
directory deep as `action.yml`. GitHub also reads `.yaml` for both, and local
actions nest, so a hidden-path upload in any of those shapes passed the gate.

It now walks the `.github` tree recursively for either extension, and a second
test plants the three shapes the old scan missed and asserts all three are
found — executable rather than a one-off manual check. Red against narrowing the
extension, against dropping the recursive walk, and against removing any single
real flag.

Also drops the action-version note under the comment rule in #2087; the version
behaviour belongs in the PR, and the assertion message already says what the
omission costs.
2026-08-27 20:08:45 +02:00
Michał Pierzchała b40debfcd8 fix(ci): skip release instead of erroring when both fixtures are cached (#2036)
* fix(ci): skip release instead of erroring when both fixtures are cached (#2034)

map(select(.build)) yields an empty include list when both the iOS and
Android fingerprints already have a trusted artifact, and GitHub Actions
rejects an empty strategy.matrix at the workflow level -- so release was
never created and the run was marked failure on every push since #1996
merged. Publish has-work alongside matrix and gate release on it, so the
both-cached steady state now completes with release skipped instead of
erroring the whole workflow.

* test(ci): fold has-work regression into the existing fingerprint test

Reviewer feedback on #2036: the standalone four-case test duplicated the
harness above it and only two states are meaningful for this regression.
Reuse the same parsed workflow, temp dir, resolver stub, and Node stub;
keep neither-cached (both platforms, has-work=true) and both-cached
(empty matrix, has-work=false, release gated). Drops the single-cache
permutations, which exercise #1996's unchanged filtering rather than
this fix.

* test(ci): cover the single-cache matrix cardinality (#2036 review)

Reduced coverage to 0-cached and 2-cached, leaving the 1-cached
cardinality unchecked -- a mistaken \`length > 1\` in the has-work
check would pass while wrongly suppressing a valid single-platform
build. Generalize the Node stub to report caching per artifact-name
suffix and add the iOS-cached case to the same reused harness.

* test(ci): extract the has-work value instead of comparing raw output lines

Thermo-nuclear review: matrix was already parsed out of its GITHUB_OUTPUT
line (prefix stripped, JSON-parsed), but hasWork returned the raw
"has-work=true" line, so assertions compared against a redundant
'has-work=true' string instead of the actual value. Slice the prefix
the same way matrix does.
2026-08-26 07:49:33 +02:00
Michał Pierzchała 30de1597d3 ci: attribute native package size and trim Apple runner (#1934)
* ci: attribute npm package size by shipped component

* refactor: modularize size reporting and trim Apple runner

* ci: preserve size reporter modules across base checkout
2026-08-21 13:46:53 +02:00
Michał Pierzchała f843dc2df1 fix(scroll): keep saturated scroll gestures out of the status bar; gate Android replays from android/emulator (#1781 A1) (#1820)
* fix(scroll): keep saturated scroll gestures out of the status bar; gate Android replays from android/emulator (#1781 A1)

`pnpm gate replay-android` failed 4/8 whenever it ran after the full-tier Android E2E
(replays-nightly run 32107665052, job 95620294899): 05-app-lifecycle, 06-swipe-gestures and
both fixture replays diverged under "A system surface covers the app". The E2E was not the
cause. Reproduced on a pixel_7 / API 36 AVD with the same cutout geometry CI's
`avdmanager --device pixel_7` produces (status bar 136px, not the 63px of a plain 1080x2400
skin):

- `03-scroll-discovery.ad` runs `scroll up 3`. The scroll planner clamps travel to the viewport
  minus a 5% band, so the touch-down landed at y=120 — inside the 136px status bar — and
  pulled the notification shade instead of scrolling. On API 36 the app window is
  edge-to-edge, so the reported viewport starts at y=0 and includes that bar.
- The shade then covered every replay until `04`'s `back` closed it. Native readdir order on
  the runner (03, 05, 06, fixture/02, fixture/01, 04, 01, 02) put four files in that window;
  the last green run (2026-07-30) had 04 right after 03, so the pull was masked.

Fix in the product, not the lane: DEFAULT_EDGE_PADDING_FRACTION 0.05 -> 0.1 in the TS scroll
planner and its Swift port. Every real Pixel has a cutout (5.7% of a Pixel 7's height) and an
iPhone's Dynamic Island status bar is 6.9%, so any saturated `scroll up` opened the shade /
Notification Center for real agents too. Parity vectors updated in both suites plus a Pixel 7
regression vector (1080x2400, amount 3 -> touch-down y=240 > 136).

Second contamination the same order exposed once the shade was gone:
`fixture/02-selector-routes-covered-diagnosis.ad` is a #1715 reproduction recipe that FAILS
BY DESIGN at step 9 (covered-target refusal) and leaves the device in landscape, yet the
gate enumerated `test/integration/replays/android` recursively. iOS keeps gate replays in
`replays/ios/simulator` and fixture recipes in `replays/ios/fixture`; Android now mirrors that:
the six Settings replays move to `replays/android/emulator`, `test:replay:android` points there,
and `fixture/` stays E2E-owned (`full:fixture-replays` already runs 01 by path). android.yml
and the workflow-evidence fixture follow the path; the replay-compat manifest keeps the
historical paths it pins at released tags.

Verified live (Pixel 7 geometry, API 36, --retries 0): control run at main head in CI order
reproduces exactly CI's 4/8; with the fix, `pnpm gate replay-android` 6/6 in both native and
CI order, and `03` leaves Settings on screen (scroll up 3 now touches down at y=240).

* test(scroll): drive the TS and Swift scroll-plan parity vectors from one table (#1820 review)

The two suites hand-mirrored the same vectors and #1820 had to edit both by hand — the drift
class the repo already closes for the tap-point rule via contracts/fixtures/tap-point-policy.json.
The scroll vectors (plus both planner constants, pinned behaviourally on a 1000px axis) now live in
contracts/fixtures/scroll-gesture.json; scroll-gesture.test.ts and RunnerTests+ScrollGesture.swift
iterate it. Verified: vitest 10/10; the four XCTests run on an iOS 26.2 simulator with the unit
flag on (Executed 4 tests, 0 failures).

Also: test/ci/android-workflow-evidence.json says what it guards.

Follow-up for content-safe viewport bounds + discovery order: #1821.
2026-08-18 14:31:54 +02:00
Michał Pierzchała f45228ae71 ci: skip device lanes for root-level docs-only changes (#1781 A9) (#1791)
* ci: skip device lanes for root-level docs-only changes (#1781 A9)

Add AGENTS.md, CHANGELOG.md, CONTEXT.md, CONTRIBUTING.md, LICENSE, and
SECURITY.md to the pull_request paths-ignore block in ios.yml,
android.yml, linux.yml, macos.yml, ci.yml, and size.yml. These
root-level docs files were the only gap left after docs/**,
website/**, and README.md — PRs #1568 (SECURITY.md only), #1697
(CONTEXT.md + docs/adr only), and #1722 (AGENTS.md + docs/) each still
triggered a full 9-15 min macOS iOS run despite touching only prose.

Why each file is safe to ignore for every one of these six workflows:

- None of the four device workflows (ios/android/linux/macos) or their
  composite actions read any of these six files at runtime; the only
  hits from `grep -rln` across scripts/, src/, test/, and
  .github/actions/ are prose comments pointing humans at CONTEXT.md or
  AGENTS.md sections (e.g. scripts/layering/check.ts,
  scripts/wire-compat/run.ts, src/mcp/tool-ref-pins.ts) — never an
  `fs.readFileSync`/`readFile` of the file itself.
- The check-affected selector (scripts/check-affected/model.ts)
  already classifies all six as pure docs: `isDocs()` matches any
  `.md` file plus the literal `LICENSE`, and `docsOwnership()` only
  special-cases `website/docs/docs/commands.md` (unrelated). So these
  files already select zero checks — they only ever produced
  `docsOnlyPaths` entries, never `SelectionReason`s.
- Because they select zero checks, the gate-manifest's path-coverage
  category derivation (`scripts/gate/model.ts` `categories()`, which
  iterates `plan.reasons`) never records a category for them, so
  ci.yml has nothing check-manifest-only that these six files would
  need to keep reachable. `pnpm check:gate-manifest` and
  `pnpm check:gate-manifest:test` both stay green after the change
  (48 checks / 33 lanes, 28/28 gate tests passing).
- size.yml's bundle-size job (scripts/size-report.mjs) measures the
  `pnpm build` dist output and startup timing only — no reference to
  any of these six files. (npm packs LICENSE/README.md into the
  publishable tarball, but that's a `pnpm check:package` node-22.12
  concern in ci.yml's packaged-cli job, which is driven by `dist`
  contents and `package.json`, not by LICENSE/README prose — already
  evidenced by README.md being ignored here since before this change.)

Scope disclosure: `mutation-affected.yml` uses a `paths:` allowlist
(not paths-ignore) so it's structurally unaffected; `test-app-build-cache.yml`
has no path filter at all. Neither was touched.

actionlint and `pnpm check:gate-manifest`/`:test` pass on the changed
workflows.

* test: pin root-doc paths-ignore entries with a regression test

Addresses review feedback on #1791 from thymikee: the docs-only
classifier for AGENTS.md/CHANGELOG.md/CONTEXT.md/CONTRIBUTING.md/
LICENSE/SECURITY.md across ios.yml/android.yml/linux.yml/macos.yml/
ci.yml/size.yml had no regression pin. Neither check:gate-manifest
(only proves a *registered check* is reachable) nor actionlint (only
validates YAML shape) nor generic Markdown coverage would catch a
single dropped entry — e.g. LICENSE reappearing in one workflow's
paths-ignore list but not another's would silently put a full 9-15 min
device run back on prose-only PRs.

test/ci/root-docs-paths-ignore.test.ts parses the six real workflow
files and asserts, using the same matchesGlob the gate-manifest model
uses to decide lane triggering, that each of the six root docs is
ignored by each workflow's pull_request paths-ignore. Registered in
vitest.config.ts's unit-core project next to its sibling
upload-agent-device-artifacts.test.ts (parse-only, no device/subprocess
lane needed).

Verified red on main (all 36 file x doc assertions fail — confirmed via
a throwaway script reading `git show main:.github/workflows/*.yml`)
and green on this branch (6/6). Full unit-core project (873 files /
6641 tests) still passes; check:gate-manifest and
check:gate-manifest:test unchanged (48 checks / 33 lanes, 28/28).
2026-08-18 09:57:09 +02:00
Michał Pierzchała cd9a7ce41b test(android): add comprehensive emulator E2E coverage (#1482)
* test(android): add catalog emulator smoke coverage

* test(android): use stable snapshot diff mutation

* test(android): assert actual back destination

* test(android): separate keyboard and fill IMEs

* fix(ci): keep Android timing report in one shell

* refactor(test): simplify simulator e2e coverage

* test(android): assert stable diff landmarks

* fix(android): release snapshot helper gracefully

* fix(android): fully release snapshot helper runtime

* test(android): report coverage classifications

* fix(android): stabilize accessibility root capture

* fix(android): bound UiAutomation connection

* ci: upload worktree daemon diagnostics

* fix(android): cancel stalled wait captures

* fix(android): bound helper fallback lifecycle

* fix(android): harden emulator e2e lifecycle

* fix: align e2e changes with kernel package

* test(android): prove alert helper reuse directly

* fix(android): cancel stalled settle captures

* fix(android): separate helper retirement budgets
2026-07-30 15:10:21 +02:00
Michał Pierzchała 6ab59630e4 ci(android): trust cached fixture APKs (#1479)
* ci(android): trust cached fixture APKs

* fix(ci): keep Android emulator script lines self-contained

* fix(ci): assert Android fixture smoke evidence

* fix(ci): harden Android fixture smoke cache path

* test(ci): verify Android fixture foreground app

* fix(ci): read Android snapshot response envelope
2026-07-28 22:51:01 +02:00
Michał Pierzchała 4e4ecdea0d test(ios): expand simulator e2e coverage (#1408)
* test(ios): expand simulator e2e coverage

* test(ios): make coverage checks host portable

* test(ios): handle deep link confirmation

* test(ios): fix deep link prompt selector

* ci: stabilize full simulator nightly

* test(ios): stabilize permission prompt lifecycle

* test(e2e): wait for route-specific landmarks

* test(e2e): reset permissions from inactive app

* test(e2e): redeliver trusted cold deep links

* test(ios): verify orientation native readback

* test(ios): stabilize simulator permission coverage

* test(ios): wait for tab target after deep link

* test(ios): paginate full event timeline

* test(ios): simplify event pagination coverage

* test(ios): stabilize simulator e2e coverage

* test(ios): model simulator recorder lifetime

* ci(test-app): cache fixture dependencies

* fix(ci): isolate test app cache by node

* fix(ios): settle fixture route navigation

* test(ci): waive unbenchmarked ios system UI help

* fix(ios): tolerate delayed simulator scale lookup

* 0.20.1

* test(ci): remove superseded system UI waiver

* fix(ios): harden simulator e2e reliability

* chore: clarify Apple runner CI steps

* fix(ios): wait before fixture home snapshot

* fix(ios): require exact catalog navigation

* chore(ci): format rebased workflows

* fix(ios): retry unobserved fixture navigation

* refactor(test): remove iOS e2e workarounds

* fix(ci): verify fixture artifact provenance

* fix(ci): align fixture artifact fingerprints

* fix(ci): use unified Android helper packager

* perf(ci): scope fixture build concurrency

* chore: format fixture artifact tests

* test(ios): update split Apple coverage owner

* fix(ios): accept deep-link confirmation alerts
2026-07-28 20:08:45 +02:00
Michał Pierzchała c984bd5902 fix(ci): upload hidden diagnostics (#1443)
* fix(ci): upload hidden diagnostics

* fix(ci): capture all hidden diagnostics
2026-07-28 12:30:42 +02:00