Commit Graph

240 Commits

Author SHA1 Message Date
Michał Pierzchała f18d0b2e92 fix: improve settle observation guidance (#1154) 2026-07-08 17:12:19 +02:00
Michał Pierzchała 9009c5aff7 0.19.0 2026-07-07 13:17:01 +02:00
Michał Pierzchała 7475415ac5 build: strip runner unit-test blocks from package (#1128)
* build: strip runner unit-test blocks from package

* ci: fix package size and fallow checks

* fix: resolve packaged recording scripts

* fix: keep recording script resolver internal
2026-07-06 12:52:05 +02:00
Michał Pierzchała f9721e7e8b test: split the Android platform test aggregation and share the scripted adb stub (#1103)
* test: split the Android platform test aggregation and share the scripted adb stub

AGENTS.md names the platform index.test.ts aggregations as offenders to
shrink opportunistically; this splits the 2,735-line Android one along
its (already well-factored) source modules, every test moved verbatim
(92 tests before and after):

- ui-hierarchy.test.ts (22): parseUiHierarchy/androidUiNodes
- app-lifecycle-install.test.ts (13): install/resolve/infer/launch
  component parsing
- app-lifecycle-open.test.ts (19): open/close, deep links, launch args,
  TV category, fallback resolve-activity
- input-actions.test.ts (11): type/fill/swipe/scroll/rotate
- settings.test.ts (14): appearance/clear-app-state/fingerprint/
  permissions
- notifications.test.ts (2), app-parsers.test.ts (1)
- keyboard state/dismiss tests (10) appended to the existing
  device-input-state.test.ts

Consistency fix folded in: the file carried a local withMockedAdb fork
because it needs scripted per-subcommand adb responses, which the shared
arg-recorder helper cannot express. The fork now lives in
src/__tests__/test-utils/mocked-binaries.ts as withScriptedAdb next to
withMockedAdb, and hands each call a fresh copy of the shared
ANDROID_EMULATOR fixture.

The copy matters: the Android TV test mutated the callback's device
(device.target = 'tv'), which the old per-call object literal absorbed
silently. With a shared fixture that mutation leaked into the next test
and flipped its launch to LEANBACK. The helper now clones per call and
the TV test builds { ...device, target: 'tv' } instead of mutating.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

* test: serialize the scripted-adb group and repoint its slow-test pins

Review follow-up for the android index.test.ts split: the monolith
implicitly serialized the env-mutating adb-stub tests (PATH,
AGENT_DEVICE_TEST_ARGS_FILE) in one worker, and the split let vitest
run them across parallel files. Make the contract explicit:

- new android-adb vitest project runs the six scripted-adb test files
  in a single fork (singleFork), keeping the pre-split execution
  semantics; ui-hierarchy and app-parsers stay in the parallel unit
  project (pure parsing, no env mutation)
- test/test:unit scripts run both projects
- the five slow-test ratchet pins that referenced index.test.ts keys
  now point at the split file names, so the pinned real-time offenders
  keep their exemption instead of failing at 2x budget under load; the
  reporter's own pinned-key fixture updated to match

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

* test: use vitest 4 android adb serialization

* docs: update unit project readiness guidance

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-06 11:23:03 +02:00
Michał Pierzchała 915f5ba374 build: support packaged CLI on Node 22.12 (#1116) 2026-07-06 07:59:37 +02:00
Michał Pierzchała 5a362ec752 build: drop tsc entirely and fix sandbox-hostile tests (#1100)
Follow-ups from the bundler/CI speed work, re-validated against latest
main. The typescript package is gone from the toolchain:

- pnpm typecheck stays on tsgo; the typecheck:tsc escape hatch is
  removed along with the typescript devDependency.
- args.test.ts extracted cli.ts dispatch literals through the
  TypeScript compiler API - the only remaining consumer. It now walks
  the same AST via oxc-parser (matching the OXC lint/format/build
  stack); both implementations extract an identical 14-literal set
  from cli.ts, verified side by side before the swap. The
  substitution-free template case ts.isStringLiteralLike covered is
  preserved.
- dts bundling is unaffected: the tsdown build uses the tsgo backend
  and builds green with no typescript package installed.

Test fixes for containerized agent environments:

- The missing-binary doctor-guidance web provider test pins Node 24
  via the file's existing withNodeRuntimeVersion helper, so it asserts
  the setup hint instead of inheriting the host Node and failing on
  Node 22 (the supported engines floor).
- The clean-xcuitest cleanup-failure smoke test skips as root: chmod
  0o500 cannot force a removal failure when the process bypasses
  directory permissions.

AGENTS.md toolchain notes updated to match.


Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-04 21:02:36 +02:00
Michał Pierzchała d167eaf0f3 ci: drop duplicate unit-test run, cache Size base dist, typecheck with tsgo (#1094)
Three measured dev-loop/CI cuts, no signal loss:

- typecheck now runs tsgo (already trusted for declaration emit by the
  tsdown build): 21.7s -> 5.3s locally, and check:tooling drops to ~18s
  total. tsc stays available as typecheck:tsc; verified tsgo fails on
  type errors and respects noUnusedLocals.
- remove the Unit Tests CI job: Coverage runs the same unit +
  provider-integration suites under coverage thresholds, so the job
  reran ~64s of tests every PR for no extra signal.
- Size workflow: skip docs-only paths (same paths-ignore as CI) and
  cache the base commit's dist keyed on base SHA, since dist is fully
  determined by that commit. Startup medians are still measured fresh
  on the same runner so the base/PR startup comparison stays
  same-machine; the cache is saved immediately after the base
  measurement so the PR build never poisons it.


Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-04 16:40:34 +02:00
Michał Pierzchała cd0cd16a9e build: migrate the library build from rslib to tsdown (Rolldown) (#1087)
* build: migrate the library build from rslib to tsdown (Rolldown)

Replace the Rspack-based rslib build with tsdown, the Rolldown-based
library bundler from the Vite toolchain family, so bundling, testing
(Vitest/Vite), linting (oxlint), and formatting (oxfmt) all run on the
same OXC/Rolldown stack.

Outcome vs the rslib baseline (size-report):
- build time: ~53s -> ~2s
- JS raw +16.2 kB (+1.1%), JS gzip +2.7 kB (+0.6%) - the residual gap
  is OXC vs SWC minifier tightness, not chunking
- npm tarball -3.0 kB
- CLI --version startup ~3 ms faster; --help within the +/-5 ms
  measurement noise of interleaved A/B runs

Chunk-merging experiments (single shared group, entries-aware groups,
small-module groups) all regressed either total size or --help startup
(a merged shared chunk adds +140 ms), so the default Rolldown split
graph is kept. Custom codeSplitting groups also currently trip a
rolldown-plugin-dts bug that re-emits type-only imports as runtime
imports.

Declarations still bundle per entry via tsgo; dist layout, entry names,
and the internal/ worker/daemon entry resolution contract are unchanged.
@microsoft/api-extractor was only consumed by rslib dts bundling and is
removed together with @rslib/core.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

* ci: only cache the pnpm store when setup installs dependencies

The layering-guard job uses setup-node-pnpm with install-deps: false, so
it never creates a pnpm store. setup-node's post-job cache save then
fails with a path validation error whenever the lockfile hash misses the
cache - which any lockfile-changing PR does. Gate the cache on
install-deps so no-install jobs skip pnpm store caching entirely.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-04 15:58:05 +02:00
Michał Pierzchała bff28009e2 0.18.3 2026-07-02 21:03:48 +02:00
Michał Pierzchała 091179e91d 0.18.2 2026-07-02 17:45:48 +02:00
Michał Pierzchała d1b56c0897 0.18.1 2026-07-01 16:26:05 +02:00
Michał Pierzchała a3e967526a refactor: rename ios-runner -> apple-runner (#981) (#996)
Finish the cosmetic ios-runner -> apple-runner rename now that the
top-level XCTest runner is the OS-agnostic Apple engine
(iOS/iPadOS/tvOS/macOS/visionOS from one Xcode project).

Cosmetic only, no behavior change:
- git mv ios-runner/ -> apple-runner/ (AgentDeviceRunner, README, RUNNER_PROTOCOL)
- Update repo project-path consumers: build-xcuitest-apple.sh, package.json
  files globs, .fallowrc.json, write-xcuitest-cache-metadata.mjs,
  runner-xctestrun.ts fingerprint/project paths, recording overlay + test,
  daemon-client-timeout kill pattern, setup-apple-replay hashFiles glob,
  ci.yml swift-compat scan, AGENTS.md.
- Rename runtime home cache/derived/lease dir default
  ~/.agent-device/ios-runner -> ~/.agent-device/apple-runner (build script,
  package/clean scripts, runner-xctestrun RUNNER_DERIVED_ROOT, runner-lease,
  runner-contract hint, cli-help/commands.md docs) and the tests asserting it.
- Rename OS-agnostic runner symbols: runIosRunnerCommand ->
  runAppleRunnerCommand, prewarmIosRunnerCache -> prewarmAppleRunnerCache,
  createIosRunnerCachePrewarmOnColdBoot / createIosRunnerCacheColdBootPrewarmForOpen
  -> createAppleRunner* (+ call sites, type aliases, test mocks).

Intentionally left as ios-runner (out of scope / would change behavior):
- prepare ios-runner CLI subcommand (user-facing command name)
- AGENT_DEVICE_IOS_RUNNER_* env var names and .tmp/ios-runner-derived CI values
- ios-runner-prebuilt cache-key-prefix, ci.yml job id, workflow/ADR filenames
- agent-device-ios-runner-<version> release artifact basenames

Part of #972 (Phase 3 - Apple PlatformPlugin).
2026-07-01 14:24:48 +02:00
Michał Pierzchała 3d70943550 feat: enforce import-direction DAG (Phase-5 layering lint) (#984)
Generalize the inline CI "Layering Guard" grep into a structured
import-direction lint (scripts/layering/check.ts) over the resolved
import graph, per plans/perfect-shape.md §5.5.

The full target DAG (kernel ◄ platforms ◄ core ◄ commands ◄ {cli,
client, daemon/server}; client ◄ daemon/client) is only partly realized
— the client/remote/metro extraction, the daemon/server split, and the
utils dissolution are still pending Phase-5 moves, so the tree still
holds legitimate back-edges (platforms→core, commands→cli, utils→*).
Enforcing the whole DAG today would need a mass import rewrite that
Phase 5 defers. The lint therefore enforces the three invariants the
completed moves (kernel/, daemon/client/) already guarantee and that are
green today:

  R1 kernel-sink      — nothing under src/kernel/ imports another zone,
                        except the one type-only kernel→contracts re-export.
  R2 commands-floor   — nothing below the command surface (kernel,
                        platforms, core, daemon) imports src/commands/.
                        Generalizes the former guard (daemon + platforms).
  R3 platforms-seam   — platforms/ is statically imported only at the
                        core interactor seam (src/core/interactors/) and by
                        the daemon server; elsewhere use a dynamic import()
                        or a type-only import, preserving CLI cold-start.

Dynamic import('../platforms/*') and `import type` stay allowed.

Fixes the three pre-existing R3 violations by converting static
platforms value imports to dynamic imports (all in already-async call
sites, behavior-preserving and cold-start-improving):
  - src/client/client.ts        debug.symbols → lazy symbolicateCrashArtifact
  - src/cli/commands/web.ts      setup/doctor → lazy agent-browser-tool
  - src/core/dispatch-interactions.ts  runner-sequence → lazy (matches the
                                       file's own dynamic-import pattern)

Wire the check into the Layering Guard CI job and add a check:layering
package.json script (also folded into check:tooling). scripts/layering/**
is excluded from fallow (untested CI script, like scripts/perf/**).
2026-07-01 10:02:45 +02:00
Michał Pierzchała 26ac865c63 refactor: consolidate Apple platform internals (#968) 2026-06-30 21:30:46 +02:00
Michał Pierzchała edc8dd059b ci: automate iOS runner request-count gate for the Apple runner unwind (Phase 3 step c prep) (#966)
Replaces the manual "run with --debug, hand-count the runner phases" check with
an automated, committed assertion so the Phase 3 step (c) runner relocation (and
future runner refactors) can prove byte-identical runner request behavior.

- src/daemon/runner-request-count.ts: pure, unit-testable counter. Parses the
  daemon --debug diagnostics ndjson and counts the iOS-runner round-trip phases,
  plus baseline parse/compare logic. Owns RUNNER_ROUND_TRIP_PHASES as the single
  source of truth, now imported by request-router.ts (was a local const) so the
  in-process cost graft and the external counter never drift.
- src/daemon/__tests__/runner-request-count.test.ts: 13 unit tests over synthetic
  ndjson fixtures (tolerant parse, counting, baseline parse/compare). Run in the
  normal unit suite; no hardware.
- scripts/runner-request-count/: assertion harness (run.ts) + committed baseline
  (expected-counts.json). Drives the existing smoke-ios replay scenario with
  --debug in an isolated --state-dir, counts runner round-trips from daemon.log,
  and asserts against the baseline. --update regenerates the baseline. Infra
  hiccups are inconclusive (don't fail); only a real count drift fails.
- .github/workflows/ios.yml: new "Assert iOS runner request count" step in the
  smoke-ios job, reusing the booted simulator.
- package.json: `validate:runner-count` script. .fallowrc.json: harness entry.

The baseline ships unarmed (established=false); the harness records observed
counts (printed + uploaded as a test/artifacts artifact) without failing, so the
maintainer arms it once from a real CI run.
2026-06-30 17:26:01 +02:00
Michał Pierzchała 1cbe446df5 chore: prune public package exports (#961)
* chore: prune public package exports

* chore: drop extra facade exports

* chore: remove dead export leftovers

* docs: align public API docs
2026-06-30 15:51:37 +02:00
Michał Pierzchała 5913480779 chore: exclude helper readmes from package (#954) 2026-06-30 12:41:47 +02:00
Michał Pierzchała 78ca8fb9be 0.18.0 2026-06-26 19:53:24 +02:00
Michał Pierzchała 053bbace30 0.17.10 2026-06-25 21:48:19 +02:00
Michał Pierzchała 63e68cd596 0.17.9 2026-06-25 10:18:41 +02:00
Michał Pierzchała 7739b71a25 refactor: centralize command family facets (#849) 2026-06-24 12:40:03 +02:00
Michał Pierzchała 51eaa7fd20 0.17.8 2026-06-23 17:38:26 +02:00
Michał Pierzchała b3b8c90fda 0.17.7 2026-06-23 10:07:07 +02:00
Michał Pierzchała 02481e8b7f docs: document minimal web support (#831)
* docs: document minimal web support

* docs: clarify web support scope

* docs: remove duplicate web help entry

* docs: trim README web support notes
2026-06-22 11:01:13 +02:00
Michał Pierzchała c6fd3dc972 test: add live web platform smoke (#832)
* test: add live web platform smoke

* test: harden web smoke cleanup
2026-06-21 13:05:51 +02:00
Michał Pierzchała f9a9662f05 0.17.6 2026-06-16 16:52:40 +01:00
Michał Pierzchała 0b499f22db chore: update stale callstackincubator references 2026-06-15 20:12:03 +01:00
Michał Pierzchała 204be34dec 0.17.5 2026-06-15 20:08:26 +01:00
Michał Pierzchała 9e165114c0 perf: optimize rslib startup build (#803) 2026-06-15 08:51:47 +02:00
Michał Pierzchała 71db11b744 0.17.4 2026-06-13 20:49:01 +02:00
Michał Pierzchała 1ce2e97175 0.17.3 2026-06-13 11:48:03 +02:00
Michał Pierzchała fa1c1d55da refactor: localize command surface modules (#772)
* refactor: localize command surface modules

* refactor: localize command runtime modules

* refactor: tighten localized command exports

* refactor: address command localization review

* refactor: deepen batch command policy

* refactor: split provider progress model

* refactor: collapse command client facets

* refactor: run progress metrics as TypeScript

* refactor: remove obsolete command shims

* fix: update localized snapshot output import

* fix: preserve debug command localization
2026-06-12 14:02:04 +02:00
Michał Pierzchała 19cdec103a 0.17.2 2026-06-10 21:13:12 +02:00
Michał Pierzchała a7efcd468f chore(fallow): align local runs with the CI diff gate and upgrade to 2.91 (#735)
* chore(fallow): fit config to repo profile so baselines stay near-empty

- Raise health thresholds in .fallowrc.json to the smallest values that
  pass on a clean tree (maxCyclomatic 58, maxCognitive 77, maxCrap 591)
  instead of grandfathering ~180 findings in fallow-baselines/health.json.
- Raise duplicates.minTokens to 66, the smallest value covering the four
  tolerated clone groups (largest is 65 tokens).
- Regenerate baselines: health.json shrinks from ~18.6 KB of grandfathered
  finding counts to refactoring-target metadata only; dead-code.json is
  empty.
- Upgrade fallow 2.52.0 -> 2.91.0: 2.87.0 made ignorePatterns silence the
  "examples/test-app is not declared as a workspace" warning, which 2.52.0
  emitted regardless of config.
- Remove the unused ensureAdb export (and its now-unused imports) from
  src/platforms/android/adb.ts; it is not re-exported by any public entry
  and has no references anywhere in the repo.
- Document local (pnpm fallow) vs CI (fallow audit) usage in
  CONTRIBUTING.md.

https://claude.ai/code/session_01LXZXzxi55sZ11DSyqWyBA2

* chore(fallow): keep default thresholds, gate locally via diff-based audit

Revise the previous commit after review: pinning global thresholds at the
repo's historical maxima (cyclomatic 58, cognitive 77, CRAP 591, minTokens
66) weakened the gate for brand-new code and left zero headroom on the
worst existing functions. Restore the original design — fallow default
thresholds with legacy findings grandfathered per-file in
fallow-baselines/health.json — and fix the local-DX problem at the script
level instead:

- .fallowrc.json: drop the health/duplicates overrides so fallow defaults
  (cyclomatic 20, cognitive 15, CRAP 30, minTokens 50) apply to new code.
- fallow-baselines/health.json: regenerate at default thresholds under
  fallow 2.91 (201 grandfathered findings across 108 files); dead-code
  baseline stays empty.
- package.json: `pnpm fallow` now runs `fallow audit --base origin/main`,
  the same diff-based gate CI uses, so it passes on a clean tree. The old
  full-tree summary moves to `pnpm fallow:all` (expected to report legacy
  findings). `check:fallow` is unchanged (CI passes an explicit --base).
- CONTRIBUTING.md: correct the fallow docs accordingly.

Verified: clean tree passes; a new unused export fails the audit; a new
cyclomatic-25 function fails the audit; +1 branch growth in an already-
grandfathered function (classifyBootFailure) is absorbed by the baseline.

https://claude.ai/code/session_01LXZXzxi55sZ11DSyqWyBA2

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-10 10:53:04 +02:00
Michał Pierzchała 645577554a ci: enforce lint and formatting, add warn-only layering guard (#732)
Add a Lint & Format job (oxlint --deny-warnings + new format:check script) and
a warn-only guard that flags imports of src/commands/* from src/daemon and
src/platforms; the guard flips to a hard failure once shared contracts move
out of the commands layer.

https://claude.ai/code/session_01LXZXzxi55sZ11DSyqWyBA2

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-10 10:36:18 +02:00
Michał Pierzchała 0f7187f543 fix: scope source daemon state by worktree (#719)
* fix: scope source daemon state by worktree

* docs: clarify worktree daemon state tradeoffs

* ci: harden Apple runner cache

* chore: keep daemon state helper internal

* ci: validate Apple runner cache restores

* ci: simplify Apple runner cache setup
2026-06-10 10:32:58 +02:00
Michał Pierzchała 264c804376 0.17.1 2026-06-09 17:25:49 +02:00
Michał Pierzchała c2b29d5600 fix: stabilize Maestro replay on iOS (#713)
* fix: stabilize Maestro replay on iOS

* fix: scope iOS runner cleanup to daemon owner

* fix: lease iOS runner ownership per device

* fix: release prepared iOS runner daemon in CI

* fix: inline runner lease release cleanup
2026-06-09 14:12:57 +02:00
Michał Pierzchała fe728814c4 0.17.0 2026-06-08 18:39:25 +02:00
Michał Pierzchała 76cee982ba fix: stabilize iOS runner navigation taps (#702)
* 0.16.14

* fix: stabilize iOS runner navigation taps

* fix: recover iOS runner after AX failures

* docs: clarify AX-unavailable snapshot recovery

* test: cover synthesized ios provider taps

* test: cover iOS runner AX failure paths

* fix: clarify sparse iOS AX recovery hint
2026-06-07 11:17:27 +02:00
Michał Pierzchała 6babdfb6c5 0.16.13 2026-06-04 15:37:22 -07:00
Michał Pierzchała 1881f6f28b 0.16.12 2026-06-03 12:20:25 -07:00
Michał Pierzchała 4eeec6093d 0.16.11 2026-06-03 05:16:24 -07:00
Michał Pierzchała 6247a3d4a5 0.16.10 2026-06-02 06:42:54 -07:00
Michał Pierzchała be5081d4d6 0.16.9 2026-06-01 17:10:06 -05:00
Michał Pierzchała 37618c989e 0.16.8 2026-06-01 20:37:26 +02:00
Michał Pierzchała 9e6537200a fix: resolve test-app dependabot alerts (#649)
* fix: resolve test-app dependabot alerts

The postcss/uuid overrides added in #464 stopped applying once test-app
ended up nested under the repo-root pnpm-workspace.yaml: pnpm only honors
overrides from a workspace root, so test-app's package.json `pnpm.overrides`
were silently ignored and the lockfile drifted back to vulnerable versions.

Move the overrides into a dedicated examples/test-app/pnpm-workspace.yaml so
test-app is its own pnpm root and the overrides are honored, and add scoped
overrides for the two remaining alerts:

- postcss   8.4.49 -> 8.5.12  (XSS in CSS stringify)
- uuid      7.0.3  -> 14.0.0  (missing buffer bounds check)
- ws@8      8.20.0 -> 8.21.0  (uninitialized memory disclosure)
- brace-expansion@5  5.0.5 -> 5.0.6  (ReDoS / max bypass)

ws and brace-expansion overrides are scoped to the vulnerable majors so the
non-vulnerable ws@7 / brace-expansion@1 copies in the tree are left untouched.

* chore: drop dead lodash-es override, document test-app workspace

- Remove the no-op `lodash-es` override from the root package.json (leftover
  from #368). lodash-es is no longer in the dependency tree, so the override
  resolved to nothing; regenerating the root lockfile is a no-op.
- Add a comment to examples/test-app/pnpm-workspace.yaml explaining why the
  file exists, so it isn't "tidied away" and the override drift reintroduced.
2026-06-01 19:32:17 +02:00
Michał Pierzchała 45cfad5cc5 feat: e2e command perf benchmark harness + nightly CI (#630)
* feat: add e2e command perf benchmark harness + nightly CI

Adds scripts/perf, a cheap end-to-end perf benchmark that drives the built
CLI through an ordered Settings tour of ~24 commands for N rounds, on a fully
isolated daemon/state-dir and self-cleaning device, and emits JSON + Markdown
reports. Per-command timing comes from wrapping each batchable command in its
own single-step batch (daemon durationMs) plus wall-clock around the process.

Wires a scheduled + workflow_dispatch CI job (perf-nightly.yml) that reuses the
cached iOS XCUITest runner (setup-apple-replay) and the Android replay host, and
runs the CLI from source via --experimental-strip-types (no dist build).

* refactor(perf): drive the harness CLI via runCmdSync, not spawnSync

Review (P2): repo rule is to spawn processes through src/utils/exec.ts, not
node:child_process directly. Switch the perf harness's invokeCli to runCmdSync
(allowFailure so non-zero exits are recorded as samples) and add a maxBuffer
option to ExecOptions/runCmdSync (snapshot payloads exceed Node's ~1MB default).

* perf(harness): warm the runner after open so the first measured command is clean

The first interaction after open/relaunch pays the one-time iOS XCUITest runner
startup (~10s+ cold) and a per-relaunch first-AX-query settle cost (~4s). That was
landing on the first measured command each round (snapshot -i), inflating it ~10x
vs the next snapshot. Run an untimed warmup snapshot -i after establishSession, after
each round's reset-open, and after every freshRoot relaunch, so no measured command
absorbs runner startup. Noted in the report header.

* refactor(perf): address review + fix Fallow CI

- exec.ts: extract spawnRejectionError + commandCloseFailure helpers, deduping the
  error/close handler clones (Fallow duplication ✗ that surfaced once the maxBuffer
  change pulled exec.ts into the audit scope).
- .fallowrc: exclude scripts/perf/** (non-shipped benchmark tooling, like examples/
  test-app) so its naturally-moderate functions don't trip the complexity gate.
- config.ts: drop unused exports CLI_BIN/DEFAULT_OUT_DIR; add readIntValue so
  --n/--rounds/--warmup report the actual flag + reject non-integers clearly.
- harness.ts: extract toSample(); type sampleError param as CliResult.
- scenario.ts: ScenarioStep is now a discriminated union on execMode (removes step.step!/
  step.args ?? []).
- comment/legend rewords (platform defaults are local-convenience/CI-overridden;
  elements = node count). check:fallow now green; typecheck/lint/unit pass.

* perf(harness): downgrade sample ok when a batch step reports ok:false

Defensive belt-and-suspenders for the Codex review note: stop-only batch already
surfaces a failed step as a top-level failure (caught by invokeCli), but if an
on-error=continue mode ever keeps the batch ok while a step fails, don't silently
count that step as a successful sample — derive ok from the step's own result.ok.
2026-05-31 14:37:59 +02:00
Michał Pierzchała efc0b213d4 0.16.7 2026-05-31 12:22:28 +02:00
Michał Pierzchała 1bdd9eeb49 0.16.6 2026-05-31 11:51:59 +02:00