* feat: add e2e command perf benchmark harness + nightly CI
Adds scripts/perf, a cheap end-to-end perf benchmark that drives the built
CLI through an ordered Settings tour of ~24 commands for N rounds, on a fully
isolated daemon/state-dir and self-cleaning device, and emits JSON + Markdown
reports. Per-command timing comes from wrapping each batchable command in its
own single-step batch (daemon durationMs) plus wall-clock around the process.
Wires a scheduled + workflow_dispatch CI job (perf-nightly.yml) that reuses the
cached iOS XCUITest runner (setup-apple-replay) and the Android replay host, and
runs the CLI from source via --experimental-strip-types (no dist build).
* refactor(perf): drive the harness CLI via runCmdSync, not spawnSync
Review (P2): repo rule is to spawn processes through src/utils/exec.ts, not
node:child_process directly. Switch the perf harness's invokeCli to runCmdSync
(allowFailure so non-zero exits are recorded as samples) and add a maxBuffer
option to ExecOptions/runCmdSync (snapshot payloads exceed Node's ~1MB default).
* perf(harness): warm the runner after open so the first measured command is clean
The first interaction after open/relaunch pays the one-time iOS XCUITest runner
startup (~10s+ cold) and a per-relaunch first-AX-query settle cost (~4s). That was
landing on the first measured command each round (snapshot -i), inflating it ~10x
vs the next snapshot. Run an untimed warmup snapshot -i after establishSession, after
each round's reset-open, and after every freshRoot relaunch, so no measured command
absorbs runner startup. Noted in the report header.
* refactor(perf): address review + fix Fallow CI
- exec.ts: extract spawnRejectionError + commandCloseFailure helpers, deduping the
error/close handler clones (Fallow duplication ✗ that surfaced once the maxBuffer
change pulled exec.ts into the audit scope).
- .fallowrc: exclude scripts/perf/** (non-shipped benchmark tooling, like examples/
test-app) so its naturally-moderate functions don't trip the complexity gate.
- config.ts: drop unused exports CLI_BIN/DEFAULT_OUT_DIR; add readIntValue so
--n/--rounds/--warmup report the actual flag + reject non-integers clearly.
- harness.ts: extract toSample(); type sampleError param as CliResult.
- scenario.ts: ScenarioStep is now a discriminated union on execMode (removes step.step!/
step.args ?? []).
- comment/legend rewords (platform defaults are local-convenience/CI-overridden;
elements = node count). check:fallow now green; typecheck/lint/unit pass.
* perf(harness): downgrade sample ok when a batch step reports ok:false
Defensive belt-and-suspenders for the Codex review note: stop-only batch already
surfaces a failed step as a top-level failure (caught by invokeCli), but if an
on-error=continue mode ever keeps the batch ok while a step fails, don't silently
count that step as a successful sample — derive ok from the step's own result.ok.
* feat: forward --launch-args to adb shell am start on Android
Stacked on top of the iOS-only --launch-args PR (#598). Removes the
Android UNSUPPORTED_OPERATION guard added with the Maestro work and
threads launchArgs through all five Android open paths.
Per-path threading:
- openAndroidPackage (-p package launch + activity-fallback)
- openAndroidPackageActivity (-n component override)
- openAndroidIntent (named intent action)
- openAndroidDeepLink (-a VIEW -d <url>, with optional -p)
- openAndroidAppBoundDeepLink (-a VIEW -d <url> -p <resolved>)
`adb shell` joins its argv with spaces and feeds the result to a
device shell, which re-tokenises. The other am-start arguments are
well-known and never contain shell-significant characters, so they
round-trip untouched. Launch arguments are user-supplied and may
contain JSON, spaces, `#`, etc.; each is single-quoted unless it
consists entirely of safe shell characters (the same approach long
used in adb-driven tooling for the same reason).
Help text on --launch-args is updated to describe the Android shape
(`adb shell am start args, e.g. --es key value` for typed Intent
extras) and macOS remains the only rejected platform.
Tests:
- src/platforms/android/__tests__/index.test.ts: five new tests
cover package, activity-override, deep-link URL, app-bound URL,
and JSON-with-shell-metacharacters quoting paths.
- src/core/__tests__/dispatch-open.test.ts: the previous
"rejects Android launch arguments" test is inverted into a
forwarding test that asserts openAndroidApp receives the args.
Validated end-to-end on a Pixel emulator running a debug build whose
launcher activity reads an Intent extra to bootstrap test
configuration: a JSON value containing `#`, `/`, `:` survived
single-quoted transit through `adb shell` and arrived at the
activity unchanged.
* fix: reject unsupported launch args openings
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
* test: remove 151 unit tests fully covered by integration tests
Per-test coverage analysis confirmed every statement and branch these
tests exercise is already covered by the provider-integration suite.
Removes 8 fully redundant test files and 132 individual tests from
54 files (2,598 lines). Coverage unchanged at 82.4% statements / 84.5%
lines, both above configured thresholds.
* fix: restore package export-map contract test
This test validates the package.json exports field, not runtime code.
Its Istanbul fingerprint is empty, making it trivially "subsumed" by
the coverage analysis — but no integration test checks the export map.
* fix: remove unused imports left behind by test pruning
* fix: restore snapshot-routing contract test, remove orphaned comment
snapshot-routing.test.ts validates that every command in the snapshot
catalog has a handler — a structural check invisible to Istanbul.
Same false-positive pattern as package-exports.test.ts.
Michał Pierzchała