Commit Graph

664 Commits

Author SHA1 Message Date
Michał Pierzchała 1efa8253d2 feat: add replay test sharding 2026-06-08 10:18:46 +02:00
Michał Pierzchała 5c083eacc6 fix: harden iOS replay runner prewarm (#705)
* fix: harden iOS replay runner prewarm

* fix: avoid stale iOS runner during relaunch

* fix: stop stale iOS runner processes

* fix: clean stale iOS runners before startup
2026-06-07 20:37:23 +02:00
Michał Pierzchała 81448c8f97 feat: add perf metrics and frames commands (#703)
* feat: add perf metrics and frames commands

* fix: tighten perf command ergonomics

* test: move perf area coverage to provider integration
2026-06-07 20:36:57 +02:00
Michał Pierzchała 86971990cb fix: scope runner diagnostics to sessions (#704) 2026-06-07 11:35:38 +02:00
Michał Pierzchała 76cee982ba fix: stabilize iOS runner navigation taps (#702)
* 0.16.14

* fix: stabilize iOS runner navigation taps

* fix: recover iOS runner after AX failures

* docs: clarify AX-unavailable snapshot recovery

* test: cover synthesized ios provider taps

* test: cover iOS runner AX failure paths

* fix: clarify sparse iOS AX recovery hint
2026-06-07 11:17:27 +02:00
Michał Pierzchała aa5b07faa5 fix: keep iOS runner hot across app closes (#700)
* fix: keep iOS runner hot across app closes

* fix: keep iOS snapshots fast after relaunch

* fix: bound compact iOS snapshots on broken AX trees

* docs: clarify iOS snapshot backend strategy

* test: update Settings replay selectors

* fix: stabilize Settings replay selectors

* fix: fall back for selector interactions

* chore: simplify flat interactive snapshot query
2026-06-06 21:26:12 +02:00
Michał Pierzchała f2424f9d3a refactor: centralize daemon command registry (#693) 2026-06-05 18:34:01 +02:00
Michał Pierzchała 7d7b467699 fix: retry iOS runner prepare launch (#692) 2026-06-05 18:05:04 +02:00
Michał Pierzchała 36012a9e6f fix: update docs router dependency (#691) 2026-06-05 14:49:21 +02:00
Michał Pierzchała 6babdfb6c5 0.16.13 v0.16.13 2026-06-04 15:37:22 -07:00
Michał Pierzchała ad7b386444 feat: cache iOS runner artifacts during prepare (#688)
* feat: cache ios runner artifacts during prepare

* refactor: deepen ios runner lifecycle

* refactor: generalize apple runner prepare

* refactor: simplify apple runner lifecycle

* fix: clarify runner prepare recovery diagnostics

* refactor: trim apple runner provider surface

* test: simplify runner recovery diagnostics assertion
2026-06-04 15:37:05 -07:00
Michał Pierzchała bf21540b9d refactor: consolidate android gesture backend selection (#689) 2026-06-04 15:15:34 -07:00
Michał Pierzchała 7ab9998645 fix: stabilize Maestro post-gesture snapshots (#681)
* fix: stabilize maestro post-gesture snapshots

* fix: harden maestro ci diagnostics and ios prepare

* fix: preserve android freshness baseline for optimized taps

* fix: close ios runner host after sessionless commands

* fix: require tracked app for ios snapshots

* fix: keep in-page swipes on visible content

* test: align lock policy probes with ios snapshot guard

* fix: compose post-gesture snapshots with android freshness

* refactor: clarify post-action snapshot policies

* fix: avoid redundant swipe stabilization flag

* fix: route android swipes through gesture helper

* test: cover android swipe helper fallback
2026-06-04 14:44:33 -07:00
Michał Pierzchała 1881f6f28b 0.16.12 v0.16.12 2026-06-03 12:20:25 -07:00
Michał Pierzchała 3a93071748 docs: update public documentation links (#684) 2026-06-03 12:01:23 -07:00
Michał Pierzchała 86db7e837f fix: align iOS runner cache target metadata (#682)
* fix: align iOS runner cache target metadata

* test: surface iOS cache metadata script stderr
2026-06-03 09:35:34 -07:00
Michał Pierzchała 4eeec6093d 0.16.11 v0.16.11 2026-06-03 05:16:24 -07:00
Michał Pierzchała 86687feba6 fix: restore snapshot request timeout (#680) 2026-06-03 05:15:55 -07:00
Michał Pierzchała 7400701857 perf: speed up iOS swipes and harden runner cache (#676)
* perf: speed up ios swipes and harden runner cache

* fix: harden maestro replay smoke tests
2026-06-03 04:59:13 -07:00
Michał Pierzchała 6946c2f156 fix: allow fresh session device binding (#677) 2026-06-02 20:16:42 -07:00
Michał Pierzchała 491ad7e922 fix: improve session ownership and recovery guidance (#674)
* fix: improve session recovery guidance

* fix: quote session recovery commands

* test: simplify record session cleanup fixtures

* test: cover recovery hint edge cases

* test: trim duplicate recovery quoting coverage

* fix: scope implicit sessions by caller workspace

* fix: surface session state directory

* fix: remove unused session store check

* test: trim duplicate session recovery coverage
2026-06-02 14:06:00 -07:00
Michał Pierzchała 041b48222f feat: add iOS runner prepare command (#673)
* feat: add ios runner prepare command

* fix: shorten snapshot request timeout

* fix: refine prepare and snapshot timeouts

* test: cover prepare ios runner provider scenario
2026-06-02 12:08:34 -07:00
Michał Pierzchała 6f637d0e93 fix: filter covered Android snapshot surfaces (#675)
* fix: filter covered android snapshot surfaces

* fix: guard android drawing order metadata
2026-06-02 12:01:34 -07:00
Michał Pierzchała 0313262a6a docs: clarify physical device signing help (#672) 2026-06-02 08:50:35 -07:00
Michał Pierzchała 6247a3d4a5 0.16.10 v0.16.10 2026-06-02 06:42:54 -07:00
Michał Pierzchała 3847f71c50 perf: avoid android hierarchy probe for scroll (#671) 2026-06-02 06:41:53 -07:00
Michał Pierzchała 66119e21bb perf: adapt ios runner uptime preflight (#662) 2026-06-01 21:15:15 -07:00
Michał Pierzchała 7f035f34b4 fix: reduce ios runner invalidation after status recovery (#666) 2026-06-01 20:58:50 -07:00
Michał Pierzchała 7e30a83057 fix: tune iOS runner response retention (#665)
* fix: tune iOS runner response retention

* fix: require explicit runner response retention
2026-06-01 20:24:04 -07:00
Michał Pierzchała 3ae2472d44 fix: keep iOS runner status transport visible (#663) 2026-06-01 20:23:29 -07:00
Michał Pierzchała f2ef68871c docs(ios): note idleTimeout: false guardrail + why swipe/mouseClick bypass performGesture (review nits) (#664) 2026-06-01 21:27:07 -05:00
Michał Pierzchała 57cd3f3a07 perf: recover iOS runner responses by status (#661)
* perf: recover ios runner responses by status

* docs: plan ios runner protocol optimizations

* fix: clarify ios runner recovery errors

* fix: retry read-only in-flight runner commands

* refactor: split ios runner status recovery handling
2026-06-01 21:26:15 -05:00
Michał Pierzchała daaf71a661 refactor(ios): safely(tag:default:) wrapper for the catch-log-and-default band-aid (#660)
Consolidate the repeated `var x = default; catchException({ x = expr }); if let m {
NSLog("..._IGNORED_EXCEPTION=%@", m); return default }; return x` shape used around
exception-prone XCUITest queries into one generic helper (RunnerTests+Exceptions.swift):

  func safely<T>(_ tag:, _ fallback:, _ block:) -> T
  func safely<T>(_ tag:, _ block: () -> T?) -> T?   // nil-default convenience

11 uniform sites adopt it: SystemModal (4), Snapshot (2), TextEntry (2), Interaction (3).
Each drops from ~7-12 lines to 1-3. The NSLog format is preserved byte-for-byte via the tag arg
(tag "MODAL_QUERY" -> "AGENT_DEVICE_RUNNER_MODAL_QUERY_IGNORED_EXCEPTION=%@"), so the
silently-logged-and-continued path keeps its searchable format and now has a single place to add
per-tag exception telemetry.

Left inline by design (not the uniform pattern): the silent `_ = catchException` KVC reads
(elementHasFocus, snapshotHasFocus), the throw-on-AX snapshot path, executeOnMainSafely's
retry-driving catch, and the bespoke pressKeyboardReturn fallback / performElementTap branches
whose result depends on the exception message.

Behavior-preserving: same defaults, same log output, same returned values. catchException is
non-escaping, so the inout capture in safeIsActionableCandidate is preserved.

Verified: xcodebuild build-for-testing -> TEST BUILD SUCCEEDED.
2026-06-01 21:25:43 -05:00
Michał Pierzchała 3f65124ea4 refactor: add iOS runner lifecycle protocol (#658)
* refactor: add ios runner lifecycle protocol

* fix: avoid journaling ios runner status probes

* fix: cap ios runner journal responses

* perf: avoid snapshot journal serialization

* perf: skip ids for ios runner status probes

* refactor: keep ios runner status off main thread
2026-06-01 20:40:09 -05:00
Michał Pierzchała cbe725d49b refactor(ios): gesture-response factory + performGesture wrapper in CommandExecution (#659)
Collapse the gesture-command boilerplate in the executeOnMain switch behind two seams:
- performGesture(app, idleTimeout:) -> (timing, outcome): folds measureGesture + the scroll
  idle-timeout/quiescence-skip wrap. Parameterized: touch gestures wrap (idleTimeout: true,
  default); synthesis gestures (pinch/rotate/transform) pass idleTimeout: false because
  RunnerSynthesizedGesture governs its own timing — a distinction that was previously implicit.
- gestureResponse(message:timing:frame:) over a GestureFrame (none/touch/drag): one factory for
  the success DataPayload (message + gesture timing + optional touch/drag visualization frame).

The 13 gesture cases (tap x3, tapSeries x2, longPress, drag, dragSeries, swipe, mouseClick, pinch,
rotateGesture, transformGesture) now use the seams; ~120 lines of repeated quartet (measureGesture +
idle-wrap + unsupported check + 8-field DataPayload) collapse. CommandExecution.swift 949 -> 867.

Behavior-preserving: identical outcomes, timing capture, DataPayload fields/messages, and the same
touch-wrapped vs synthesis-unwrapped distinction. No wire change (the factory emits the already
wire-locked DataPayload internally). mouseClick (throws) and swipe (returns an optional frame) keep
their bespoke measure/wrap but route the success payload through gestureResponse.

Verified: xcodebuild build-for-testing -> TEST BUILD SUCCEEDED (no warnings).
2026-06-01 20:32:18 -05:00
Michał Pierzchała e96de318cd fix: stabilize android perf harness setup (#657) 2026-06-01 20:10:09 -05:00
Michał Pierzchała b1aaa3a4b6 chore: update glama mcp metadata (#655) 2026-06-01 19:29:50 -05:00
Michał Pierzchała 25d5a2a72e fix: bound iOS simulator app termination (#654) 2026-06-01 19:24:11 -05:00
Michał Pierzchała be5081d4d6 0.16.9 v0.16.9 2026-06-01 17:10:06 -05:00
Michał Pierzchała ef3c5121af fix: align Maestro assertVisible timeout (#653) 2026-06-01 17:08:00 -05:00
Michał Pierzchała 4d016db3b8 refactor(ios): extract the text-entry engine into RunnerTests+TextEntry.swift (#651)
Behavior-preserving relocation: move the text-entry concern out of the (formerly 1805-line)
RunnerTests+Interaction.swift — the repo's most-changed file — into a new
RunnerTests+TextEntry.swift. Interaction.swift drops to 1068 lines (-737).

Moved verbatim (no logic changes):
- value types: TextTypingRepairMode, TextEntryTiming, TextEntryResult, TextEntryTarget
- the focus -> type -> verify -> repair pipeline (typeTextReliably + focus orchestration +
  readiness polling + dropped-char/repair heuristics)
- clearTextInput and the text-entry leaf helpers (editableTextValue, isPlaceholderValue,
  isGenericTextInputLabel, normalizedElementText, moveCaretToEnd, estimatedDeleteCount,
  keyboardBecameVisible, keyboardElementExists)

The whole cluster is text-entry-exclusive (its only callers are within the moved code or the
already-internal command entry points), so every symbol keeps its original visibility — no
access widened. Shared helpers used by gestures/snapshot/get-text (isKeyboardVisible,
visibleKeyboardFrame, textInputAt, textInputCandidatesAt, readableText, ...) stay in
Interaction. The new file is auto-included via the project's file-system-synchronized group.

This is the safe, cosmetic half of the "TextEntry engine" architecture candidate. A deeper
extraction behind a real seam/type is intentionally deferred: it carries the #245 revert risk
and wants stronger text-entry e2e coverage first.

Verified: xcodebuild build-for-testing -> TEST BUILD SUCCEEDED. Pure relocation, no behavior change.
2026-06-01 16:53:21 -05:00
Michał Pierzchała d5c94b0296 fix: improve maestro test output (#652) 2026-06-01 16:51:25 -05:00
Michał Pierzchała 37618c989e 0.16.8 2026-06-01 20:37:26 +02:00
Michał Pierzchała 2546ca3103 fix: dismiss RN overlays in Maestro compat (#650) 2026-06-01 20:34:14 +02:00
Michał Pierzchała 9e6537200a fix: resolve test-app dependabot alerts (#649)
* fix: resolve test-app dependabot alerts

The postcss/uuid overrides added in #464 stopped applying once test-app
ended up nested under the repo-root pnpm-workspace.yaml: pnpm only honors
overrides from a workspace root, so test-app's package.json `pnpm.overrides`
were silently ignored and the lockfile drifted back to vulnerable versions.

Move the overrides into a dedicated examples/test-app/pnpm-workspace.yaml so
test-app is its own pnpm root and the overrides are honored, and add scoped
overrides for the two remaining alerts:

- postcss   8.4.49 -> 8.5.12  (XSS in CSS stringify)
- uuid      7.0.3  -> 14.0.0  (missing buffer bounds check)
- ws@8      8.20.0 -> 8.21.0  (uninitialized memory disclosure)
- brace-expansion@5  5.0.5 -> 5.0.6  (ReDoS / max bypass)

ws and brace-expansion overrides are scoped to the vulnerable majors so the
non-vulnerable ws@7 / brace-expansion@1 copies in the tree are left untouched.

* chore: drop dead lodash-es override, document test-app workspace

- Remove the no-op `lodash-es` override from the root package.json (leftover
  from #368). lodash-es is no longer in the dependency tree, so the override
  resolved to nothing; regenerating the root lockfile is a no-op.
- Add a comment to examples/test-app/pnpm-workspace.yaml explaining why the
  file exists, so it isn't "tidied away" and the override drift reintroduced.
2026-06-01 19:32:17 +02:00
Michał Pierzchała 3785a17554 fix(ios): unify multi-touch gestures on two-finger synthesis + hinted unsupported errors (#645)
* fix(ios): unify multi-touch gestures on two-finger synthesis + hinted unsupported errors

Make RunnerSynthesizedGesture the single iOS multi-touch engine and drop the older
incompatible models:

- rotateGesture now drives the two-finger XCTest synthesis path (dx=dy=0, scale=1,
  degrees), mirroring the pinch migration in #634. The native XCUIElement.rotate(withVelocity:)
  injected a single synthetic rotation that React Native's rotation recognizer did not read
  reliably; synthesis fixes it. velocity is ignored on iOS (kept in the wire contract for
  compatibility; rotation direction comes from the sign of degrees).
- pinch is now synthesis on iOS and a clear UNSUPPORTED_OPERATION on tvOS/macOS. The macOS
  coordinate double-tap+drag heuristic (performCoordinatePinch) is removed: synthesis is
  iOS-only, so macOS multi-touch is reported honestly as unsupported rather than approximated.
- RunnerInteractionOutcome.unsupported now carries an actionable hint, mapped to ErrorPayload.hint
  (#639). Every unsupported gesture/tvOS path returns a concise message plus a next-step hint
  (existing messages kept verbatim).

Net: on iOS, pinch/rotate/transform all flow through one synthesis primitive. swipe/scroll/pan/
fling remain single-finger drags (correct, unchanged).

Coverage: examples/test-app/replays/gesture-lab.ad exercises pinch + rotate against the gesture
lab and asserts "pinch changed yes" / "rotate changed yes".

* fix(ios): fail-fast macOS pinch + align capability/docs with synthesis-only

Follow-through for removing the macOS coordinate pinch path (the runner now returns
UNSUPPORTED_OPERATION for macOS pinch): reject it at admission instead of round-tripping.

- capabilities.ts: pinch now matches rotate-gesture/transform-gesture (Android + iOS
  simulator only); macOS dropped. Removes the now-unused isMacOsOrMobileAppleSimulator helper.
- capabilities.test.ts: pinch expected unsupported on macOS and tvOS.
- website/docs/docs/commands.md: pinch listed for Android + iOS simulators only (removed from
  the macOS app-session list); documents that iOS rotate ignores the optional velocity arg
  (synthesis uses a fixed duration; direction comes from the sign of degrees).

Addresses PR #645 review HIGH #2 and MEDIUM #3.

* fix(ios): surface a hinted unsupported error for synthesis gestures at admission

Removing macOS pinch from the capability matrix makes macOS pinch (and the already-excluded
rotate-gesture/transform-gesture on macOS/tvOS/physical iOS) fail fast in ensureGenericCommandReady
before reaching the runner. That left the runner's macOS-specific hint unreachable on the daemon
path, so callers only saw the generic "<cmd> is not supported on this device".

Add an optional unsupportedHint to the capability matrix and surface it at admission, so the
synthesis-only gestures fail fast (no runner round-trip) AND return an actionable hint pointing to
where they work (Android + iOS simulator). Applied to pinch / rotate-gesture / transform-gesture.

Addresses PR #645 review (P2: route macOS pinch to the hinted failure).
2026-06-01 19:30:36 +02:00
Michał Pierzchała eb3ca302bd fix: normalize maestro loading ellipsis (#648) 2026-06-01 19:09:33 +02:00
Michał Pierzchała 5d2b2ed5c7 fix: harden react native overlay dismissal (#641) 2026-06-01 18:24:37 +02:00
Michał Pierzchała 4f8e0af898 fix: improve maestro test output (#647) 2026-06-01 18:23:41 +02:00
Michał Pierzchała 5492cf4642 refactor(ios): single CommandTraits table for runner command classification (#642)
* refactor(ios): single CommandTraits table for runner command classification

Replace the three hand-maintained switches in RunnerTests+Lifecycle.swift
(isInteractionCommand / isReadOnlyCommand / isRunnerLifecycleCommand) with one
source of truth: CommandType.traits, an exhaustive switch returning a
CommandTraits struct (interaction / readOnly / lifecycle axes), collocated with
CommandType in RunnerTests+Models.swift.

Pure refactor: every command's classification is reproduced verbatim, and the
three predicates become one-line lookups with unchanged signatures, so call
sites are untouched. The exhaustive switch makes it a compile error to add a
CommandType without classifying it, closing the drift that historically let
tapSeries/dragSeries/keyboardReturn fall out of isInteractionCommand.

readOnly is a 3-state enum (.always/.never/.conditional); .conditional preserves
alert's action-dependent read-only behavior, resolved in isReadOnlyCommand.
Classification feeds ADR-0002 session invalidation (the read-only retry that
nulls currentApp/currentBundleId), so behavior is intentionally unchanged.

Adds the "Runner command traits" term to CONTEXT.md.

* docs(ios): note CommandTraits.readOnly .conditional is alert-only (review follow-up)

* fix(ios): classify tapSeries/dragSeries/keyboardReturn as interaction commands (#643)

* fix(ios): classify tapSeries/dragSeries/keyboardReturn as interaction commands

tapSeries and dragSeries are the series forms of tap/drag (already interaction
commands); keyboardReturn is the sibling of keyboardDismiss (already an
interaction command). All three were missing from the historical
isInteractionCommand switch — a drift the new CommandTraits table (#642) makes
visible. Classifying them as interaction commands gives them the foreground-guard
+ stabilization preflight that their single-shot/sibling forms already get.

Behavior change: these three commands now re-activate a backgrounded target to
foreground and pay the stabilization delays before running. Ships separately from
the CommandTraits refactor (#642) and should land after that bakes.

mouseClick left unchanged: macOS-only and the foreground guard interacts with
bespoke macOS activation, so it needs a macOS smoke check first.

* test: cover iOS runner series commands in perf harness
2026-06-01 18:11:04 +02:00