Commit Graph

23 Commits

Author SHA1 Message Date
Michał Pierzchała 11e0a1f187 feat: add WebView accessibility lab (#1397)
* feat: add WebView accessibility lab

* refactor: tighten iOS snapshot presentation rules

* fix: preserve semantic WebView containers
2026-07-25 11:40:27 +02:00
Michał Pierzchała 19cea66c8b chore(deps): resolve dependabot alerts (#1372)
Pin transitive dependencies past their vulnerable ranges via pnpm
overrides, scoped to the affected major so unrelated majors elsewhere
in the tree stay untouched:

- undici 7.24.7 -> 7.28.0 (root/website): @limrun/api pins undici to
  an exact version even in its latest release (0.44.0), so bumping the
  direct dependency can't fix this; override the transitive resolution
  instead.
- shell-quote 1.8.4 -> 1.10.0 (examples/test-app)
- js-yaml 4.1.1 -> 4.3.0 (examples/test-app)
- brace-expansion 5.0.6 -> 5.0.7 (examples/test-app)
- @babel/core 7.29.0 -> 7.29.7 (examples/test-app)
- ws 7.5.10 -> 7.5.13 (examples/test-app)

Resolves all 13 open Dependabot alerts (7 high, 3 moderate, 3 low).
2026-07-23 09:56:24 +02:00
Michał Pierzchała ef118b9d11 ci(test-app): fingerprint-keyed build cache — disk locally, Release artifacts in CI (#1321)
Splits the test app's build caching by context instead of running one remote
cache for both.

Locally, `expo run:*` caches the native build on disk via the
expo-build-disk-cache provider, keyed by the Expo fingerprint. A second run with
no native change reuses the first build; a screen edit never rebuilds, because
Metro serves JS. This is the original ask — "next time we don't build unless
native changes" — and needs no token, no network, and no custom provider.

In CI, test-app-build-cache.yml builds a Release binary per platform when the
fingerprint has no artifact yet, and publishes it as a GitHub Actions artifact
named `fingerprint.<hash>.<platform>`. Release, not dev-client, so the JS bundle
is embedded and a consuming job needs no Metro. setup-fixture-app installs it by
downloading the artifact and refreshing the JS with @expo/repack-app, so keying
on the native-only fingerprint stays correct — a JS-only change reuses the same
native binary in seconds. It falls back to an inline build when no artifact
exists yet, so a caller is never left without an app.

Release removes the sharp edges the dev-client cache needed. Its simulator .app
is universal (x86_64+arm64) rather than the active-arch-only slice a debug build
emits, so no architecture tag. It links against the SDK but loading is gated by
the deployment target, which the fingerprint already covers, so no toolchain
tag. And the CLI only narrows *debug* builds to the device ABI, so a Release APK
spans every ABI without the undocumented --all-arch flag. The artifact name
collapses to fingerprint plus platform.

This deletes build-cache-provider.js entirely — with it goes the custom Expo
provider that had to reach GitHub from inside @expo/cli, and every workaround
that forced: the fetch-nodeshim User-Agent shim, the arch/Xcode identity, the
upload-intent handoff. CI now talks to the artifacts API with plain `gh api`
outside the patched fetch, and locally the disk cache never hits the network.

The fingerprint comes from @expo/fingerprint's own `fingerprint:generate` (no
--platform, matching what @expo/cli hashes). Gitignoring /ios and /android is
what makes it machine-independent: the library asks the VCS whether the platform
markers are ignored and, concluding CNG, skips hashing them — so a developer's
prebuild output and a fresh CI checkout agree.

conformance-differential consumes setup-fixture-app, so it gains
`permissions: actions: read` for the artifact lookup.

The artifact lookup is non-fatal: a query outage leaves the id empty and
falls through to an inline build like a miss does, rather than exiting the
composite under set -e and turning a cache blip into a caller failure.
test/scripts/setup-fixture-app-fallback-smoke.sh drives that step's real shell
against a failing gh and asserts source=build; ci.yml runs it.
2026-07-18 09:36:32 +02:00
Michał Pierzchała a84caa8182 test: give the tap-retry differential a fixture control that forces the retry (#1327)
* test: give the tap-retry differential a fixture control that forces the retry

tap-retry-if-no-change was parked in #1289 for being a coin flip: tapRetries
measured 0 in run 29504440599 and 1 in 29510020718 with no change to the flow or
commit. This re-adds it with a control that holds still, so the retry fires every
run.

The original diagnosis (a dynamic cart badge in the tapped title's subtree) had
the right shape but the wrong scope. maestroSnapshotSignature hashes EVERY node
on screen, not the tapped subtree, so no "static region" of the home screen could
have worked. The actual coin flip is the gesture lab's remote image
(reactnative.dev/img/logo-share.png): whether it lands before or after the tap
decides whether the engine sees "changed" and skips the retry.

So the fixture gets a dedicated inert surface — no state, effects, timers,
images, or pressables — presented as a full-screen modal so iOS detaches the
presenting screen and the tab bar's live badges leave the hierarchy too. On a
real run it is 9 nodes against Settings' 55.

Reached by a launcher on Settings via the Settings TAB, which is a deliberate
choice twice over. A deep link would have kept the launcher out of every other
screen's snapshot, but simctl openurl raises a SpringBoard "Open in app?"
confirmation on iOS 26 even cold, and Maestro's openLink goes through the same
path. And home's "Open settings" button sits below the fold, so reaching it needs
scrollUntilVisible — the engine bug already waived under #1299.

The flow carries no waitForAnimationToEnd: a navigating tap defers a stability
requirement that the next tap settles before resolving its target, so the
baseline signature is already captured on a settled screen. Adding one fails the
flow outright, which is a real engine divergence filed as #1326.

Verified on device (iPhone 17 Pro Max, iOS 26.2, Maestro 2.5.1): 10/10
consecutive differential runs ok, tapRetries [0,0,1] every run — the two
navigating taps correctly do not retry, the inert tap retries exactly once. A
single green run proves nothing here, which is the trap #1300 fell into.

The parking guard in invariants.test.ts is replaced by three guards: the scenario
stays active, carries its tapRetries invariant, and is never waived by a
knownDivergence — a flaky scenario must be fixed, not declared.

Fixes #1300

* chore: gitignore expo prebuild output in the test app

Building the fixture app locally (what .github/actions/setup-fixture-app does in
CI) runs expo prebuild and generates examples/test-app/ios/. It is generated and
untracked but not ignored, so it shows up in git status and a `git commit -a`
would sweep the whole native project in. Same for android/ when building there.

Scoped to examples/test-app, so the repo-root android/ — which holds real tracked
sources like android/ime-helper — is unaffected. Nothing is tracked under either
path today, and the CI fixture-app cache key hashes src/**, app/**, modules/**
and the config/lockfiles, so it does not reference these and is unaffected.
2026-07-17 14:29:04 +02:00
Michał Pierzchała 117f78107e feat: add direct Limrun provider runtime (#1278)
* feat: add direct Limrun cloud runtime

* refactor: reuse Android provider runtime for Limrun

* refactor: pass runner context to provider runtimes

* fix: remove Android gesture swipe fallback

* fix: reconcile Limrun direct runtime with main

* refactor: compose Android provider interactors in core

* fix: satisfy packaged Limrun runtime checks

* perf: load Limrun provider runtime on demand

* docs: document Limrun device cloud flow

* refactor: reuse Android reverse provider for Limrun

* fix: isolate provider-owned iOS sessions

* fix: preserve provider runtime boundaries

* refactor: split close repair lifecycle

* fix: reject unavailable provider leases

* fix: reconcile provider runtime review feedback

* test: stabilize alert deadline smoke assertion

* fix: recover expired provider leases

* fix: limit Limrun to remote simulators

* fix: make Limrun provider cleanup durable

* test: cover Limrun connect through CLI

* fix: make provider expiry recovery durable

* refactor: remove Limrun compatibility cleanup

* fix: release live provider leases on expiry
2026-07-16 19:26:20 +02:00
Kenichi Saito 236016ed8a fix(ios): support remote-hosted alerts on physical devices (#1232)
* fix(ios): probe remote-hosted system modals (AccessorySetupKit picker) when the springboard mirror yields no hittable actions

* fix(ios): fail closed on host state, guard dismissal re-query, unit-test probe routing

Addresses review on #1232:
- Gate the remote-host probe to a foreground host
  (RemoteHostedSystemModalPolicy.isEligibleHostState); background/unknown hosts
  fail closed instead of substituting an unrelated action tree.
- Wrap the alert-resolution fallback query in safeElementsQuery so a dismissed
  remote host raising kAXErrorServerNotFound is absorbed.
- Extract routing/gating into RemoteHostedSystemModalPolicy and add
  simulator-free unit tests under AGENT_DEVICE_RUNNER_UNIT_TESTS.

* refactor(ios): centralize blocking system modal resolution

* fix(ios): bound alert dismissal rechecks

* feat(ios): enable alerts on physical devices

* fix(ios): bound alert system modal resolution

* test(ios): add AccessorySetupKit picker fixture

* fix(ios): validate remote-hosted system modal interactions

* chore: keep pnpm checks non-interactive

* fix(ios): share alert command deadline

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
2026-07-14 17:19:17 +02:00
Michał Pierzchała f474f0784e feat: unify gesture planning and multi-touch execution (#1212)
* feat: unify gesture planning and multi-touch execution

* fix: correct unified gesture helper behavior

* refactor: tighten unified gesture architecture

* fix: preserve gesture routing contracts

* test: account for fresh gesture viewport

* refactor: remove retired gesture series

* fix: preserve example app navigation targets

* test: reconcile unified gestures with helper ownership

* docs: update Android helper gesture protocol

* fix: refresh Maestro percentage swipe frames

* refactor: remove stale Maestro frame cache

* fix: harden unified gesture execution

* fix: model gesture viewport in providers

* refactor: remove legacy gesture paths

* fix: remove unused swipe preset parser

* refactor: tighten unified gesture boundaries

* fix: close gesture review gaps

* fix: preserve gesture compatibility contracts

* fix: preserve multi-touch recording semantics

* fix: refresh Apple runner state after app relaunch

* test: lock Apple fling fallback route

* fix: close Apple runner review gaps

* refactor: tighten unified gesture seams

* refactor: consolidate gesture planning policy

* fix: preserve swipe response compatibility

* fix: keep gesture lab aligned with replay coordinates
2026-07-13 13:16:38 +02:00
Michał Pierzchała 1d916b1d76 fix: capture ios device logs with devicectl console (#1026)
* fix: report unsupported ios device log streaming

* docs: clarify test app device verification

* docs: clarify alternate metro port flow

* refactor: split app log diagnostics helpers

* refactor: tighten app log diagnostics shape

* fix: capture ios device logs with devicectl console

* fix: tighten ios device log capture state

* refactor: clarify ios log capture support flow

* refactor: consolidate ios log diagnostics flow
2026-07-02 21:03:35 +02:00
Michał Pierzchała 4cd40aa621 feat: polish replay test progress reporter (#998)
* feat: polish replay test progress reporter

* test: stabilize replay reporter cursor test in CI

* refactor: dedupe replay reporter live progress checks

* fix: make Expo build cache path configurable
2026-07-01 16:21:56 +02:00
Michał Pierzchała 56b41a53ab feat: add cross-platform audio probe (#880)
* feat: add web audio probe

* fix: stabilize web audio probe

* test: cover audio probe review gaps

* fix: address audio probe review feedback

* feat: support macOS audio probe

* docs: document audio probe help

* feat: support simulator audio probe

* test: account for host audio platform support

* refactor: deepen audio probe lifecycle

* perf: trim audio probe package size

* refactor: address audio probe review comments

* refactor: remove audio probe leftovers

* fix: encode audio probe eval options as data

* fix: document audio probe eval sanitization

* fix: sanitize audio probe eval options

* fix: use codeql-recognized eval option sanitizer

* fix: allowlist audio probe eval options

* fix: avoid json-stringified audio eval action

* refactor: trim audio probe input surface

* fix: align audio probe with apple helper paths

* test: update audio capability parity oracle

* refactor: isolate host audio probe backend

* fixup! refactor: isolate host audio probe backend

* fixup! refactor: isolate host audio probe backend

* fixup! test: update audio capability parity oracle
2026-07-01 13:27:45 +02:00
Michał Pierzchała 6ae0612ebc fix: clean up maestro test reporter output (#935)
* fix: clean up maestro test reporter output

* chore: enable expo build disk cache

* refactor: simplify replay progress detail formatting

* fix: surface replay runner recovery hints

* fix: prioritize ios runner recovery hint

* fix: avoid trailing punctuation in runner state hint

* fix: keep internal cleanup scripts out of runner hints

* chore: remove redundant maestro test app open flag

* fix: make test app maestro flow self-contained

* fix: simplify maestro test duration output

* fix: refine maestro test summary output

* fix: dim maestro live progress counters

* test: clear test app state before maestro flow

* test: update maestro reporter progress expectations

* chore: remove maestro app open flag handling

* fix: apply maestro reporter cleanup to default reporter
2026-06-29 21:02:30 +02:00
Michał Pierzchała 98c0b1d3bf test: migrate test app to expo dev client (#881)
* test: migrate test app to expo dev client

* docs: align test app device targeting

* docs: clarify dev client setup tradeoffs

* docs: remove stale sdk reference
2026-06-25 21:43:14 +02:00
Michał Pierzchała df490ee859 fix: recover Android snapshots from system-only helper output (#861)
* fix: recover Android snapshots from system-only helper output

* fix: tighten Android snapshot recovery follow-up

* fix: preserve Android foreground container pruning
2026-06-25 07:37:23 +02:00
Yoni Samlan 93a6998188 fix: rotate synthesized iOS taps into native screen space (#804)
* fix: rotate synthesized iOS taps into native screen space

* fix: rotate synthesized iOS transform gestures

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
2026-06-14 11:24:31 +02:00
Michał Pierzchała 4f95ca8881 fix(daemon): timing-safe token comparison, daemon.json hardening, shell-quote CVE (#731)
* fix(daemon): timing-safe token comparison and daemon.json permission hardening

Use crypto.timingSafeEqual (via SHA-256 digests, length-independent) for the
three daemon token checks, and chmod daemon.json to 0600 after writes since
writeFileSync only applies mode on creation.

https://claude.ai/code/session_01LXZXzxi55sZ11DSyqWyBA2

* fix(deps): clear CVE-2026-9277 by overriding shell-quote to >=1.8.4 in test-app

Override added to examples/test-app/pnpm-workspace.yaml (package.json-level
overrides are silently ignored for this nested app, see the comment there).
Lockfile change is limited to shell-quote 1.8.3 -> 1.8.4; pnpm audit is clean.

https://claude.ai/code/session_01LXZXzxi55sZ11DSyqWyBA2

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-10 11:00:19 +02:00
Michał Pierzchała 9e6537200a fix: resolve test-app dependabot alerts (#649)
* fix: resolve test-app dependabot alerts

The postcss/uuid overrides added in #464 stopped applying once test-app
ended up nested under the repo-root pnpm-workspace.yaml: pnpm only honors
overrides from a workspace root, so test-app's package.json `pnpm.overrides`
were silently ignored and the lockfile drifted back to vulnerable versions.

Move the overrides into a dedicated examples/test-app/pnpm-workspace.yaml so
test-app is its own pnpm root and the overrides are honored, and add scoped
overrides for the two remaining alerts:

- postcss   8.4.49 -> 8.5.12  (XSS in CSS stringify)
- uuid      7.0.3  -> 14.0.0  (missing buffer bounds check)
- ws@8      8.20.0 -> 8.21.0  (uninitialized memory disclosure)
- brace-expansion@5  5.0.5 -> 5.0.6  (ReDoS / max bypass)

ws and brace-expansion overrides are scoped to the vulnerable majors so the
non-vulnerable ws@7 / brace-expansion@1 copies in the tree are left untouched.

* chore: drop dead lodash-es override, document test-app workspace

- Remove the no-op `lodash-es` override from the root package.json (leftover
  from #368). lodash-es is no longer in the dependency tree, so the override
  resolved to nothing; regenerating the root lockfile is a no-op.
- Add a comment to examples/test-app/pnpm-workspace.yaml explaining why the
  file exists, so it isn't "tidied away" and the override drift reintroduced.
2026-06-01 19:32:17 +02:00
Michał Pierzchała 2068f604bb fix: improve ios selector reads and maestro reliability (#636) 2026-06-01 14:04:27 +02:00
Michał Pierzchała c72cf0e1d2 fix: clarify Android gesture transform behavior (#584)
* fix: clarify Android gesture transform behavior

* fix: stabilize Android transform injection
2026-05-26 09:47:07 +02:00
Michał Pierzchała 47b981c8ad feat: add gesture command coverage (#576)
* feat: add gesture command coverage

* fix: align iOS fling provider fixture

* feat: group gesture commands

* fix: clarify android gesture support

* feat: add android multitouch gestures

* fix: address gesture review feedback

* refactor: simplify gesture plumbing

* fix: keep gesture subcommands internal

* fix: update iOS provider pan transcript
2026-05-22 18:01:58 +02:00
Michał Pierzchała 896adcc625 feat: add maestro replay compatibility (#561) 2026-05-19 20:49:34 +02:00
Michał Pierzchała 5df37ec9c9 fix: improve android fill verification diagnostics (#495) 2026-05-11 20:06:59 +02:00
Michał Pierzchała 999b475126 fix: resolve security alerts (#464) 2026-04-28 16:56:18 -04:00
Michał Pierzchała 7c5b7670c8 feat: add skillgym tests (#453) 2026-04-26 20:49:59 -04:00