Files
software-mansion__argent/packages/tool-server/test/flows/flow-ios-tree-no-windows.test.ts
Jakub Piasecki 2e182f7971 fix(flows): pin iOS tree reads to the launched app instead of auto-resolving every connection (#772)
## The problem

Flow runs on CI iOS fail with `ViewInspector RPC timed out:
Application.getState` at seemingly random steps - mid-flow after typing,
on the first read after a fresh launch, in a probe flow 25 s after the
original failure - while the app under test is provably healthy:
spindumps taken at failure time show its main thread idle in the
runloop, display links firing, and the injected devtools queue empty.
The same flows pass locally.

The chain: argent's iOS injection is simulator-wide (`launchctl setenv
DYLD_INSERT_LIBRARIES` inside the simulator's launchd), so every process
the simulator spawns afterwards loads the devtools dylib, not just the
app under test. The dylib's constructor connects unconditionally (no
UIKit or `com.apple.*` guard) and the tool-server registers whatever
bundle id shakes hands, so Spotlight workers (`mdworker_shared`),
background-launched system apps doing first-boot work (`MobileCal`), and
bare daemons (nil bundle id, registered as `"unknown"`) all become
connections indistinguishable from the app under test.

Flow tree reads auto-resolved the target on every read:
`queryFullHierarchyTree` passed `undefined` to `resolveNativeTargetApp`,
which `Promise.all`s `Application.getState` over every connection. The
dylib serves `getState` with a `dispatch_sync` onto the main thread, and
a background-launched or suspended process never services its main
queue, so its `getState` parks until the 5 s RPC timeout and the
`Promise.all` rejects. One poisoned sibling connection fails every tree
read in the run; the app under test is never even asked.

That one mechanism explains everything the failures did. CI spindumps
caught the poisoner red-handed (`MobileCal` parked in
`HandleViewInspectorRequest -> CurrentApplicationState`) next to a
healthy app. "Permanent wedge" vs "recovery" was just poisoner lifetime:
a Spotlight worker exits and its connection drops with it, so reads
recover; Calendar lingers and the whole job stays poisoned, fresh app
launches and probe flows included. And it never reproduced locally
because a warm simulator did its first-boot indexing and migrations long
ago, while CI boots a pristine device and does exactly that work during
the test window - which also made the failures correlate with typing and
tapping purely by timing.

## The fix

Pin the target instead of resolving it per read. `runLaunch` records the
launched app on the run state once the tree-source gate passes, as
`ActionEnv.treeTarget` - a `FlowTreeTarget` carrying the `bundleId`,
whether the runner still vouches for it (`pinned`), and whether its
probe has ever answered in this run (`probeAnswered`). `fetchFlowTree`
threads that target down to `queryFullHierarchyTree`, the only platform
source that consumes it.

A pinned read talks to the pinned app and nobody else: it never calls
`resolveNativeTargetApp` at all, and runs four gates of its own instead
(`flow-ios-tree.ts`). A `com.apple.*` id is refused first, terminally
and before anything is spent - a system app is never the app under test.
Then `isConnected`, a map lookup, so a dead pin reports the dropped
connection instead of driving env repair from 300 ms read polls, which
could otherwise latch the device's process-wide give-up. Then a probe of
the pinned app's own `getAppState`, and only that app's: a timeout
before this pin has ever answered is ridden out as the cold start it
usually is, one after an earlier answer is refused as an app that
stopped servicing its main queue. Finally a foreground check on the
answered state, as lenient over one app as auto-resolve is, so a flow
that left the app fails with the crisp diagnosis instead of describing a
screen that is not on screen. A poisoned sibling connection is never
touched.

The pin is only trusted while the runner can vouch for what is on
screen, and it comes off in two tiers. The five
`FOREGROUND_CHANGING_TOOLS` (`launch-app`, `restart-app`,
`reinstall-app`, `open-url`, `button`) drop the target outright, because
the launched app may no longer be on screen at all; `launch-app` and
`restart-app` then re-set it from their own `bundleId` arg, unpinned,
since they name the app they switched to - which is how a recorded flow
switches apps, the recorder keeping them as raw `tool:` steps. Every
other raw `tool:` step demotes the pin to an unpinned hint, its effect
on the foreground being opaque to the runner; a launch attempt drops the
target outright until it succeeds. Either tier also retires a proven
tree-outage memo: a verdict proven against the pinned branch's four
gates says nothing about the path the next read runs.

An unpinned read is the pre-pin behavior with one arbiter added.
`resolveNativeTargetApp` fans out over every connection, and only when
that fan-out times out - the CI condition this change exists for - does
the hint take the read, and only after passing the same terminal
`com.apple.*` policy gate, the connections check, and a
foreground-checked probe of its own. A resolution that answers always
wins, including its deliberate "single app but backgrounded" refusal, so
the arbiter never overrides a guard that fired: it only rides out a
probe a stalled sibling made unanswerable.

Nested `run:` fragments share the run state, so they inherit the pin,
its demotion and its clearing. E2e flows begin with a `launch` by
definition, so every selector read in a CI run happens pinned. Only iOS
consumes the target; the other platforms' tree sources are per-device
and never auto-resolve.

The review rounds also surfaced that the `isInjectableBundleId` "library
validation, the dylib can never load" rationale was disproven by this
PR's own spindump evidence, so that text - and every other home of the
same claim, `unreadableHierarchyReason` and `treeSourceGate` included -
is rewritten as the policy it actually is (system apps are never the app
under test; their observed connections are background processes whose
reads hang or lie) with the gates' behavior unchanged.

## What this doesn't cover

Auto-resolve's own fan-out is still all-or-nothing, so every read that
reaches it without a hint the gates accept stays poisonable: top-level
fragment runs, `describe` without an explicit bundle id, and the stretch
after a raw `tool:` step whose hint is refused or cannot answer.
Recording (`captureTapSelector`) now passes the session's last `launch`
id as an unpinned hint, so it arbitrates the same timed-out fan-out, but
it never pins - the recorder cannot vouch for the foreground app the way
a `launch:` step can. Closing the rest means hardening auto-resolve
itself (exclude connections that don't answer instead of failing the
fan-out) and refusing the connections at the source (dylib constructor
bail for non-UIKit and `com.apple.*` processes, plus a server-side
handshake filter to cover already-shipped dylibs). Both are follow-ups.

## Tests

Three files, and every behavior below is mutation-proven: disabling the
seam, dropping the target from any single call site, removing an
invalidation, or removing any one of the four gates each fails at least
one dedicated test. The reviewers' repros were turned into tests rather
than argued away.

`flow-ios-tree-pinned-target.test.ts` works at the tree source. It
reproduces the failure shape - a healthy app plus a poisoned
`com.apple.mobilecal` sibling whose `getState` only times out - and
proves a pinned read probes only the pinned app while an unpinned read
still fails. It then covers the states auto-resolve could never reach:
the terminal `com.apple.*` refusal, including its position ahead of the
connections gate so a disconnected system app is not sent into a restart
loop; a dead pin, where the dropped connection is reported and no env
repair runs; both sides of the `probeAnswered` split; and the foreground
guard over the states a real device produces (`inactive` under a
permission dialog, `inactive` with no scenes, `background` still holding
a foreground-inactive scene, and `background` with none). A second block
covers the unpinned arbiter: the hint read on a timeout, refused when
its own probe answers from the background, and rethrowing the fan-out
timeout when the hint is disconnected, cannot answer, or when the
resolution failed for any other reason.

`flow-launch-pins-tree-target.test.ts` proves the runner wiring through
the real `fetchFlowTree` seam: every read after a `launch` carries the
pin across all five read paths (assert, tap, type, rotate, idle), retry
polls included; fragments inherit it; a foreground-neutral `tool:` step
demotes it and a foreground-changing one that names no app drops it
outright; a later launch re-pins on a fresh target with the ride-out
re-armed; and a launch-free run keeps the fallback.

`flow-tool-step-tree-target-e2e.test.ts` drives `createRunFlowTool` with
only `resolveService` stubbed, so the real `fetchFlowTree` and
`queryFullHierarchyTree` run. It pins what the two tiers cost against a
wedged sibling: the launched app still arbitrating across a
foreground-neutral `tool:` step, the outage memo being retired when that
step demotes the pin, the `com.apple.*` and foreground refusals staying
in force through the demote, and a `hidden:` assert failing after the
demote instead of going green against a backgrounded app.
2026-08-25 13:26:46 +02:00

120 lines
4.4 KiB
TypeScript

import { describe, it, expect } from "vitest";
import type { DeviceInfo, Registry } from "@argent/registry";
import type { NativeAppState, NativeDevtoolsApi } from "../../src/blueprints/native-devtools";
import { queryFullHierarchyTree } from "../../src/tools/flows/flow-ios-tree";
// A getFullHierarchy read that returns no windows is the signal of an untrusted
// read — a backgrounded app, a window that has not attached yet, or a system
// process the frontmost resolver still picked on an unpinned read.
// The flow tree source must THROW there rather than hand back an empty tree:
// an empty tree is the one thing a `hidden`/absent check accepts as satisfied,
// so degrading to it turns an unreadable target into a false green pass.
const IOS_DEVICE = {
id: "00000000-0000-0000-0000-0000000000ab",
platform: "ios",
} as unknown as DeviceInfo;
const APP = "com.example.app";
function appState(bundleId: string): NativeAppState {
return {
bundleId,
applicationState: "active",
foregroundActiveSceneCount: 1,
foregroundInactiveSceneCount: 0,
backgroundSceneCount: 0,
unattachedSceneCount: 0,
isFrontmostCandidate: true,
};
}
/**
* Minimal NativeDevtoolsApi: one connected, foreground app; `queryViewHierarchy`
* returns whatever `hierarchy` yields (called with the resolved bundleId).
*/
function nativeApi(hierarchy: () => unknown, bundleId = APP): NativeDevtoolsApi {
return {
listConnectedBundleIds: () => [bundleId],
getAppState: async (id: string) => appState(id),
queryViewHierarchy: async () => hierarchy(),
} as unknown as NativeDevtoolsApi;
}
function registryFor(api: NativeDevtoolsApi): Registry {
return {
resolveService: async () => api,
} as unknown as Registry;
}
function windowSpanning() {
return {
className: "UIWindow",
frame: { x: 0, y: 0, width: 400, height: 800 },
windowFrame: { x: 0, y: 0, width: 400, height: 800 },
children: [
{
className: "RCTView",
identifier: "root",
windowFrame: { x: 0, y: 0, width: 400, height: 800 },
children: [],
},
],
};
}
/** A window whose subtree is pure scaffolding — no identifier, label, semantic
* role, or first responder anywhere — so the flatten emits zero leaves. */
function windowBare() {
return {
className: "UIWindow",
frame: { x: 0, y: 0, width: 400, height: 800 },
windowFrame: { x: 0, y: 0, width: 400, height: 800 },
children: [
{
className: "UIView",
windowFrame: { x: 0, y: 0, width: 400, height: 800 },
children: [],
},
],
};
}
describe("queryFullHierarchyTree — untrusted (no windows) reads", () => {
it("throws when the target returns an empty windows array", async () => {
const registry = registryFor(nativeApi(() => ({ windows: [] })));
await expect(queryFullHierarchyTree(registry, IOS_DEVICE)).rejects.toThrow(
/returned no windows for com\.example\.app/i
);
});
it("throws when the payload carries no windows field at all", async () => {
const registry = registryFor(nativeApi(() => ({})));
await expect(queryFullHierarchyTree(registry, IOS_DEVICE)).rejects.toThrow(/no windows/i);
});
it("returns a tree when windows are present", async () => {
const registry = registryFor(nativeApi(() => ({ windows: [windowSpanning()] })));
const { tree, source } = await queryFullHierarchyTree(registry, IOS_DEVICE);
expect(source).toBe("native-devtools");
expect(tree.children.length).toBeGreaterThan(0);
});
// The guard keys on the raw `windows` array, not the flattened child count:
// windows-present-but-zero-leaves is a genuinely sparse, trusted screen, so
// it must resolve (letting a `hidden` assert pass) rather than throw.
it("resolves an empty trusted tree when windows are present but flatten to no leaves", async () => {
const registry = registryFor(nativeApi(() => ({ windows: [windowBare()] })));
const { tree, source } = await queryFullHierarchyTree(registry, IOS_DEVICE);
expect(source).toBe("native-devtools");
expect(tree.children.length).toBe(0);
});
it("still throws on an explicit getFullHierarchy error (unchanged)", async () => {
const registry = registryFor(nativeApi(() => ({ error: "hierarchy walk failed" })));
await expect(queryFullHierarchyTree(registry, IOS_DEVICE)).rejects.toThrow(
/getFullHierarchy failed.*hierarchy walk failed/i
);
});
});