mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
2e182f7971
## The problem Flow runs on CI iOS fail with `ViewInspector RPC timed out: Application.getState` at seemingly random steps - mid-flow after typing, on the first read after a fresh launch, in a probe flow 25 s after the original failure - while the app under test is provably healthy: spindumps taken at failure time show its main thread idle in the runloop, display links firing, and the injected devtools queue empty. The same flows pass locally. The chain: argent's iOS injection is simulator-wide (`launchctl setenv DYLD_INSERT_LIBRARIES` inside the simulator's launchd), so every process the simulator spawns afterwards loads the devtools dylib, not just the app under test. The dylib's constructor connects unconditionally (no UIKit or `com.apple.*` guard) and the tool-server registers whatever bundle id shakes hands, so Spotlight workers (`mdworker_shared`), background-launched system apps doing first-boot work (`MobileCal`), and bare daemons (nil bundle id, registered as `"unknown"`) all become connections indistinguishable from the app under test. Flow tree reads auto-resolved the target on every read: `queryFullHierarchyTree` passed `undefined` to `resolveNativeTargetApp`, which `Promise.all`s `Application.getState` over every connection. The dylib serves `getState` with a `dispatch_sync` onto the main thread, and a background-launched or suspended process never services its main queue, so its `getState` parks until the 5 s RPC timeout and the `Promise.all` rejects. One poisoned sibling connection fails every tree read in the run; the app under test is never even asked. That one mechanism explains everything the failures did. CI spindumps caught the poisoner red-handed (`MobileCal` parked in `HandleViewInspectorRequest -> CurrentApplicationState`) next to a healthy app. "Permanent wedge" vs "recovery" was just poisoner lifetime: a Spotlight worker exits and its connection drops with it, so reads recover; Calendar lingers and the whole job stays poisoned, fresh app launches and probe flows included. And it never reproduced locally because a warm simulator did its first-boot indexing and migrations long ago, while CI boots a pristine device and does exactly that work during the test window - which also made the failures correlate with typing and tapping purely by timing. ## The fix Pin the target instead of resolving it per read. `runLaunch` records the launched app on the run state once the tree-source gate passes, as `ActionEnv.treeTarget` - a `FlowTreeTarget` carrying the `bundleId`, whether the runner still vouches for it (`pinned`), and whether its probe has ever answered in this run (`probeAnswered`). `fetchFlowTree` threads that target down to `queryFullHierarchyTree`, the only platform source that consumes it. A pinned read talks to the pinned app and nobody else: it never calls `resolveNativeTargetApp` at all, and runs four gates of its own instead (`flow-ios-tree.ts`). A `com.apple.*` id is refused first, terminally and before anything is spent - a system app is never the app under test. Then `isConnected`, a map lookup, so a dead pin reports the dropped connection instead of driving env repair from 300 ms read polls, which could otherwise latch the device's process-wide give-up. Then a probe of the pinned app's own `getAppState`, and only that app's: a timeout before this pin has ever answered is ridden out as the cold start it usually is, one after an earlier answer is refused as an app that stopped servicing its main queue. Finally a foreground check on the answered state, as lenient over one app as auto-resolve is, so a flow that left the app fails with the crisp diagnosis instead of describing a screen that is not on screen. A poisoned sibling connection is never touched. The pin is only trusted while the runner can vouch for what is on screen, and it comes off in two tiers. The five `FOREGROUND_CHANGING_TOOLS` (`launch-app`, `restart-app`, `reinstall-app`, `open-url`, `button`) drop the target outright, because the launched app may no longer be on screen at all; `launch-app` and `restart-app` then re-set it from their own `bundleId` arg, unpinned, since they name the app they switched to - which is how a recorded flow switches apps, the recorder keeping them as raw `tool:` steps. Every other raw `tool:` step demotes the pin to an unpinned hint, its effect on the foreground being opaque to the runner; a launch attempt drops the target outright until it succeeds. Either tier also retires a proven tree-outage memo: a verdict proven against the pinned branch's four gates says nothing about the path the next read runs. An unpinned read is the pre-pin behavior with one arbiter added. `resolveNativeTargetApp` fans out over every connection, and only when that fan-out times out - the CI condition this change exists for - does the hint take the read, and only after passing the same terminal `com.apple.*` policy gate, the connections check, and a foreground-checked probe of its own. A resolution that answers always wins, including its deliberate "single app but backgrounded" refusal, so the arbiter never overrides a guard that fired: it only rides out a probe a stalled sibling made unanswerable. Nested `run:` fragments share the run state, so they inherit the pin, its demotion and its clearing. E2e flows begin with a `launch` by definition, so every selector read in a CI run happens pinned. Only iOS consumes the target; the other platforms' tree sources are per-device and never auto-resolve. The review rounds also surfaced that the `isInjectableBundleId` "library validation, the dylib can never load" rationale was disproven by this PR's own spindump evidence, so that text - and every other home of the same claim, `unreadableHierarchyReason` and `treeSourceGate` included - is rewritten as the policy it actually is (system apps are never the app under test; their observed connections are background processes whose reads hang or lie) with the gates' behavior unchanged. ## What this doesn't cover Auto-resolve's own fan-out is still all-or-nothing, so every read that reaches it without a hint the gates accept stays poisonable: top-level fragment runs, `describe` without an explicit bundle id, and the stretch after a raw `tool:` step whose hint is refused or cannot answer. Recording (`captureTapSelector`) now passes the session's last `launch` id as an unpinned hint, so it arbitrates the same timed-out fan-out, but it never pins - the recorder cannot vouch for the foreground app the way a `launch:` step can. Closing the rest means hardening auto-resolve itself (exclude connections that don't answer instead of failing the fan-out) and refusing the connections at the source (dylib constructor bail for non-UIKit and `com.apple.*` processes, plus a server-side handshake filter to cover already-shipped dylibs). Both are follow-ups. ## Tests Three files, and every behavior below is mutation-proven: disabling the seam, dropping the target from any single call site, removing an invalidation, or removing any one of the four gates each fails at least one dedicated test. The reviewers' repros were turned into tests rather than argued away. `flow-ios-tree-pinned-target.test.ts` works at the tree source. It reproduces the failure shape - a healthy app plus a poisoned `com.apple.mobilecal` sibling whose `getState` only times out - and proves a pinned read probes only the pinned app while an unpinned read still fails. It then covers the states auto-resolve could never reach: the terminal `com.apple.*` refusal, including its position ahead of the connections gate so a disconnected system app is not sent into a restart loop; a dead pin, where the dropped connection is reported and no env repair runs; both sides of the `probeAnswered` split; and the foreground guard over the states a real device produces (`inactive` under a permission dialog, `inactive` with no scenes, `background` still holding a foreground-inactive scene, and `background` with none). A second block covers the unpinned arbiter: the hint read on a timeout, refused when its own probe answers from the background, and rethrowing the fan-out timeout when the hint is disconnected, cannot answer, or when the resolution failed for any other reason. `flow-launch-pins-tree-target.test.ts` proves the runner wiring through the real `fetchFlowTree` seam: every read after a `launch` carries the pin across all five read paths (assert, tap, type, rotate, idle), retry polls included; fragments inherit it; a foreground-neutral `tool:` step demotes it and a foreground-changing one that names no app drops it outright; a later launch re-pins on a fresh target with the ride-out re-armed; and a launch-free run keeps the fallback. `flow-tool-step-tree-target-e2e.test.ts` drives `createRunFlowTool` with only `resolveService` stubbed, so the real `fetchFlowTree` and `queryFullHierarchyTree` run. It pins what the two tiers cost against a wedged sibling: the launched app still arbitrating across a foreground-neutral `tool:` step, the outage memo being retired when that step demotes the pin, the `com.apple.*` and foreground refusals staying in force through the demote, and a `hidden:` assert failing after the demote instead of going green against a backgrounded app.
216 lines
9.2 KiB
TypeScript
216 lines
9.2 KiB
TypeScript
import { describe, it, expect, vi } from "vitest";
|
|
import type { NativeDevtoolsApi } from "../src/blueprints/native-devtools";
|
|
|
|
// Two consumers of the measured state: the full-hierarchy path flows resolve
|
|
// selectors against, and the ios-remote host's `inspectRunningApp`. Both decide
|
|
// what an agent is told to do next, and on ios-remote the running/indeterminate
|
|
// split is the ONLY distinction available — it drives `requiresRestart` and
|
|
// `describe`'s `should_restart`.
|
|
|
|
const remote = vi.hoisted(() => ({ stdout: "", calls: 0, fail: false }));
|
|
|
|
vi.mock("@argent/native-devtools-ios", () => ({
|
|
bootstrapDylibPath: () => "/fake/dylibs/libArgentInjectionBootstrap.dylib",
|
|
bootstrapDylibPathTcp: () => "/fake/dylibs/tcp/libArgentInjectionBootstrap.dylib",
|
|
bootstrapDylibPathTvos: () => "/fake/dylibs/tvos/libArgentInjectionBootstrap.dylib",
|
|
tcpInjectionDylibs: () => [],
|
|
axServiceBinaryPath: () => "/fake/ax-service",
|
|
axServiceBinaryPathTcp: () => "/fake/ax-service-tcp",
|
|
}));
|
|
|
|
vi.mock("../src/utils/sim-remote", async (importOriginal) => ({
|
|
...(await importOriginal<typeof import("../src/utils/sim-remote")>()),
|
|
simctlSpawn: vi.fn(async () => {
|
|
remote.calls += 1;
|
|
if (remote.fail) throw new Error("tunnel down");
|
|
return { stdout: remote.stdout, stderr: "" };
|
|
}),
|
|
}));
|
|
|
|
import { remoteIosHost } from "../src/utils/ios-host";
|
|
import { queryFullHierarchyTree } from "../src/tools/flows/flow-ios-tree";
|
|
import type { DeviceInfo, Registry } from "@argent/registry";
|
|
|
|
const UDID = "AAAAAAAA-1111-2222-3333-444444444444";
|
|
const BUNDLE = "com.example.app";
|
|
const DEVICE: DeviceInfo = { id: UDID, platform: "ios", kind: "simulator" };
|
|
// The launched app as the runner hands it over once a raw `tool:` step has
|
|
// spent the pin.
|
|
const UNPINNED = { bundleId: BUNDLE, pinned: false, probeAnswered: false };
|
|
|
|
describe("remoteIosHost.inspectRunningApp", () => {
|
|
it("reports running-ness from the orchestrator and leaves the process unknown", async () => {
|
|
remote.stdout = `4242\t0\tUIKitApplication:${BUNDLE}[dffa][rb-legacy]\n`;
|
|
remote.calls = 0;
|
|
|
|
const inspection = await remoteIosHost.inspectRunningApp(UDID, BUNDLE);
|
|
|
|
// `running` must come off the real row: assuming true makes every stopped
|
|
// remote app `indeterminate`, answered with a restart of an app that is not
|
|
// there, and assuming false makes every running one `not_running`.
|
|
expect(inspection.running).toBe(true);
|
|
// App processes live on the orchestrator, so the local process table has
|
|
// nothing to say — a fabricated process here would be judged against this
|
|
// listener and reported as a definite verdict.
|
|
expect(inspection.process).toBeNull();
|
|
expect(remote.calls).toBe(1);
|
|
});
|
|
|
|
it("reports not running when no row backs the bundle", async () => {
|
|
remote.stdout = `4242\t0\tUIKitApplication:com.other.app[dffa][rb-legacy]\n`;
|
|
|
|
await expect(remoteIosHost.inspectRunningApp(UDID, BUNDLE)).resolves.toEqual({
|
|
running: false,
|
|
process: null,
|
|
});
|
|
});
|
|
});
|
|
|
|
describe("queryFullHierarchyTree surfaces the measured diagnosis", () => {
|
|
// Both accessors come off ONE connected set, as the real factory derives them
|
|
// from its `connections` map. Stubbing them independently lets a test assert a
|
|
// pairing the service cannot produce — a listed-but-unconnected app — so a
|
|
// diagnosis reached only that way is reached only by tests. Every unconnected
|
|
// state means an EMPTY set, hence an app auto-targeting cannot name; the id the
|
|
// flow launched is what makes it nameable.
|
|
function registryWith(connected: string[], overrides: Partial<NativeDevtoolsApi> = {}): Registry {
|
|
const api = {
|
|
listConnectedBundleIds: () => connected,
|
|
isConnected: (bundleId: string) => connected.includes(bundleId),
|
|
getAppState: async (bundleId: string) => ({
|
|
bundleId,
|
|
applicationState: "active",
|
|
foregroundActiveSceneCount: 1,
|
|
foregroundInactiveSceneCount: 0,
|
|
backgroundSceneCount: 0,
|
|
unattachedSceneCount: 0,
|
|
isFrontmostCandidate: true,
|
|
}),
|
|
appConnectionState: async (bundleId: string) =>
|
|
connected.includes(bundleId) ? "connected" : "unregistered",
|
|
...overrides,
|
|
} as unknown as NativeDevtoolsApi;
|
|
return { resolveService: async () => api } as unknown as Registry;
|
|
}
|
|
|
|
it("raises the state's own remedy rather than a blanket relaunch", async () => {
|
|
// `unregistered` is the case that matters: telling a flow author to relaunch
|
|
// here sends them round a loop the app cannot exit.
|
|
const registry = registryWith([]);
|
|
|
|
await expect(queryFullHierarchyTree(registry, DEVICE, UNPINNED)).rejects.toThrow(
|
|
/argent server stop && argent server start --detach/
|
|
);
|
|
await expect(queryFullHierarchyTree(registry, DEVICE, UNPINNED)).rejects.not.toThrow(
|
|
/relaunch it/
|
|
);
|
|
});
|
|
|
|
it("keeps the auto-target error when the run never launched an app", async () => {
|
|
// A fragment brings the device to its entry state out of band, so there is
|
|
// no launched id to measure and the auto-target error — which names its own
|
|
// next steps — is the best answer available.
|
|
const registry = registryWith([]);
|
|
|
|
await expect(queryFullHierarchyTree(registry, DEVICE)).rejects.toThrow(
|
|
/No native-devtools-connected apps are available for auto-targeting/
|
|
);
|
|
});
|
|
|
|
it("does not measure when auto-targeting resolved an app", async () => {
|
|
// The measurement is for the app the flow launched, not a second-guess of a
|
|
// resolution that succeeded. The stub serves a real window because an empty
|
|
// `windows` is its own error and would end the call before the assertion.
|
|
const appConnectionState = vi.fn(async () => "connected" as const);
|
|
const registry = registryWith([BUNDLE], {
|
|
appConnectionState,
|
|
queryViewHierarchy: async () => ({
|
|
windows: [
|
|
{
|
|
className: "UIWindow",
|
|
frame: { x: 0, y: 0, width: 400, height: 800 },
|
|
windowFrame: { x: 0, y: 0, width: 400, height: 800 },
|
|
children: [],
|
|
},
|
|
],
|
|
}),
|
|
} as unknown as Partial<NativeDevtoolsApi>);
|
|
|
|
await queryFullHierarchyTree(registry, DEVICE, UNPINNED);
|
|
|
|
expect(appConnectionState).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("degrades a rejected measurement instead of leaking the subprocess error", async () => {
|
|
// The measurement re-applies the launchd env before it can answer, so a sim
|
|
// that goes away mid-run rejects here. The other consumers degrade to
|
|
// `indeterminate`; a raw `Command failed: xcrun simctl spawn …` carries none
|
|
// of the diagnosis's guidance.
|
|
const registry = registryWith([], {
|
|
appConnectionState: async () => {
|
|
throw new Error("Command failed: xcrun simctl spawn UDID launchctl setenv");
|
|
},
|
|
});
|
|
|
|
await expect(queryFullHierarchyTree(registry, DEVICE, UNPINNED)).rejects.toThrow(
|
|
/could not be inspected/
|
|
);
|
|
await expect(queryFullHierarchyTree(registry, DEVICE, UNPINNED)).rejects.not.toThrow(
|
|
/Command failed/
|
|
);
|
|
});
|
|
|
|
// Reachable, though it reads as dead: `appConnectionState` re-reads the live
|
|
// connections map AFTER its env re-apply and process probe — several simctl
|
|
// round-trips past the empty list that routed the call here — precisely so a
|
|
// dial landing in that window is not reported as an app the service never
|
|
// registered. Folding the case away leaves it throwing a sentence that names
|
|
// no app and no remedy.
|
|
it("says the connection arrived mid-read when the measurement finds it connected", async () => {
|
|
const registry = registryWith([], { appConnectionState: async () => "connected" });
|
|
|
|
await expect(queryFullHierarchyTree(registry, DEVICE, UNPINNED)).rejects.toThrow(
|
|
/connection arrived mid-read/
|
|
);
|
|
// It must still name the app and an action; the measured states' remedies
|
|
// do not apply, because nothing is wrong with the app.
|
|
await expect(queryFullHierarchyTree(registry, DEVICE, UNPINNED)).rejects.toThrow(
|
|
new RegExp(`${BUNDLE}[\\s\\S]*Retry`)
|
|
);
|
|
await expect(queryFullHierarchyTree(registry, DEVICE, UNPINNED)).rejects.not.toThrow(
|
|
/argent server stop|restart-app/
|
|
);
|
|
});
|
|
|
|
it("gives a system app the terminal reason instead of a measured remedy", async () => {
|
|
// The launch gate lets a refused app through so a coordinate-driven flow
|
|
// still runs; selector resolution is where that refusal bites, so this is
|
|
// where it is said - and said terminally, since every measured state's
|
|
// remedy is a retry of something.
|
|
const registry = registryWith([]);
|
|
|
|
await expect(
|
|
queryFullHierarchyTree(registry, DEVICE, {
|
|
bundleId: "com.apple.Preferences",
|
|
pinned: false,
|
|
probeAnswered: false,
|
|
})
|
|
).rejects.toThrow(/Apple system app/);
|
|
await expect(
|
|
queryFullHierarchyTree(registry, DEVICE, {
|
|
bundleId: "com.apple.Preferences",
|
|
pinned: false,
|
|
probeAnswered: false,
|
|
})
|
|
).rejects.not.toThrow(/argent server stop|restart-app/);
|
|
// The remedy has to name a step form that exists AND reads no tree.
|
|
await expect(
|
|
queryFullHierarchyTree(registry, DEVICE, {
|
|
bundleId: "com.apple.Preferences",
|
|
pinned: false,
|
|
probeAnswered: false,
|
|
})
|
|
).rejects.toThrow(/tap: \{ x: [\d.]+, y: [\d.]+ \}/);
|
|
});
|
|
});
|