mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
4c71bc911d
Rebased onto `main` (was stacked on #720), with a single hunk dropped — see **Restack notes** at the bottom. Two commits: `a71c0f0` is exactly this PR's own delta from the stacked version, `54fe761` is a review pass over it on the post-rebase base. Squash on merge. ## Depth cap: 40 → 100 The flow selector tree capped raw `UIView` nesting at **40**. That does not clear the invisible wrapper layers React Native stacks on every screen — nested navigators (`RNSScreenStackView` → `RNSScreenView`, often two deep) plus a drawer/root wrapper routinely bury on-screen content **40–60 levels down** before the first tappable view. In a deeply nested production app, plainly visible interactive elements sat at depths **41–62**. So `id:` and `text:` selectors silently failed to resolve, and only coordinate taps worked — which is the origin of coordinate-heavy flows on those apps. The overflow is **silent**: nothing reports "truncated", selectors just stop matching. So the new cap carries generous headroom rather than hugging the deepest observed measurement. **What it costs.** The tree itself is internal to selector resolution — consumed by `selectorToFrame`/`evaluateCondition`, never returned — so no tool result carries it. Text derived from it does reach the agent, though: `assertReason`'s `text` arm quotes the matched node's hoisted `subtreeText` verbatim into a failing `assert`/`await` reason, and nothing truncates it downstream. Descendants a depth-40 read dropped now hoist, so that quoted string grows with the cap. That is the trade — under the old cap those selectors did not resolve at all. Otherwise the cap only grows the `getFullHierarchy` payload over the native-devtools socket, which is already field-limited: ~11KB at depth 40, ~15KB at 48, and nothing at all past the tree's real depth. ## Targeting errors that could not be acted on **`Provide bundleId explicitly`** — `resolveNativeTargetApp`'s own errors close with that advice. A flow selector step cannot follow it: the call hardcodes auto-targeting. Each failure now carries the remedy that actually exists: - **ambiguous connected set** — foreground the intended app with `launch-app` (which does not terminate, so the instrumentation it already has survives), and clear the others with `xcrun simctl terminate` (argent exposes no terminate tool, and `restart-app` would just bring that app back to the front). The command is resolved through `simctlTargetForUdid`, so it carries the device's own CoreSimulator `--set` — a simulator from a configured `ios.additionalDeviceSets` resolves — and passes the same entitlement gate an actual spawn does. An external provider (#735) that grants `native-devtools` but withholds `simctl` gets the clause dropped whole, rather than a command argent itself would refuse and its device-set path quoted back. - **a lone connected app that is not foreground** — same foreground remedy; it is already instrumented, so a relaunch would only discard state. - **the state probe failed while connections are live** — which app is missing decides the remedy. While the app the flow launched is still in the live connections map it is merely suspended: foreground it, do not relaunch, since a relaunch discards the state the flow built up and cannot help when another app is the stale one. When it is *absent* from that map it is the one thing a relaunch fixes, so the reason names it and sends it to `restart-app` — `launch-app` cannot instrument a process that is already gone. - **no connected app** — this is the one state a relaunch fixes, through `restart-app` or a flow `launch` step. **`launch-app, or a flow launch step`** — this was wrong in a way that wastes a cycle. `launch-app` does **not** terminate first, so against an app already running from Metro/Expo, Xcode, or its home-screen icon it only foregrounds that same uninstrumented process, and the next read fails identically. Only `restart-app` (terminate + relaunch) guarantees an instrumented launch whatever was already running. These reasons are repeated per step — the recorder embeds one in the warning for every captured tap, and a failing `await:` repeats it per poll — so the two that enumerate connected apps are capped at two entries plus a count of what was left out, and the whole set is held under a documented ceiling by a test that iterates every branch. That ceiling is 800 characters. It is set by the `unregistered` reason, which measures 738 plus the bundle id, so it holds for the ids seen in practice and not for every one: a 63-character id clears it. Every figure quoted at `MAX_LISTED_APPS` and `MAX_TARGETING_REASON_CHARS` was re-measured against the current code. `FailureError` data is preserved when these messages wrap an underlying error, so the failure code still reaches callers that switch on it. ## Recorder: role-only selectors A tapped node with no identifier and no visible text replays on role alone, which holds only while that element keeps winning `selectorToFrame`'s ranking. The raised depth cap makes this more common — an unlabeled icon the old cap truncated away, which left `nodeAtPoint` to pick its `testID` container, is now present and is the smaller frame under the tap. The recorder now warns instead of recording that silently. ## Launch gate: why the wait is per bundle `treeSourceGate` already waits for every bundle, `com.apple.*` included, and withholds only the *verdict* for a non-injectable one. This PR documents why that wait is load-bearing rather than a readiness formality: it is what ties the launched bundle to the app a later selector step auto-targets, since `resolveNativeTargetApp(api, undefined)` never compares the two. Skipping it per bundle could hand the next step a different app's hierarchy and still report the run green. The matching fallback advice lands in the no-connected-app reason: for a `com.apple.*` bundle that never connects, drive the app with raw point taps and `tool: await-ui-element` steps, which read the AX tree instead of selectors. ## Restack notes - `a71c0f0` keeps the stacked version's per-file insert/delete counts unchanged. `54fe761` is the review pass on top: the entitlement gate above, the launched-app-gone remedy, four comments this branch left stale or overstated, and the re-measured figures. - No docs update needed — nothing in `packages/docs` documents the depth cap, the targeting reasons or the recorder warning. - Conflict resolved in `flow-ios-tree.ts`: #944 (physical iOS support) landed new device-tree functions in the same region. Both blocks kept — the targeting helpers sit next to `queryFullHierarchyTree`, which they serve; the device block follows. - **One hunk dropped:** a 35-line test in `flow-hidden-blank-reads.test.ts` asserting that an `exists` miss does not quote hoisted `subtreeText`. It is written against `compatibilityMissNote` / "typographic variant", which #720 introduces and `main` does not have. Worth re-adding to that PR, or here once #720 lands. - Verified on this base: `tsc --build` and `typecheck:tests` clean; 254 tests across the 4 touched test files pass; full tool-server suite 433 files / 6190 passed / 1 skipped (one run showed a single unrelated load flake, green on a clean re-run); eslint, prettier and both knip passes clean. All 8 CI checks green on `54fe761`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_014dA4nMpWrzECNhV6if4YXZ <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - iOS hierarchy inspection now supports substantially deeper views, improving selector and condition handling for deeply nested content. - iOS targeting failures provide clearer, actionable recovery guidance, including relevant app and device-state details. - System-app flows can continue successfully when apps do not establish a connection, including coordinate-based actions. - **Bug Fixes** - Captured tap steps now warn when selectors rely only on an element’s role. - Related selector warnings are combined into a single concise warning. - iOS failure messages no longer include misleading recovery advice. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
3471 lines
139 KiB
TypeScript
3471 lines
139 KiB
TypeScript
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
|
|
import * as fs from "node:fs/promises";
|
|
import * as os from "node:os";
|
|
import * as path from "node:path";
|
|
import {
|
|
ArtifactStore,
|
|
FLOW_FILE_NAME_PATTERN,
|
|
FLOW_NAME_PATTERN,
|
|
getFailureSignal,
|
|
type Registry,
|
|
} from "@argent/registry";
|
|
import {
|
|
createRunFlowTool,
|
|
flowLaunchGateReason,
|
|
LAUNCH_TO_VERDICT_MS,
|
|
MAX_RUN_DEPTH,
|
|
NATIVE_READY_TIMEOUT_MS,
|
|
type FlowRunResult,
|
|
} from "../../src/tools/flows/flow-run";
|
|
import { serializeFlow, parseFlow } from "../../src/tools/flows/flow-utils";
|
|
import { bindDeviceArgs, stripDeviceKeys } from "../../src/tools/flows/flow-device";
|
|
import { runSnapshot } from "../../src/tools/flows/flow-visual";
|
|
|
|
// Stub the snapshot differ: the baseline-anchoring test asserts only WHERE the
|
|
// runner points it (root flowsDir + root flow name), not the diffing itself.
|
|
vi.mock("../../src/tools/flows/flow-visual", () => ({
|
|
DEFAULT_MAX_MISMATCH: 0.5,
|
|
runSnapshot: vi.fn(async () => ({ status: "pass", reason: "snapshot stubbed" })),
|
|
}));
|
|
|
|
// Mock the Electron launcher: no test here may legitimately reach it — the
|
|
// chromium-ordering test below pins exactly that — so a misplaced upload guard
|
|
// records a boot attempt instead of spawning a real process.
|
|
const bootElectronApp = vi.fn(async () => ({
|
|
platform: "chromium" as const,
|
|
id: "chromium-cdp-12345",
|
|
port: 12345,
|
|
pid: 4242,
|
|
appPath: "/abs/e2e-app",
|
|
booted: true as const,
|
|
}));
|
|
vi.mock("../../src/tools/devices/boot-electron", () => ({
|
|
bootElectronApp: (...args: unknown[]) =>
|
|
(bootElectronApp as (...a: unknown[]) => unknown)(...args),
|
|
killChromiumByPort: vi.fn(),
|
|
}));
|
|
|
|
// Five tests here drive the launch gate's real 1.5 s settle plus the whole
|
|
// connect wait in fake time, pumping a real event-loop turn between advances so
|
|
// the run's disk I/O can settle. Each pump is a real macrotask, so the cost is
|
|
// the pump count, not the fake duration — under full-suite load it outruns
|
|
// vitest's 5 s default. Widening the advance is NOT the fix: coarser steps
|
|
// starve that I/O and take longer in real time (measured: 1 s steps took the
|
|
// file from 46 s to 52 s). Budget the pumping, and prefer unit-testing the pure
|
|
// message mapping over driving the whole runner once per case.
|
|
vi.setConfig({ testTimeout: 60_000 });
|
|
|
|
/**
|
|
* Ceiling on those pumps. The loop has to cover the settle plus the entire
|
|
* connect wait in fake time — one 250 ms advance each — and still have turns
|
|
* spare for the real-async steps interleaved between them, which multiply as
|
|
* parallel workers contend for the disk.
|
|
*
|
|
* Derived from the connect budget rather than fixed, because running out is
|
|
* silent and mimics nothing else: the run never settles, the loop abandons it
|
|
* pending, and the test hangs on `await pending` until the timeout — so it reads
|
|
* as slowness, and no timeout value fixes it. A settled run leaves the loop
|
|
* immediately, so the headroom costs nothing.
|
|
*/
|
|
const PUMP_LIMIT = Math.ceil(NATIVE_READY_TIMEOUT_MS / 250) * 10;
|
|
|
|
const DEVICE = "00000000-0000-0000-0000-0000000000ab";
|
|
let tmpDir: string;
|
|
|
|
/**
|
|
* `props`, when given, makes `getTool` report that schema for EVERY tool id —
|
|
* so a fixture using it must be a single step and must pass an explicit device,
|
|
* otherwise unrelated dispatches (list-devices) would be handed a bogus schema.
|
|
*/
|
|
function mockRegistry(props?: Record<string, unknown>): Registry {
|
|
return {
|
|
invokeTool: vi.fn(async (id: string) => {
|
|
if (id === "list-devices") return { devices: [] };
|
|
return { ok: true };
|
|
}),
|
|
getTool: vi.fn(() => (props ? { inputSchema: { properties: props } } : undefined)),
|
|
// iOS launch steps gate on a native-devtools connection: report connected
|
|
// so the run proceeds. The selector directives that do run here read a
|
|
// com.apple.* target, so they are refused before the stubs below matter.
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => true,
|
|
listConnectedBundleIds: () => [],
|
|
// No windows, so a read that does reach the tree source fails there
|
|
// rather than degrading to an empty tree.
|
|
queryViewHierarchy: async () => ({ windows: [] }),
|
|
getAppState: async (bundleId: string) => ({
|
|
bundleId,
|
|
applicationState: "active",
|
|
foregroundActiveSceneCount: 1,
|
|
foregroundInactiveSceneCount: 0,
|
|
backgroundSceneCount: 0,
|
|
unattachedSceneCount: 0,
|
|
isFrontmostCandidate: true,
|
|
}),
|
|
})),
|
|
} as unknown as Registry;
|
|
}
|
|
|
|
async function writeFlow(name: string, yaml: Parameters<typeof serializeFlow>[0]): Promise<void> {
|
|
const dir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(dir, { recursive: true });
|
|
await fs.writeFile(path.join(dir, `${name}.yaml`), serializeFlow(yaml), "utf8");
|
|
}
|
|
|
|
/**
|
|
* Write a straight `run:` chain root → n1 → … → n{links} → tail, one `run:`
|
|
* step per flow. The final hop fires with the root plus every link on the run
|
|
* stack, i.e. a stack of exactly `links + 1` entries — which is what lets the
|
|
* depth-boundary tests below land on the guard's threshold precisely.
|
|
*/
|
|
async function writeRunChain(links: number, tail: string): Promise<void> {
|
|
await writeFlow("root", { executionPrerequisite: "", steps: [{ kind: "run", flow: "n1.yaml" }] });
|
|
for (let i = 1; i <= links; i++) {
|
|
await writeFlow(`n${i}`, {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: i === links ? tail : `n${i + 1}.yaml` }],
|
|
});
|
|
}
|
|
}
|
|
|
|
function asRun(r: FlowRunResult | { notice: string }): FlowRunResult {
|
|
if (!("steps" in r)) throw new Error(`expected a run result, got notice: ${r.notice}`);
|
|
return r;
|
|
}
|
|
|
|
beforeEach(async () => {
|
|
// realpath'd so the path math below matches the runner's canonical anchors:
|
|
// macOS's tmpdir lives behind the /var → /private/var symlink, which would
|
|
// otherwise skew every equality for reasons unrelated to what a test pins.
|
|
tmpDir = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "flow-compose-")));
|
|
});
|
|
afterEach(async () => {
|
|
await fs.rm(tmpDir, { recursive: true, force: true });
|
|
});
|
|
|
|
describe("flow composition (run:)", () => {
|
|
it("expands a referenced fragment's steps inline", async () => {
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "On login screen",
|
|
steps: [
|
|
{ kind: "echo", message: "logging in" },
|
|
{ kind: "tool", name: "tap", args: { x: 0.5 } },
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "login.yaml" },
|
|
{ kind: "echo", message: "done" },
|
|
],
|
|
});
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
|
|
// run marker, login's echo + tap, then main's echo.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"echo:pass",
|
|
"tool:pass",
|
|
"echo:pass",
|
|
]);
|
|
// The expanded steps are attributed to the fragment.
|
|
expect(result.steps[1].flow).toBe("login");
|
|
expect(result.steps[3].flow).toBe("main");
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("composes a bare run: name from the containing flow's own directory", async () => {
|
|
// The legacy layout: both flows saved flat in .argent/flows, main written
|
|
// when a run: target was a saved-flow NAME resolved in that directory. A
|
|
// bare target now anchors to the containing file's directory instead —
|
|
// which for this layout IS that directory, so the flow replays unchanged.
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "On login screen",
|
|
steps: [{ kind: "echo", message: "logging in" }],
|
|
});
|
|
await fs.writeFile(
|
|
path.join(tmpDir, ".argent", "flows", "main.yaml"),
|
|
"steps:\n - run: login\n - echo: done\n",
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"echo:pass",
|
|
"echo:pass",
|
|
]);
|
|
// Parse completed the extension, so the report names the file it opened
|
|
// while attribution keeps the same stem the spelled-out form produces.
|
|
expect(result.steps[0]).toMatchObject({ target: "login.yaml", flow: "login" });
|
|
expect(result.steps[1].flow).toBe("login");
|
|
});
|
|
|
|
it("resolves a bare run: name against a nested fragment's directory", async () => {
|
|
// The one place the two schemes disagree: a bare name inside a fragment
|
|
// that does NOT sit in the flows dir root. `helpers/steps.yaml` composing
|
|
// `login` gets helpers/login.yaml — its own sibling — not the flows-dir
|
|
// copy a name lookup would have found. Both exist here with different
|
|
// messages, so the assertion distinguishes them.
|
|
const helpers = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpers, { recursive: true });
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "FLOWS-DIR-COPY" }],
|
|
});
|
|
await fs.writeFile(
|
|
path.join(helpers, "login.yaml"),
|
|
"steps:\n - echo: HELPERS-SIBLING\n",
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(path.join(helpers, "steps.yaml"), "steps:\n - run: login\n", "utf8");
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/steps.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => s.message)).toContain("HELPERS-SIBLING");
|
|
expect(result.steps.map((s) => s.message)).not.toContain("FLOWS-DIR-COPY");
|
|
});
|
|
|
|
it("refuses a bare run: name whose casing is not the one on disk", async () => {
|
|
// Completing the extension must not open a casing hole: `run: Login`
|
|
// becomes Login.yaml and meets the same on-disk spelling gate as the
|
|
// spelled-out form, so a macOS-green / Linux-red tree is still refused.
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "RAN-THE-LOWERCASE-FILE" }],
|
|
});
|
|
await fs.writeFile(
|
|
path.join(tmpDir, ".argent", "flows", "main.yaml"),
|
|
"steps:\n - run: Login\n",
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
expect(result.steps[0]?.reason ?? "").toMatch(/^mis-cased fragment reference "Login\.yaml": /);
|
|
expect(result.steps.map((s) => s.message)).not.toContain("RAN-THE-LOWERCASE-FILE");
|
|
});
|
|
|
|
it("resolves run: fragments beside an explicit top-level flow_path", async () => {
|
|
const externalDir = path.join(tmpDir, "external-flows");
|
|
const mainPath = path.join(externalDir, "main.yaml");
|
|
await fs.mkdir(externalDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(externalDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "sibling fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
mainPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "login.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute(
|
|
{},
|
|
{ project_root: tmpDir, flow_path: mainPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_path: {
|
|
clientPath: mainPath,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
statVerified: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
);
|
|
|
|
expect(result.flow).toBe("main");
|
|
expect(result.steps.map((step) => `${step.kind}:${step.status}`)).toEqual([
|
|
"run:pass",
|
|
"echo:pass",
|
|
]);
|
|
expect(result.steps[1]).toMatchObject({ flow: "login", message: "sibling fragment" });
|
|
});
|
|
|
|
it("names the lowercase requirement when only the flow_path extension's case is wrong", async () => {
|
|
const upperPath = path.join(tmpDir, "Main.YAML");
|
|
await fs.writeFile(
|
|
upperPath,
|
|
serializeFlow({ executionPrerequisite: "", steps: [{ kind: "echo", message: "hi" }] }),
|
|
"utf8"
|
|
);
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
await expect(
|
|
runFlow.execute(
|
|
{},
|
|
{ project_root: tmpDir, flow_path: upperPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_path: {
|
|
clientPath: upperPath,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
statVerified: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
).rejects.toThrow('flow files must use the lowercase .yaml extension, not ".YAML"');
|
|
});
|
|
|
|
it("resolves a run: path against the containing flow's own directory, not the root's", async () => {
|
|
// Root (in .argent/flows) → ../../shared/login.yaml → helper.yaml, where
|
|
// helper.yaml is login's OWN sibling in shared/ and absent from the root's
|
|
// directory — only per-file anchoring resolves it.
|
|
const sharedDir = path.join(tmpDir, "shared");
|
|
await fs.mkdir(sharedDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedDir, "helper.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "from helper" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(sharedDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helper.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../shared/login.yaml" }],
|
|
});
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"run:pass",
|
|
"echo:pass",
|
|
]);
|
|
// Attribution is the basename stem; the report target keeps the as-written path.
|
|
expect(result.steps[0]).toMatchObject({ flow: "login", target: "../../shared/login.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ flow: "helper", target: "helper.yaml" });
|
|
expect(result.steps[2]).toMatchObject({ flow: "helper", message: "from helper" });
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("does not flag two different files sharing a basename as a cycle", async () => {
|
|
const subDir = path.join(tmpDir, ".argent", "flows", "sub");
|
|
await fs.mkdir(subDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(subDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inner login" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
// Root flow "login" runs sub/login.yaml: same stem, different file.
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "sub/login.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("disambiguates a fragment whose stem collides with the root flow's name", async () => {
|
|
// Root login.yaml composes helpers/login.yaml — two different files, one
|
|
// stem. A bare-stem attribution would make every fragment step carry
|
|
// flow === the report's top-level flow, and renderers that mark fragment
|
|
// steps by that inequality (the CLI's `[fragment]` suffix) would read a
|
|
// failure inside the fragment as a failure of the root flow itself.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inside the shared fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/login.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.flow).toBe("login");
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
// On collision the attribution is the as-written path minus the .yaml
|
|
// extension — never the bare stem, which would equal result.flow — on the
|
|
// run marker and every expanded step alike.
|
|
expect(result.steps[0]).toMatchObject({ flow: "helpers/login", target: "helpers/login.yaml" });
|
|
expect(result.steps[1]).toMatchObject({
|
|
flow: "helpers/login",
|
|
message: "inside the shared fragment",
|
|
});
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("keeps the bare stem attribution when the fragment's stem does not collide with the root's", async () => {
|
|
// The SAME fragment as the collision test above, composed from a root
|
|
// with a different name: attribution stays the documented basename stem,
|
|
// because only a root-stem collision makes the stem ambiguous downstream.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inside the shared fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("checkout", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/login.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "checkout", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.flow).toBe("checkout");
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[0]).toMatchObject({ flow: "login", target: "helpers/login.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ flow: "login", message: "inside the shared fragment" });
|
|
});
|
|
|
|
it("qualifies a nested fragment's bare same-stem spelling as ./<stem>", async () => {
|
|
// Root login.yaml → helpers/steps.yaml → run: login.yaml. The bare
|
|
// spelling resolves against the CONTAINING file's directory, so it names
|
|
// helpers/login.yaml — a genuine fragment, not the root, and no cycle.
|
|
// Stripping the extension off a bare spelling reproduces the stem, which
|
|
// would equal result.flow and drop the renderers' fragment marker; the
|
|
// collision rule qualifies it as ./login — an equivalent spelling of the
|
|
// as-written reference that keeps the inequality.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "steps.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "login.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inside the nested sibling" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/steps.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.flow).toBe("login");
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"run:pass",
|
|
"echo:pass",
|
|
]);
|
|
// The nested run marker and the fragment's expanded step carry ./login —
|
|
// never the bare "login", which would read as the root flow itself.
|
|
expect(result.steps[1]).toMatchObject({ flow: "./login", target: "login.yaml" });
|
|
expect(result.steps[2]).toMatchObject({
|
|
flow: "./login",
|
|
message: "inside the nested sibling",
|
|
});
|
|
expect(result.steps[1]?.flow).not.toBe(result.flow);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("applies the root-stem collision disambiguation to a failed run: report too", async () => {
|
|
// Root login.yaml references helpers/login.yaml, which is MISSING — a
|
|
// moved or deleted fragment is the most common way a run: fails, and the
|
|
// errored marker (execRunStep's fail(), which stamps `flow: display`) is
|
|
// the line that names the missing file. Every other test reaching fail()
|
|
// uses a non-colliding root, so weakening just that line to the bare stem
|
|
// (`flow: runTargetName(target)`) would slip through the rest of the
|
|
// suite — here it would collapse flow to "login" === result.flow, and
|
|
// renderers that mark fragment lines by that inequality (the CLI's
|
|
// `[fragment]` suffix) would read the load failure as the root flow's own.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/login.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.flow).toBe("login");
|
|
expect(result.steps[0]).toMatchObject({
|
|
kind: "run",
|
|
status: "error",
|
|
flow: "helpers/login",
|
|
target: "helpers/login.yaml",
|
|
});
|
|
expect(result.steps[0]?.reason).toMatch(/^could not load fragment "helpers\/login\.yaml": /);
|
|
expect(result.steps[0]?.reason).toContain("ENOENT");
|
|
// The exact inequality renderers key on to mark fragment lines.
|
|
expect(result.steps[0]?.flow).not.toBe(result.flow);
|
|
});
|
|
|
|
it("detects a cycle reached through a different relative spelling", async () => {
|
|
const subDir = path.join(tmpDir, ".argent", "flows", "sub");
|
|
await fs.mkdir(subDir, { recursive: true });
|
|
await writeFlow("a", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "sub/b.yaml" }],
|
|
});
|
|
await fs.writeFile(
|
|
path.join(subDir, "b.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../a.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "a", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const errored = result.steps.find((s) => s.status === "error");
|
|
// The closing hop shares the root's stem (a cycle back to the root always
|
|
// does), so the collision disambiguation renders it as-written — which
|
|
// names the exact edge that closes the loop, rather than the bare "a"
|
|
// that reads like a self-reference from the root's own directory.
|
|
expect(errored?.reason).toMatch(/cyclic flow reference: a → b → \.\.\/a/);
|
|
// Where the as-written path differs from the stem it is the value that
|
|
// locates the reference — `run ../a.yaml`, not a stem-derived fallback.
|
|
expect(errored?.target).toBe("../a.yaml");
|
|
});
|
|
|
|
it("anchors a symlinked root flow's run: at the real file's directory", async () => {
|
|
// .argent/flows/main.yaml is a symlink to shared/flows/main.yaml, which
|
|
// references ../helpers/x.yaml. Only the canonical anchor resolves the
|
|
// shared helper; the un-canonicalized one would hit the project decoy.
|
|
const sharedFlows = path.join(tmpDir, "shared", "flows");
|
|
const sharedHelpers = path.join(tmpDir, "shared", "helpers");
|
|
const decoyHelpers = path.join(tmpDir, ".argent", "helpers");
|
|
await fs.mkdir(sharedFlows, { recursive: true });
|
|
await fs.mkdir(sharedHelpers, { recursive: true });
|
|
await fs.mkdir(decoyHelpers, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedFlows, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../helpers/x.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(sharedHelpers, "x.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "shared helper" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(decoyHelpers, "x.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "project decoy" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.symlink(path.join(sharedFlows, "main.yaml"), path.join(flowsDir, "main.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps[1]).toMatchObject({ kind: "echo", message: "shared helper" });
|
|
});
|
|
|
|
it("collides a fragment's stem against a symlinked root's SPELLED name, not its real file's", async () => {
|
|
// .argent/flows/smoke.yaml is a symlink to shared/flows/main.yaml —
|
|
// renaming on the link side is the ordinary reason to symlink a saved
|
|
// flow — whose only step is run: helpers/smoke.yaml. The fragment's stem
|
|
// "smoke" collides with the SPELLED root name (what result.flow carries),
|
|
// not the real file's stem "main", so the disambiguation must fire.
|
|
// Seeding the runStack with the canonical stem instead — `display:
|
|
// path.basename(canonicalPath, ".yaml")` in execute()'s seed — survives
|
|
// every other fixture, because each one points its symlink at a real file
|
|
// with the SAME basename (the two candidate values are equal); here it
|
|
// collapses the fragment's flow to "smoke" === result.flow, and renderers
|
|
// that mark fragment steps by that inequality (the CLI's `[fragment]`
|
|
// suffix) would print the fragment's steps as the root flow's own.
|
|
const sharedFlows = path.join(tmpDir, "shared", "flows");
|
|
const sharedHelpers = path.join(sharedFlows, "helpers");
|
|
await fs.mkdir(sharedHelpers, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedFlows, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/smoke.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(sharedHelpers, "smoke.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inside the shared fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.symlink(path.join(sharedFlows, "main.yaml"), path.join(flowsDir, "smoke.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "smoke", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.flow).toBe("smoke");
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
// The run marker and the fragment's expanded step both carry the
|
|
// as-written path minus the extension — never the bare stem.
|
|
expect(result.steps[0]).toMatchObject({ flow: "helpers/smoke", target: "helpers/smoke.yaml" });
|
|
expect(result.steps[1]).toMatchObject({
|
|
flow: "helpers/smoke",
|
|
message: "inside the shared fragment",
|
|
});
|
|
// The exact inequality renderers key on to mark fragment lines.
|
|
expect(result.steps[0]?.flow).not.toBe(result.flow);
|
|
expect(result.steps[1]?.flow).not.toBe(result.flow);
|
|
});
|
|
|
|
it("resolves a run: `..` after a symlinked component with kernel semantics", async () => {
|
|
// .argent/flows/link is a symlink to lex/other, so on disk
|
|
// `link/../frag.yaml` means lex/frag.yaml — `..` names the parent of the
|
|
// link's TARGET. A lexical collapse of the spelling (path.resolve before
|
|
// realpath) instead names the flows-dir sibling frag.yaml; with a decoy
|
|
// planted there, only kernel-faithful resolution runs the file the
|
|
// written path denotes.
|
|
const lexDir = path.join(tmpDir, "lex");
|
|
await fs.mkdir(path.join(lexDir, "other"), { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(lexDir, "frag.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "kernel-resolved fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "lexical decoy" }],
|
|
});
|
|
await writeFlow("root", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "link/../frag.yaml" }],
|
|
});
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.symlink(path.join(lexDir, "other"), path.join(flowsDir, "link"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "root", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[1]).toMatchObject({ kind: "echo", message: "kernel-resolved fragment" });
|
|
expect(result.steps.map((s) => s.message)).not.toContain("lexical decoy");
|
|
});
|
|
|
|
it("errors on a run: `..` through a dangling symlink instead of executing a lexical impostor", async () => {
|
|
// .argent/flows/dangling points at a missing target, so the kernel refuses
|
|
// the spelling `dangling/../frag.yaml` with ENOENT — realpath fails on both
|
|
// the full path and its dirname. A lexical collapse of the spelling would
|
|
// name the flows-dir sibling frag.yaml (planted as a decoy) and run it with
|
|
// run:pass; the fallback must instead surface the kernel's ENOENT for the
|
|
// spelling itself and never execute the decoy.
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "lexical decoy" }],
|
|
});
|
|
await writeFlow("root", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "dangling/../frag.yaml" }],
|
|
});
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.symlink(path.join(tmpDir, "gone"), path.join(flowsDir, "dangling"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "root", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
expect(result.steps[0]?.reason).toMatch(/could not load fragment "dangling\/\.\.\/frag\.yaml"/);
|
|
expect(result.steps[0]?.reason).toMatch(/ENOENT/);
|
|
expect(result.steps.map((s) => s.message)).not.toContain("lexical decoy");
|
|
});
|
|
|
|
it("runs a run: target that reaches above the project root", async () => {
|
|
// There is no path fence on `run:`. A target is reachable when the tool
|
|
// server can read it — the same reach the front door already grants,
|
|
// since flow_path can name any YAML on the host. The old fence made
|
|
// admission depend on project_root, which the CLI sets to the operator's
|
|
// cwd, so the identical file passed from the repo root and failed from a
|
|
// subdirectory. Project root is proj/, the target resolves above it, and
|
|
// it must run.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const flowsDir = path.join(proj, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(tmpDir, "outside.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "executed from outside project root" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(flowsDir, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../../outside.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: proj, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[0]).toMatchObject({ flow: "outside", target: "../../../outside.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ message: "executed from outside project root" });
|
|
});
|
|
|
|
it("reports a MISSING target above the project root as that file's own ENOENT", async () => {
|
|
// With the fence gone, a target that resolves nowhere gets the ordinary
|
|
// per-file load error naming the as-written spelling — the same report an
|
|
// in-project typo produces, wherever it points. The old boundary error
|
|
// ("resolves outside the project root") answered a typo above the root
|
|
// with a policy message about a boundary that no longer exists, sending
|
|
// the author to look at project_root instead of at the spelling.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const flowsDir = path.join(proj, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(flowsDir, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../../nothing-here.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: proj, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
expect(result.steps[0]?.reason).toMatch(
|
|
/could not load fragment "\.\.\/\.\.\/\.\.\/nothing-here\.yaml"/
|
|
);
|
|
expect(result.steps[0]?.reason).toMatch(/ENOENT/);
|
|
});
|
|
|
|
it("gives the same verdict for one flow file under two different project_root values", async () => {
|
|
// The regression the fence caused: admission was decided by
|
|
// `params.project_root`, which `argent flow run` sets to process.cwd(),
|
|
// so `cd packages/app && argent flow run .argent/flows/e2e.yaml` failed
|
|
// while the identical file passed from the repo root — green locally, red
|
|
// in CI, for a reason nothing in the flow file expresses. The two roots
|
|
// here STRADDLE the target: shared/login.yaml sits under tmpDir, which the
|
|
// old fence's project-root zone admitted, and outside proj, which neither
|
|
// that zone nor the containing file's directory (proj/.argent/flows)
|
|
// covered — so the fence passed the first and refused the second. Same
|
|
// flow_path, same sideways `run:` target: the two must now reach the same
|
|
// verdict and execute the fragment.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const flowsDir = path.join(proj, ".argent", "flows");
|
|
const sharedDir = path.join(tmpDir, "shared");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.mkdir(sharedDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "shared login" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const mainPath = path.join(flowsDir, "main.yaml");
|
|
await fs.writeFile(
|
|
mainPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../../shared/login.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
// The flow_path route with a boundary-resolved, stat-verified client path
|
|
// is exactly what `argent flow run` sends (see packages/argent-cli/src/flow.ts).
|
|
const runFrom = async (projectRoot: string): Promise<FlowRunResult> =>
|
|
asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ project_root: projectRoot, flow_path: mainPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_path: {
|
|
clientPath: mainPath,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
statVerified: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
);
|
|
|
|
const fromAboveProject = await runFrom(tmpDir);
|
|
const fromProject = await runFrom(proj);
|
|
|
|
for (const result of [fromAboveProject, fromProject]) {
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[1]).toMatchObject({ flow: "login", message: "shared login" });
|
|
}
|
|
});
|
|
|
|
it("runs a symlinked-out vendored tree's own subtree fragments", async () => {
|
|
// The vault/vendored layout with the real tree OUTSIDE the project root:
|
|
// proj/.argent/flows/main.yaml is a symlink to vendor/flows/main.yaml,
|
|
// which composes helpers/x.yaml below its own real directory. Every path
|
|
// here is anchored on the containing file's CANONICAL directory, so the
|
|
// fragment is looked up beside the real main.yaml in vendor/flows, not
|
|
// beside the symlink in the project — which is what makes the
|
|
// SKILL-recommended shared-tree layout work at all.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const projFlows = path.join(proj, ".argent", "flows");
|
|
const vendorFlows = path.join(tmpDir, "vendor", "flows");
|
|
await fs.mkdir(projFlows, { recursive: true });
|
|
await fs.mkdir(path.join(vendorFlows, "helpers"), { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "helpers", "x.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "vendored helper" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/x.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.symlink(path.join(vendorFlows, "main.yaml"), path.join(projFlows, "main.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: proj, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[1]).toMatchObject({ kind: "echo", message: "vendored helper" });
|
|
});
|
|
|
|
it("lets a nested fragment in a symlinked-out tree reach a sibling of its root flow", async () => {
|
|
// Same vendored layout, one level deeper: main.yaml → helpers/b.yaml →
|
|
// ../sibling.yaml, where sibling.yaml sits beside main.yaml in the vendor
|
|
// tree. This is the case the review filed. The old fence narrowed on
|
|
// every descent — from helpers/ the admitted zone was helpers/ itself,
|
|
// and the vendor tree is by definition not under the project root — so a
|
|
// fragment could reach that sibling from main.yaml but not from one
|
|
// directory down, capping a shared flows tree at a single level. Nothing
|
|
// narrows now: each hop anchors on its own containing file's canonical
|
|
// directory and the target is read if it is there.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const projFlows = path.join(proj, ".argent", "flows");
|
|
const vendorFlows = path.join(tmpDir, "vendor", "flows");
|
|
await fs.mkdir(path.join(vendorFlows, "helpers"), { recursive: true });
|
|
await fs.mkdir(projFlows, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "sibling.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "vendored sibling" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "helpers", "b.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../sibling.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/b.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.symlink(path.join(vendorFlows, "main.yaml"), path.join(projFlows, "main.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: proj, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"run:pass",
|
|
"echo:pass",
|
|
]);
|
|
expect(result.steps[1]).toMatchObject({ flow: "sibling", target: "../sibling.yaml" });
|
|
expect(result.steps[2]).toMatchObject({ message: "vendored sibling" });
|
|
});
|
|
|
|
it("detects a cycle through a symlinked spelling", async () => {
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await writeFlow("a", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "alias.yaml" }],
|
|
});
|
|
await fs.symlink(path.join(flowsDir, "a.yaml"), path.join(flowsDir, "alias.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "a", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
// Exact-match on the whole report, not a substring: the symlink-aware
|
|
// guard trips on the FIRST hop (realpath equates alias.yaml with the
|
|
// running a.yaml), one error step and nothing expanded. Without realpath
|
|
// the run recurses a level before the resolved-path comparison catches
|
|
// it — "a → alias → alias" after an expanded pass — which a substring
|
|
// match on the shorter chain would still accept.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}:${s.depth ?? 0}`)).toEqual([
|
|
"run:error:0",
|
|
]);
|
|
expect(result.steps[0]?.reason).toBe("cyclic flow reference: a → alias");
|
|
});
|
|
|
|
it("names a cycle that closes exactly at the run-depth limit as a cycle", async () => {
|
|
// The closing hop back to root fires with a run stack of exactly
|
|
// MAX_RUN_DEPTH — the same hop the depth guard would reject. It is still a
|
|
// loop, and the chain is what tells the author which edge to cut, so the
|
|
// cycle must win here; ordering the depth guard first swallows both.
|
|
const links = MAX_RUN_DEPTH - 1;
|
|
await writeRunChain(links, "root.yaml");
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "root", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
// The closing hop is the bare spelling `root.yaml`, whose stem is the
|
|
// root's own name — the collision rule qualifies a bare same-stem
|
|
// spelling as `./<stem>`, so the chain's final entry renders as ./root.
|
|
const chain = ["root", ...Array.from({ length: links }, (_, i) => `n${i + 1}`), "./root"];
|
|
const errored = result.steps.find((s) => s.status === "error");
|
|
expect(errored?.reason).toBe(`cyclic flow reference: ${chain.join(" → ")}`);
|
|
expect(errored?.reason).not.toContain("max run depth");
|
|
});
|
|
|
|
it("reports a non-cyclic chain past the run-depth limit as excessive depth", async () => {
|
|
// Same shape and same threshold as the cycle above, but every flow is
|
|
// distinct — this is the depth guard's remaining reachable case, and it
|
|
// must keep firing now that the cycle guard precedes it.
|
|
const links = MAX_RUN_DEPTH - 1;
|
|
await writeRunChain(links, `n${MAX_RUN_DEPTH}.yaml`);
|
|
await writeFlow(`n${MAX_RUN_DEPTH}`, {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "one hop too deep to reach" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "root", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const errored = result.steps.find((s) => s.status === "error");
|
|
expect(errored?.reason).toBe("max run depth exceeded");
|
|
expect(errored?.flow).toBe(`n${MAX_RUN_DEPTH}`);
|
|
// The as-written path rides the depth error too — the report line must
|
|
// name the reference that overflowed, not just the stem.
|
|
expect(errored?.target).toBe(`n${MAX_RUN_DEPTH}.yaml`);
|
|
});
|
|
|
|
it("reports a missing run: target as a step error, not a tool-level rejection", async () => {
|
|
// A typo'd or moved fragment path is the most common way a run: fails,
|
|
// and it must land in the per-step report — resolved, ok: false, with the
|
|
// underlying cause — never reject the whole tool call. The catch around
|
|
// the fragment read is the only thing between the two.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "gone.yaml" },
|
|
{ kind: "echo", message: "never reached" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps[0]).toMatchObject({
|
|
kind: "run",
|
|
status: "error",
|
|
flow: "gone",
|
|
target: "gone.yaml",
|
|
});
|
|
expect(result.steps[0]?.reason).toMatch(/^could not load fragment "gone\.yaml": /);
|
|
expect(result.steps[0]?.reason).toContain("ENOENT");
|
|
// The failure hard-stops the flow like any other step error.
|
|
expect(result.steps[1]).toMatchObject({ kind: "echo", status: "skip" });
|
|
});
|
|
|
|
it("reports a malformed run: target as a step error carrying the parse failure", async () => {
|
|
// The same catch covers parseFlow, so a fragment that exists but does not
|
|
// parse degrades identically — the parse diagnostic reaches the step
|
|
// report instead of rejecting the run.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "broken.yaml" }],
|
|
});
|
|
// Written raw — serializeFlow could never produce an invalid step.
|
|
await fs.writeFile(
|
|
path.join(tmpDir, ".argent", "flows", "broken.yaml"),
|
|
"steps:\n - frobnicate: nope\n",
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps[0]).toMatchObject({
|
|
kind: "run",
|
|
status: "error",
|
|
flow: "broken",
|
|
target: "broken.yaml",
|
|
});
|
|
expect(result.steps[0]?.reason).toMatch(/^could not load fragment "broken\.yaml": /);
|
|
expect(result.steps[0]?.reason).toContain("unrecognized step kind");
|
|
});
|
|
|
|
it("bounds the reason when a run: target is in-project YAML but not a flow", async () => {
|
|
// A mistyped run: path can select any YAML file inside the project root —
|
|
// here a CI config whose sole top-level key happens to be `steps`. Its
|
|
// first entry lands in the parse diagnostic, and the reason ships verbatim
|
|
// to `argent flow run` stdout and into the MCP content block, so badEntry's
|
|
// cap must keep the file's values from surviving to those surfaces in full.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../ci/deploy.yaml" }],
|
|
});
|
|
await fs.mkdir(path.join(tmpDir, "ci"), { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(tmpDir, "ci", "deploy.yaml"),
|
|
`steps:\n - db_password: "hunter2-${"x".repeat(4000)}-PROD-TAIL"\n`,
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps[0]).toMatchObject({ kind: "run", status: "error" });
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^could not load fragment "\.\.\/\.\.\/ci\/deploy\.yaml": /);
|
|
expect(reason).toContain("…(+");
|
|
expect(reason).not.toContain("PROD-TAIL");
|
|
expect(reason.length).toBeLessThan(500);
|
|
});
|
|
|
|
it("refuses a mis-cased run: target instead of running the case-folded file", async () => {
|
|
// These tests run on macOS, whose case-insensitive APFS is precisely what
|
|
// makes this reachable: `run: Frag.yaml` opens the file really named
|
|
// frag.yaml, the fragment's steps run, and every one of them is attributed
|
|
// to "Frag" — a name no directory entry carries. The same tree on a
|
|
// case-sensitive checkout (Linux CI) dies with ENOENT, so the slip must be
|
|
// refused where it is authored, not where it is replayed. Nothing of the
|
|
// fragment may execute: the refusal precedes the read.
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "RAN-THE-LOWERCASE-FILE" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "Frag.yaml" },
|
|
{ kind: "echo", message: "never reached" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error", "echo:skip"]);
|
|
expect(result.steps[0]).toMatchObject({ kind: "run", flow: "Frag", target: "Frag.yaml" });
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^mis-cased fragment reference "Frag\.yaml": /);
|
|
expect(reason).toContain(`no directory entry is named "Frag.yaml"`);
|
|
expect(reason).toContain(`case-insensitively to "frag.yaml"`);
|
|
// The recovery quotes a target the run: gate would itself accept.
|
|
expect(reason).toContain(`reference it as "frag.yaml"`);
|
|
// Not the load error: the file was never read, so nothing of it ran.
|
|
expect(reason).not.toContain("could not load fragment");
|
|
expect(result.steps.map((s) => s.message)).not.toContain("RAN-THE-LOWERCASE-FILE");
|
|
});
|
|
|
|
it("runs a run: target whose mixed-case spelling is the one really on disk", async () => {
|
|
// The gate compares byte-for-byte against the listing, not against a
|
|
// lowercase convention: an on-disk Frag.yaml is a perfectly portable
|
|
// fragment and `run: Frag.yaml` must compose it, with the mixed-case stem
|
|
// keying its steps. A gate that merely lowercased the target would fail
|
|
// this and break every camelCase flow tree.
|
|
await writeFlow("Frag", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "from the mixed-case file" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "Frag.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[0]).toMatchObject({ flow: "Frag", target: "Frag.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ flow: "Frag", message: "from the mixed-case file" });
|
|
});
|
|
|
|
it("runs a symlinked run: spelling whose real file is named differently", async () => {
|
|
// The listing consulted must be the directory the target is SPELLED in,
|
|
// compared against the SPELLED basename. Comparing against
|
|
// path.basename(canonical) instead — realpath rewrites a symlink to its
|
|
// target — would see "a.yaml" for `run: alias.yaml` and refuse a link that
|
|
// every directory entry backs and that the cycle guard already treats as a
|
|
// first-class spelling.
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await writeFlow("a", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "reached through the link" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "alias.yaml" }],
|
|
});
|
|
await fs.symlink(path.join(flowsDir, "a.yaml"), path.join(flowsDir, "alias.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[0]).toMatchObject({ flow: "alias", target: "alias.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ message: "reached through the link" });
|
|
});
|
|
|
|
it("reports a target absent from the listing as ENOENT, not as a casing slip", async () => {
|
|
// Only a case-FOLDED verdict may refuse. A mixed-case target with no
|
|
// case-insensitive neighbour at all is an ordinary missing fragment, and
|
|
// the per-file ENOENT from the read names the spelling and the reason far
|
|
// better than a casing complaint could — a gate that fired on `absent` too
|
|
// would answer every typo with advice about capitalization.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "Gone.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^could not load fragment "Gone\.yaml": /);
|
|
expect(reason).toContain("ENOENT");
|
|
expect(reason).not.toContain("case-insensitively");
|
|
});
|
|
|
|
it("refuses a mis-cased run: target in a subdirectory", async () => {
|
|
// A multi-segment target is checked in whatever directory it lands in, not
|
|
// only in the containing flow's own — the phantom spelling is exactly as
|
|
// unportable one level down, and a check anchored on the anchor directory
|
|
// alone would list the wrong directory and wave it through.
|
|
const subDir = path.join(tmpDir, ".argent", "flows", "sub");
|
|
await fs.mkdir(subDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(subDir, "frag.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "RAN-THE-SUBDIR-FILE" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "sub/Frag.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^mis-cased fragment reference "sub\/Frag\.yaml": /);
|
|
expect(reason).toContain(`case-insensitively to "frag.yaml"`);
|
|
// The hint keeps the target's own directory prefix, so it is the line the
|
|
// author can paste back into the flow.
|
|
expect(reason).toContain(`reference it as "sub/frag.yaml"`);
|
|
expect(result.steps.map((s) => s.message)).not.toContain("RAN-THE-SUBDIR-FILE");
|
|
});
|
|
|
|
it("asks for a rename when the on-disk spelling no run: target could reach", async () => {
|
|
// The on-disk name here is frag.YAML, which the run: extension gate
|
|
// (parseRunTarget) refuses outright — quoting it back as a replacement
|
|
// target would send the author to an error one layer down. The only way
|
|
// out is the rename, so that is what the recovery asks for.
|
|
await fs.mkdir(path.join(tmpDir, ".argent", "flows"), { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(tmpDir, ".argent", "flows", "frag.YAML"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "RAN-THE-SHOUTY-FILE" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "frag.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^mis-cased fragment reference "frag\.yaml": /);
|
|
expect(reason).toContain(`rename "frag.YAML" to "frag.yaml"`);
|
|
expect(reason).not.toContain("reference it as");
|
|
expect(result.steps.map((s) => s.message)).not.toContain("RAN-THE-SHOUTY-FILE");
|
|
});
|
|
|
|
it("rejects run: composition when the root flow was uploaded (no shared filesystem)", async () => {
|
|
// A remote client's flow arrives as content and is materialized to a temp
|
|
// file — the files its run: paths reference stayed on the client, and a
|
|
// same-named file on the server must never be read in their place. The
|
|
// rejection is a preflight contract error, so no step (e.g. a leading
|
|
// launch or tap) executes before it fires.
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "echo", message: "before" },
|
|
{ kind: "run", flow: "login.yaml" },
|
|
{ kind: "echo", message: "after" },
|
|
],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(tmpDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "server-local file" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
await expect(
|
|
createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
).rejects.toThrow(/co-located/i);
|
|
// Preflight, not mid-run: nothing was dispatched to the device.
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects an uploaded flow whose run: hides behind a when: block that would not fire", async () => {
|
|
// Without the preflight walking when: children, this flow reports green on
|
|
// iOS and only errors when the android guard first fires (e.g. in CI).
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n - when:\n platform: android\n steps:\n - run: login.yaml\n",
|
|
"utf8"
|
|
);
|
|
|
|
await expect(
|
|
createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
).rejects.toThrow(/co-located/i);
|
|
});
|
|
|
|
it("rejects an uploaded flow whose run: sits two when: blocks deep", async () => {
|
|
// Pins walkSteps' recursion DEPTH, which the single-level test above
|
|
// cannot: a one-level peek (`yield* inner` in place of
|
|
// `yield* walkSteps(inner)`) still passes that neighbor but never reaches
|
|
// this run:, so the flow reports ok: true with the run line inside a block
|
|
// skip — the silent-green CI outcome the preflight exists to prevent. The
|
|
// parser nests when: to MAX_BLOCK_DEPTH (20), so two levels is ordinary
|
|
// authorable YAML, not an edge case.
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n" +
|
|
" - when:\n" +
|
|
" platform: android\n" +
|
|
" steps:\n" +
|
|
" - when:\n" +
|
|
" platform: android\n" +
|
|
" steps:\n" +
|
|
" - run: login.yaml\n",
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
const err = await createRunFlowTool(registry)
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
() => null,
|
|
(e: unknown) => e
|
|
);
|
|
expect(err).toBeInstanceOf(Error);
|
|
expect((err as Error).message).toContain('run: composition ("run: login.yaml")');
|
|
expect(getFailureSignal(err)?.failure_stage).toBe("flow_upload_run_composition");
|
|
// Preflight, not mid-run: nothing was dispatched to the device.
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects an uploaded chromium e2e flow's run: before booting the launch's Electron app", async () => {
|
|
// The guard's docstring promises the rejection fires "before anything
|
|
// executes" — a POSITIONAL contract its mere existence doesn't keep. For
|
|
// an uploaded chromium e2e flow that also carries a run:, the guard
|
|
// sitting above resolveRunDevice is what decides whether an Electron
|
|
// instance is booted (and orphaned by the throw, which lands before the
|
|
// teardown finally) just to deliver a contract error. The absolute app
|
|
// path is deliberate — legal for an upload — so nothing else stands
|
|
// between the boot and the error: move the guard call below
|
|
// `const device = resolved.device;` and the boot fires first.
|
|
bootElectronApp.mockClear();
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n - launch: { chromium: /abs/e2e-app }\n - run: login.yaml\n",
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
await expect(
|
|
createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
).rejects.toThrow(/run: composition/);
|
|
// Preflight, not post-boot: no Electron instance was spawned and no
|
|
// device was resolved before the rejection.
|
|
expect(bootElectronApp).not.toHaveBeenCalled();
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects an uploaded fragment's run: instead of answering the prerequisite handshake", async () => {
|
|
// The other half of the guard's "before anything executes" position: it
|
|
// sits above the executionPrerequisite early return. A prerequisite is
|
|
// fragments-only (no leading launch) and fragments can carry run:, so an
|
|
// uploaded fragment can declare both. With the blocks
|
|
// swapped, the un-acknowledged call gets the prerequisite notice back —
|
|
// sending the agent off to do manual device setup for a flow the very
|
|
// next call rejects. The contract error must win.
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "Signed into the demo account",
|
|
steps: [{ kind: "run", flow: "login.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const outcome = await createRunFlowTool(mockRegistry())
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
(r) => r,
|
|
(e: unknown) => e
|
|
);
|
|
// A rejection — not a { notice, executionPrerequisite } return: on a
|
|
// failure this prints the notice that leaked through instead.
|
|
expect(outcome).toBeInstanceOf(Error);
|
|
expect((outcome as Error).message).toContain('run: composition ("run: login.yaml")');
|
|
expect(getFailureSignal(outcome)?.failure_stage).toBe("flow_upload_run_composition");
|
|
});
|
|
|
|
it("rejects an uploaded flow quoting the first run:'s as-written directory-qualified path", async () => {
|
|
// The guard's quote is the author's remediation pointer: with two
|
|
// same-stem steps (`run: ios/login.yaml`, then `run: android/login.yaml`)
|
|
// the throw fires on the FIRST offender, and only the as-written
|
|
// directory says which of the two lines it means. Reducing the target to
|
|
// its basename stem (runTargetName — what the report attribution sites
|
|
// use, and what `step.flow` carried before targets kept their spelling)
|
|
// would print `run: login`, a string appearing nowhere in the flow and
|
|
// ambiguous between the two steps. The bare-spelling rejections above
|
|
// pin only the `.yaml` suffix (there the as-written path and the stem
|
|
// differ by nothing else), so this test is what holds the directory
|
|
// component — the contract content.ts pins for the MCP renderer
|
|
// ("`run ios/login.yaml` and `run android/login.yaml` must render
|
|
// distinctly, not as one stem"); this error path is that same reasoning
|
|
// on the tool-server surface.
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n - run: ios/login.yaml\n - run: android/login.yaml\n",
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
const err = await createRunFlowTool(registry)
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
() => null,
|
|
(e: unknown) => e
|
|
);
|
|
expect(err).toBeInstanceOf(Error);
|
|
expect((err as Error).message).toContain('run: composition ("run: ios/login.yaml")');
|
|
expect(getFailureSignal(err)?.failure_stage).toBe("flow_upload_run_composition");
|
|
// Preflight, not mid-run: nothing was dispatched to the device.
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("allows run: composition for a co-located flow_file resolved in place", async () => {
|
|
// The everyday co-located client: the flow_file boundary resolves the
|
|
// exact ${project_root}/.argent/flows/${name}.yaml path on a shared
|
|
// filesystem (presentOnHost, NOT an upload). This return is what carries
|
|
// the whole feature — misclassifying it as an upload would reject every
|
|
// local run: composition with the co-located contract error, so the
|
|
// upload-rejection tests above need this inverse pin.
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "composed fragment ran" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "login.yaml" }],
|
|
});
|
|
const flowFile = path.join(tmpDir, ".argent", "flows", "main.yaml");
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: flowFile, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: flowFile,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps[1]).toMatchObject({
|
|
kind: "echo",
|
|
status: "pass",
|
|
message: "composed fragment ran",
|
|
});
|
|
});
|
|
|
|
it("rejects a snapshot step when the root flow was uploaded (no durable baseline dir)", async () => {
|
|
// Baselines anchor beside the flow's file, and an uploaded flow
|
|
// materializes to a fresh temp directory each call — a plain snapshot can
|
|
// only fail on a missing baseline, and updateBaselines would write PNGs no
|
|
// later run can find. Rejected preflight, like uploaded run: composition.
|
|
vi.mocked(runSnapshot).mockClear();
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "echo", message: "before" },
|
|
{ kind: "snapshot", name: "home", maxMismatch: 0.5 },
|
|
],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
const err = await createRunFlowTool(registry)
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
() => null,
|
|
(e: unknown) => e
|
|
);
|
|
expect(err).toBeInstanceOf(Error);
|
|
expect((err as Error).message).toContain('snapshot step ("snapshot: home")');
|
|
expect(getFailureSignal(err)?.failure_stage).toBe("flow_upload_snapshot_baseline");
|
|
// Preflight, not mid-run: nothing was dispatched to the device and the
|
|
// differ was never pointed at the temp materialization dir.
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
expect(vi.mocked(runSnapshot)).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects an uploaded flow whose snapshot hides behind a when: block that would not fire", async () => {
|
|
// Same preflight walk as run: — a guard-gated snapshot must not report
|
|
// green on one platform and only surface the contract error where the
|
|
// guard first fires (e.g. in CI).
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n - when:\n platform: android\n steps:\n - snapshot:\n name: home\n",
|
|
"utf8"
|
|
);
|
|
|
|
const err = await createRunFlowTool(mockRegistry())
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
() => null,
|
|
(e: unknown) => e
|
|
);
|
|
expect(getFailureSignal(err)?.failure_stage).toBe("flow_upload_snapshot_baseline");
|
|
});
|
|
|
|
it("allows a snapshot step for a co-located flow_file resolved in place", async () => {
|
|
// The inverse pin for the snapshot upload rejection above: the everyday
|
|
// co-located client (presentOnHost, NOT an upload) keeps its durable
|
|
// baseline directory beside the flow file, so the snapshot path still runs.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
});
|
|
const flowFile = path.join(tmpDir, ".argent", "flows", "main.yaml");
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: flowFile, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: flowFile,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps[0]).toMatchObject({ kind: "snapshot", status: "pass" });
|
|
expect(vi.mocked(runSnapshot)).toHaveBeenCalledWith(
|
|
expect.anything(),
|
|
expect.objectContaining({
|
|
flowsDir: path.join(tmpDir, ".argent", "flows"),
|
|
flowName: "main",
|
|
})
|
|
);
|
|
});
|
|
|
|
it("attributes a hard-stop-skipped run: step to its target stem, like every other path", async () => {
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "never runs" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: { android: "com.acme.app" } }, // DEVICE is iOS → errors
|
|
{ kind: "run", flow: "login.yaml" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "run:skip"]);
|
|
// Same attribution as an executed/errored run marker: the target stem,
|
|
// with the as-written path in target — not the enclosing flow.
|
|
expect(result.steps[1]).toMatchObject({ flow: "login", target: "login.yaml" });
|
|
});
|
|
|
|
it("applies the root-stem collision disambiguation on the hard-stop skip path too", async () => {
|
|
// Same setup as above, but the root shares the fragment's stem — the skip
|
|
// report must carry the same disambiguated name an executed run marker
|
|
// would, or attribution would depend on whether the step ever ran.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "never runs" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: { android: "com.acme.app" } }, // DEVICE is iOS → errors
|
|
{ kind: "run", flow: "helpers/login.yaml" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "run:skip"]);
|
|
expect(result.steps[1]).toMatchObject({ flow: "helpers/login", target: "helpers/login.yaml" });
|
|
});
|
|
|
|
it("attributes a fragment's when marker and skipped block steps to the fragment, not the root", async () => {
|
|
// A platform guard is static (no tree fetch) and android never matches the
|
|
// iOS DEVICE, so the block skips with the fragment on the run stack. The
|
|
// marker line (execWhenStep) and the authored-step skip line
|
|
// (reportBlockSkipped → stepFlow's non-run branch) each derive attribution
|
|
// separately — all of them must name the fragment, or the CLI's
|
|
// `[fragment]` suffix would tag these lines with the wrong flow.
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "echo", message: "inside fragment" },
|
|
{
|
|
kind: "when",
|
|
condition: { kind: "platform", platform: "android" },
|
|
steps: [{ kind: "echo", message: "android only" }],
|
|
},
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "frag.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"echo:pass",
|
|
"when:skip",
|
|
"echo:skip",
|
|
]);
|
|
// Executed echo, unmet-guard marker, and block-skip line: all "frag".
|
|
expect(result.steps[1]).toMatchObject({ flow: "frag", message: "inside fragment" });
|
|
expect(result.steps[2]).toMatchObject({ flow: "frag", depth: 1 });
|
|
expect(result.steps[3]).toMatchObject({ flow: "frag", depth: 2, message: "android only" });
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("attributes hard-stop skips inside a fragment to the fragment, and the root's own to the root", async () => {
|
|
// The fragment's leading launch declares no iOS app id, so it errors and
|
|
// hard-stops the run with the fragment still on the stack. Every post-stop
|
|
// skip line inside the fragment — plain step, when marker, and the when
|
|
// block's expansion — must stay attributed to the fragment, while the
|
|
// root's trailing step flips back to the root: attribution follows the
|
|
// run stack, not where the stop happened.
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: { android: "com.acme.app" } }, // DEVICE is iOS → errors
|
|
{ kind: "echo", message: "fragment echo" },
|
|
{
|
|
kind: "when",
|
|
condition: { kind: "platform", platform: "ios" },
|
|
steps: [{ kind: "echo", message: "guarded echo" }],
|
|
},
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "frag.yaml" },
|
|
{ kind: "echo", message: "root echo" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"launch:error",
|
|
"echo:skip",
|
|
"when:skip",
|
|
"echo:skip",
|
|
"echo:skip",
|
|
]);
|
|
expect(result.steps[2]).toMatchObject({ flow: "frag", message: "fragment echo" });
|
|
expect(result.steps[3]).toMatchObject({ flow: "frag", kind: "when" });
|
|
expect(result.steps[4]).toMatchObject({ flow: "frag", message: "guarded echo" });
|
|
expect(result.steps[5]).toMatchObject({ flow: "main", message: "root echo" });
|
|
expect(result.ok).toBe(false);
|
|
});
|
|
|
|
it("keys and stores a composed fragment's snapshot with the root flow", async () => {
|
|
const sharedDir = path.join(tmpDir, "shared");
|
|
await fs.mkdir(sharedDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedDir, "visual.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../shared/visual.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
// The snapshot ran from shared/visual.yaml but stays anchored to the ROOT
|
|
// flow: baselines under the root flow's directory, keyed by the root name.
|
|
expect(vi.mocked(runSnapshot)).toHaveBeenCalledWith(
|
|
expect.anything(),
|
|
expect.objectContaining({
|
|
flowsDir: path.join(tmpDir, ".argent", "flows"),
|
|
flowName: "main",
|
|
})
|
|
);
|
|
});
|
|
|
|
it("anchors a symlinked root flow's snapshot baselines beside the real file", async () => {
|
|
// .argent/flows/visual.yaml is a symlink to shared/flows/visual.yaml. The
|
|
// baseline anchor must be the real file's directory — the same canonical
|
|
// anchor `run:` targets resolve against — not the symlink's spelling, or
|
|
// one root file reached through two spellings would keep two baseline sets.
|
|
const sharedFlows = path.join(tmpDir, "shared", "flows");
|
|
await fs.mkdir(sharedFlows, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedFlows, "visual.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.symlink(path.join(sharedFlows, "visual.yaml"), path.join(flowsDir, "visual.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "visual", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
// The real directory, not the spelling the run was addressed by. (The
|
|
// link and its target share a basename here, so this fixture cannot tell
|
|
// the two candidate KEYS apart — the vault tests below do that.)
|
|
expect(vi.mocked(runSnapshot)).toHaveBeenCalledWith(
|
|
expect.anything(),
|
|
expect.objectContaining({ flowsDir: sharedFlows, flowName: "visual" })
|
|
);
|
|
});
|
|
|
|
/**
|
|
* The baseline store is `<flowsDir>/__baselines__/<flowName>`, and flowsDir
|
|
* is the CANONICAL root flow's directory — so the key has to name the
|
|
* canonical file too, or the pair identifies no single file. These pin that
|
|
* agreement from both sides: two flows that share an anchor must not share a
|
|
* store, and a flow that is its own real file must not move.
|
|
*/
|
|
it("gives two projects' same-named symlinks into one vault separate baseline stores", async () => {
|
|
// The shared-vault layout: each project has its own legitimately named
|
|
// .argent/flows/smoke.yaml, both symlinked into one vault at differently
|
|
// named real files. The canonical anchor is vault/ for both, so keying by
|
|
// the as-written stem would give them ONE vault/__baselines__/smoke/:
|
|
// whichever project ran --update-baselines last would silently replace the
|
|
// other's reviewed, committed PNG — reported as "baseline updated", as
|
|
// though it had refreshed its own — and the other project's next run would
|
|
// then fail against a screen it never captured. Nothing is printed either
|
|
// way, so only the store paths can catch this.
|
|
const vault = path.join(tmpDir, "vault");
|
|
await fs.mkdir(vault, { recursive: true });
|
|
const snapshotFlow = serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
});
|
|
await fs.writeFile(path.join(vault, "a-smoke.yaml"), snapshotFlow, "utf8");
|
|
await fs.writeFile(path.join(vault, "b-smoke.yaml"), snapshotFlow, "utf8");
|
|
const vaultedProject = async (proj: string, target: string): Promise<string> => {
|
|
const flows = path.join(tmpDir, proj, ".argent", "flows");
|
|
await fs.mkdir(flows, { recursive: true });
|
|
await fs.symlink(path.join(vault, target), path.join(flows, "smoke.yaml"));
|
|
return path.join(tmpDir, proj);
|
|
};
|
|
const projA = await vaultedProject("projA", "a-smoke.yaml");
|
|
const projB = await vaultedProject("projB", "b-smoke.yaml");
|
|
|
|
vi.mocked(runSnapshot).mockClear();
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const a = asRun(
|
|
await runFlow.execute({}, { name: "smoke", project_root: projA, device: DEVICE })
|
|
);
|
|
const b = asRun(
|
|
await runFlow.execute({}, { name: "smoke", project_root: projB, device: DEVICE })
|
|
);
|
|
|
|
expect([a.ok, b.ok]).toEqual([true, true]);
|
|
// The caller-visible identity is untouched — both runs still report as the
|
|
// flow the user asked to run. Only the baseline key follows the real file.
|
|
expect([a.flow, b.flow]).toEqual(["smoke", "smoke"]);
|
|
|
|
const stores = vi
|
|
.mocked(runSnapshot)
|
|
.mock.calls.map(([, opts]) => path.join(opts.flowsDir, "__baselines__", opts.flowName));
|
|
expect(stores).toEqual([
|
|
path.join(vault, "__baselines__", "a-smoke"),
|
|
path.join(vault, "__baselines__", "b-smoke"),
|
|
]);
|
|
});
|
|
|
|
it("leaves a regular (non-symlinked) root flow's baseline store exactly where it was", async () => {
|
|
// The no-regression pin for every existing user: when the root flow is a
|
|
// real file its canonical stem IS the as-written one, so deriving the key
|
|
// from the canonical path must not move the store anyone has already
|
|
// seeded and committed under .argent/flows/__baselines__/<flow>/.
|
|
await writeFlow("checkout", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
});
|
|
|
|
vi.mocked(runSnapshot).mockClear();
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "checkout", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
const [, opts] = vi.mocked(runSnapshot).mock.calls[0]!;
|
|
expect(path.join(opts.flowsDir, "__baselines__", opts.flowName)).toBe(
|
|
path.join(tmpDir, ".argent", "flows", "__baselines__", "checkout")
|
|
);
|
|
});
|
|
|
|
it("keys a fragment's snapshot to a symlinked root's REAL file, not the fragment or the link", async () => {
|
|
// The root-anchoring pin above, extended to the symlinked-root case the
|
|
// key change touches: the snapshot is authored in vault/frag.yaml, so
|
|
// three names are in play — the fragment's stem ("frag"), the spelling the
|
|
// run was addressed by ("smoke"), and the root's real file ("a-smoke").
|
|
// Only the last agrees with the anchor the store sits in.
|
|
const vault = path.join(tmpDir, "vault");
|
|
await fs.mkdir(vault, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(vault, "a-smoke.yaml"),
|
|
serializeFlow({ executionPrerequisite: "", steps: [{ kind: "run", flow: "frag.yaml" }] }),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(vault, "frag.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.symlink(path.join(vault, "a-smoke.yaml"), path.join(flowsDir, "smoke.yaml"));
|
|
|
|
vi.mocked(runSnapshot).mockClear();
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "smoke", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "snapshot:pass"]);
|
|
expect(vi.mocked(runSnapshot)).toHaveBeenCalledWith(
|
|
expect.anything(),
|
|
expect.objectContaining({ flowsDir: vault, flowName: "a-smoke" })
|
|
);
|
|
// Report attribution is unchanged by the key: the root still reports under
|
|
// the name it was run as, the expanded step under the fragment's.
|
|
expect(result.flow).toBe("smoke");
|
|
expect(result.steps[1]?.flow).toBe("frag");
|
|
});
|
|
|
|
it("stamps nesting depth on expanded fragment steps (omitted at top level)", async () => {
|
|
await writeFlow("inner", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "deepest" }],
|
|
});
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "tool", name: "tap", args: { x: 0.5 } },
|
|
{ kind: "run", flow: "inner.yaml" },
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "login.yaml" },
|
|
{ kind: "echo", message: "done" },
|
|
],
|
|
});
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
|
|
// Each run marker sits at its enclosing depth; the fragment it expands runs
|
|
// one deeper. Top-level steps omit the field entirely, so a flow with no
|
|
// nesting steps reports byte-identically to the pre-depth shape.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.depth ?? 0}`)).toEqual([
|
|
"run:0",
|
|
"tool:1",
|
|
"run:1",
|
|
"echo:2",
|
|
"echo:0",
|
|
]);
|
|
expect(result.steps[0].depth).toBeUndefined();
|
|
expect(result.steps[4].depth).toBeUndefined();
|
|
});
|
|
|
|
it("expands a referenced e2e flow inline, launch step and all", async () => {
|
|
await writeFlow("other-e2e", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "echo", message: "in nested e2e" },
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "other-e2e.yaml" }],
|
|
});
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
// run marker, then the nested e2e's launch + echo expanded inline.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"launch:pass",
|
|
"echo:pass",
|
|
]);
|
|
expect(result.steps[1].flow).toBe("other-e2e");
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("runs a nested flow-execute against the run device, not the recorded one (issue #607)", async () => {
|
|
// The raw `tool: flow-execute` form is what the recorder falls back to when
|
|
// the target is not a resolvable sibling — and what a remote recording always
|
|
// produces. Its device parameter is named `device`, which was not a bind key,
|
|
// so the sub-run drove the id baked in at record time. Here the flow carries
|
|
// a device that does not exist while the run is given a real one.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{
|
|
kind: "tool",
|
|
name: "flow-execute",
|
|
args: { name: "b-only", project_root: "/elsewhere", device: "STALE-ID" },
|
|
},
|
|
],
|
|
});
|
|
|
|
// Single step + explicit device, per mockRegistry's contract.
|
|
const registry = mockRegistry({ name: {}, project_root: {}, device: {} });
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(registry.invokeTool).toHaveBeenCalledWith(
|
|
"flow-execute",
|
|
expect.objectContaining({ device: DEVICE })
|
|
);
|
|
expect(registry.invokeTool).not.toHaveBeenCalledWith(
|
|
"flow-execute",
|
|
expect.objectContaining({ device: "STALE-ID" })
|
|
);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("detects a cyclic run reference", async () => {
|
|
await writeFlow("a", { executionPrerequisite: "", steps: [{ kind: "run", flow: "b.yaml" }] });
|
|
await writeFlow("b", { executionPrerequisite: "", steps: [{ kind: "run", flow: "a.yaml" }] });
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "a.yaml" }],
|
|
});
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
const errored = result.steps.find((s) => s.status === "error");
|
|
expect(errored?.reason).toMatch(/cyclic/i);
|
|
// The chain renders the human-readable stems, not canonical paths.
|
|
expect(errored?.reason).toContain("main → a → b → a");
|
|
// The error report carries the as-written path: on a failed resolution it
|
|
// is what locates the bad reference (`run ../shared/x.yaml`, not a bare
|
|
// stem), and stepLabel in argent-mcp falls back to the stem without it.
|
|
expect(errored?.target).toBe("a.yaml");
|
|
// The cycle is detected two fragments down; its error marker keeps that
|
|
// depth (the fail() path stamps depthOf(scope) like the success marker),
|
|
// so the error line renders inside the block that caused it.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}:${s.depth ?? 0}`)).toEqual([
|
|
"run:pass:0",
|
|
"run:pass:1",
|
|
"run:error:2",
|
|
]);
|
|
});
|
|
|
|
it("executes a leading launch step from scratch (restart-app) and reports it", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "echo", message: "running" },
|
|
],
|
|
});
|
|
const registry = mockRegistry();
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:pass", "echo:pass"]);
|
|
// e2e contract: terminate + relaunch, so a running copy can't leak state.
|
|
expect(registry.invokeTool).toHaveBeenCalledWith("restart-app", { bundleId: "com.acme.app" });
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("errors the launch step when no app id is declared for the platform", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: { android: "com.acme.app" } }, // DEVICE is iOS
|
|
{ kind: "echo", message: "should never run" },
|
|
],
|
|
});
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "echo:skip"]);
|
|
expect(result.steps[0].reason).toMatch(/no app id declared for platform/i);
|
|
expect(result.ok).toBe(false);
|
|
});
|
|
|
|
it("errors the launch step when native devtools never connects on iOS", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "echo", message: "should never run" },
|
|
],
|
|
});
|
|
// Registry whose native-devtools service is unavailable: the launch step
|
|
// must fail rather than let selectors silently fall back to the AX tree.
|
|
// (An unresolvable service fails fast; a resolvable-but-never-connected
|
|
// one hits the same guard after the connect timeout.)
|
|
const resolveService = vi.fn(async () => {
|
|
throw new Error("native-devtools unavailable");
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService,
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "echo:skip"]);
|
|
expect(result.steps[0].reason).toMatch(/could not connect to native devtools/i);
|
|
// Every reason names the bundle id, so the prefix must not — doubled, it
|
|
// reads as two separate failures. Asserted here and on the measured and
|
|
// system-app shapes below, since the rule is worth nothing partially held.
|
|
expect(result.steps[0].reason?.match(/com\.acme\.app/g)).toHaveLength(1);
|
|
// Resolution fails for reasons the flow author can act on and cannot
|
|
// otherwise see — a socket already bound, a device of the wrong platform —
|
|
// and the step's reason is the only place any of them surfaces.
|
|
expect(result.steps[0].reason).toContain("native-devtools unavailable");
|
|
expect(result.ok).toBe(false);
|
|
expect(resolveService).toHaveBeenCalled();
|
|
});
|
|
|
|
it("waits out the gate for a com.apple.* launch, then withholds the verdict", async () => {
|
|
// The gate ties the launched bundle to the app a later selector step
|
|
// auto-targets, so the wait runs for every bundle (see `treeSourceGate`).
|
|
// Only the verdict is withheld for `com.apple.*`: the first selector read
|
|
// reports the missing hierarchy instead.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
// The bundle prefix match is case-insensitive.
|
|
{ kind: "launch", app: "com.APPLE.Preferences" },
|
|
{ kind: "echo", message: "should never run" },
|
|
],
|
|
});
|
|
const resolveService = vi.fn(async () => ({ isConnected: () => false }));
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService,
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:pass", "echo:pass"]);
|
|
expect(result.ok).toBe(true);
|
|
// The wait ran: a per-bundle skip would never touch the service.
|
|
expect(resolveService).toHaveBeenCalled();
|
|
// The pass is clean: no connection failure for a hierarchy the gate cannot
|
|
// wait for.
|
|
expect(result.steps[0].reason ?? "").not.toMatch(/could not connect to native devtools/i);
|
|
expect(result.steps[0].reason ?? "").not.toMatch(/stale or duplicate argent server/i);
|
|
// The launch spends the post-launch settle plus the full 15s
|
|
// NATIVE_DEVTOOLS_CONNECT_BUDGET_MS, so the 30s budget covers both on a
|
|
// loaded host.
|
|
}, 30000);
|
|
|
|
it("runs a coordinate-only flow green against an app that never connects", async () => {
|
|
// A raw `tool: restart-app` step dispatches through the registry, not
|
|
// `runLaunch`, so it never reaches `treeSourceGate`. Point taps and `tool:`
|
|
// steps resolve no selectors.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "tool", name: "restart-app", args: { bundleId: "com.apple.Preferences" } },
|
|
{
|
|
kind: "tool",
|
|
name: "await-ui-element",
|
|
args: { condition: "visible", selector: { text: "General" } },
|
|
},
|
|
{ kind: "tap", x: 0.5, y: 0.35 },
|
|
],
|
|
});
|
|
const resolveService = vi.fn(async () => ({ isConnected: () => false }));
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService,
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"tool:pass",
|
|
"tool:pass",
|
|
"tap:pass",
|
|
]);
|
|
expect(result.ok).toBe(true);
|
|
// No step gates on the connection this flow never gets. A gesture without a
|
|
// selector still settles the screen first, so the tap goes out unsettled and
|
|
// warns.
|
|
expect(result.steps[2].warning).toContain("without settling the screen");
|
|
});
|
|
|
|
it("passes the gate for a com.apple.* app that does connect", async () => {
|
|
// Argent treats `com.apple.*` as non-injectable, but simulator system apps
|
|
// do connect after a restart-app (measured on iOS 18.3 and 26.5).
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "echo", message: "runs once the system app has connected" },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({ isConnected: () => true })),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:pass", "echo:pass"]);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
// Argent refuses an Apple system app a flow tree, so its hierarchy never
|
|
// becomes readable — which is not a reason to fail the LAUNCH. The step
|
|
// started the app, and a flow that taps by coordinate needs nothing else; the
|
|
// refusal belongs where a selector actually needs the hierarchy.
|
|
it("lets a system-app launch through so a coordinate-driven flow still runs", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:pass", "tool:pass"]);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
// A blocked precheck comes back RESOLVED, and returns before the terminate and
|
|
// the launch — so this is a step whose app was never started. The gate's
|
|
// remedies are all written for one that was: read as a launch, `not_running`
|
|
// becomes "it exited after launch", which sends the author after a crash that
|
|
// never happened and drops the message that named the real cause.
|
|
it("fails a launch whose restart-app was blocked before it started anything", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) => {
|
|
if (id === "list-devices") return { devices: [] };
|
|
if (id === "restart-app") {
|
|
return {
|
|
status: "init_failed",
|
|
attempts: 3,
|
|
message:
|
|
`Native devtools failed to initialize for ${DEVICE} after 3 attempts. ` +
|
|
`Last error: Native devtools dylib not found: /x/libArgentInjectionBootstrap.dylib. ` +
|
|
`Try shutting down and re-booting the simulator, or restart CoreSimulatorService.`,
|
|
};
|
|
}
|
|
return { ok: true };
|
|
}),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
// What the gate would measure for an app nothing launched.
|
|
appConnectionState: async () => "not_running" as const,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "tool:skip"]);
|
|
expect(result.steps[0].reason).toContain("dylib not found");
|
|
expect(result.steps[0].reason).not.toContain("exited after launch");
|
|
});
|
|
|
|
// The connected case above never reaches the gate's verdict. This is the one
|
|
// that matters: a system app that never connects at all. The launch must STILL
|
|
// pass, where every measured state would have failed it — and failed it with a
|
|
// remedy that cannot apply to such an app.
|
|
it("passes a system-app launch that never connects at all", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
// The launchd env carrying the bootstrap dylib is simulator-wide, so a
|
|
// system app's process inherits the injection tokens and scores as a
|
|
// live app the service merely never registered.
|
|
appConnectionState: async () => "unregistered" as const,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"launch:pass",
|
|
"tool:pass",
|
|
]);
|
|
expect(result.ok).toBe(true);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
// The verdict is withheld BEFORE the state is measured: that measurement is
|
|
// several uninterruptible simctl round-trips, and no arm consults it for such
|
|
// an app — pure latency on every system-app launch step.
|
|
it("does not measure a state it will not report", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "launch", app: "com.apple.Preferences" }],
|
|
});
|
|
const appConnectionState = vi.fn(async () => "unregistered" as const);
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps[0].status).toBe("pass");
|
|
expect(appConnectionState).not.toHaveBeenCalled();
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
// The gate consults the service before withholding its verdict, so an
|
|
// unresolvable service was the one remaining way a system-app launch could
|
|
// fail — on a failure irrelevant to an app it was never going to serve.
|
|
it("passes a system-app launch even when native-devtools cannot resolve", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => {
|
|
throw new Error("listen EADDRINUSE: address already in use /tmp/argent-nd-00000000.sock");
|
|
}),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:pass", "tool:pass"]);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
// The control for the test above: an injectable app in the identical state
|
|
// must still fail, or the pass-through would be excusing every service
|
|
// failure rather than the one it reasons about.
|
|
it("still fails an injectable launch when native-devtools cannot resolve", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => {
|
|
throw new Error("listen EADDRINUSE: address already in use /tmp/argent-nd-00000000.sock");
|
|
}),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps[0].status).toBe("error");
|
|
expect(result.steps[0].reason).toMatch(/native-devtools service is unavailable/);
|
|
});
|
|
|
|
// The other half of letting the launch through: a SELECTOR step against the
|
|
// same app has to say why it cannot resolve, terminally. Also the only
|
|
// end-to-end proof that the launched bundle id reaches the tree source —
|
|
// without it the author gets the stock "Launch or restart the app first"
|
|
// auto-target text, the restart loop this measurement exists to break.
|
|
it("gives a selector step against a system app the terminal reason, not the auto-target text", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "assert", selector: { text: "General" }, condition: "visible" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps[0].status).toBe("pass");
|
|
const reason = result.steps[1].reason ?? "";
|
|
expect(reason).toMatch(/Apple system app/);
|
|
expect(reason).toMatch(/com\.apple\.Preferences/);
|
|
// The reason must also name the coordinate remedy this launch was let
|
|
// through for.
|
|
expect(reason).toContain("`tap: { x: 0.5, y: 0.35 }` takes a point directly and reads no tree");
|
|
// Not the native-* dead-end warning: none of those tools is a flow step.
|
|
expect(reason).not.toMatch(/native-describe-screen|native-find-views/);
|
|
// The auto-target text is what auto-resolution alone can produce here, and
|
|
// its remedy is the loop.
|
|
expect(reason).not.toMatch(/auto-targeting/);
|
|
expect(reason).not.toMatch(/Launch or restart the app first/);
|
|
});
|
|
|
|
// `assert`/`await` read the tree through `waitForCondition`; `tap`, `type`,
|
|
// `scroll-to` and `long-press` reach it through `settleTree`, a separate call
|
|
// site. Pinning only the first left the id droppable at the second — the one
|
|
// every action directive uses — with the suite green.
|
|
it("threads the launched id to the settleTree read an action directive uses", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tap", selector: { text: "General" } },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const reason = result.steps[1].reason ?? "";
|
|
expect(reason).toMatch(/Apple system app/);
|
|
expect(reason).not.toMatch(/Launch or restart the app first/);
|
|
});
|
|
|
|
// `launch-app` / `restart-app` are in FOREGROUND_CHANGING_TOOLS, which drops
|
|
// the launched id — but unlike the rest of that set they NAME the app they
|
|
// switched to. Dropping it there sends every read from that step onward back
|
|
// to the auto-target text, in the middle of a run that has a launched app.
|
|
it.each(["restart-app", "launch-app"])(
|
|
"keeps the launched id across a %s tool step that names the same app",
|
|
async (name) => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name, args: { bundleId: "com.apple.Preferences" } },
|
|
{ kind: "assert", selector: { text: "General" }, condition: "visible" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const reason = result.steps[2].reason ?? "";
|
|
// Both reasons say "Apple system app", so that phrase no longer separates
|
|
// the two paths. Only the launched-id diagnosis NAMES the bundle: it is
|
|
// handed the id the tool step preserved. Drop that id and the read falls
|
|
// through to auto-targeting, whose no-connection text names no app.
|
|
expect(reason).toMatch(/com\.apple\.Preferences is an Apple system app/);
|
|
expect(reason).not.toMatch(/no app is connected to native devtools/);
|
|
}
|
|
);
|
|
|
|
// The control that keeps the restore scoped: `button` can send the app to the
|
|
// background with nothing naming what replaced it, so there the id really is
|
|
// unknown and auto-targeting is the honest fallback.
|
|
it("still drops the launched id across a foreground-changing tool that names no app", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name: "button", args: { name: "home" } },
|
|
{ kind: "assert", selector: { text: "General" }, condition: "visible" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
// A flow selector step cannot name a bundleId, so `resolveNativeTargetApp`'s
|
|
// own "Launch or restart the app first" advice is dropped. The auto-target
|
|
// reason that replaces it names no bundle and reports nothing is connected.
|
|
const reason = result.steps[2].reason ?? "";
|
|
expect(reason).toMatch(/no app is connected to native devtools/);
|
|
expect(reason).not.toMatch(/platform binary with library validation/);
|
|
});
|
|
|
|
// The measured half says what is wrong with the app; without this half a flow
|
|
// author is told to restart a tool-server with no mention that a selector
|
|
// needed a hierarchy.
|
|
it("says why the tree was being read, not just what is wrong with the app", async () => {
|
|
// Through the UNPINNED read: the `tool:` step demotes the launch's pin,
|
|
// which is where a launched id that no longer resolves is measured.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "tool", name: "screenshot", args: {} },
|
|
{ kind: "assert", selector: { text: "General" }, condition: "visible" },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => true,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => "unregistered" as const,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const reason = result.steps[2].reason ?? "";
|
|
expect(reason).toMatch(/argent server stop && argent server start --detach/);
|
|
expect(reason).toMatch(/Flows resolve selectors against the full view hierarchy/);
|
|
});
|
|
|
|
// Replacing the whole diagnosis with a literal left every flow test green.
|
|
// `unregistered` is the case that matters — the remedy inverts, and this gate
|
|
// runs right after a launch, so "re-run to relaunch" is the loop one level up.
|
|
it("reports the measured reason when the connection times out", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "launch", app: "com.acme.app" }],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => "unregistered" as const,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
// Leave setImmediate real: the run reads the flow off disk between sleeps,
|
|
// and that I/O needs actual event-loop turns. Pumping one between advances
|
|
// elapses the 1.5 s settle and 8 s connect wait in fake time.
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps[0].status).toBe("error");
|
|
expect(result.steps[0].reason).toContain(
|
|
"argent server stop && argent server start --detach"
|
|
);
|
|
expect(result.steps[0].reason).not.toMatch(/restart-app/);
|
|
expect(result.steps[0].reason?.match(/com\.acme\.app/g)).toHaveLength(1);
|
|
// This gate is the one caller whose wait was bounded: a cold start slower
|
|
// than LAUNCH_TO_VERDICT_MS reads identically to a genuinely unregistered
|
|
// app, so "restarting cannot help" must not be the last word here.
|
|
expect(result.steps[0].reason).toMatch(/cold start/i);
|
|
expect(result.steps[0].reason).toMatch(/re-run the flow/i);
|
|
// The figure sizing that judgement is the whole wait the step performed:
|
|
// the poll reads the live map once before its first sleep, so the
|
|
// post-launch settle counts and quoting the poll alone understates it.
|
|
expect(result.steps[0].reason).toContain(`${LAUNCH_TO_VERDICT_MS} ms`);
|
|
expect(result.steps[0].reason).not.toContain(`${NATIVE_READY_TIMEOUT_MS} ms`);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
// `buildAppStateMessage` is written for a reader who has not launched
|
|
// anything, so "call launch-app (or restart-app)" is the missing step there
|
|
// and the step's own action here — emitted verbatim it hands the flow author
|
|
// back the relaunch that just ran, which re-runs into the identical state.
|
|
//
|
|
// Exercised directly rather than through the runner: it is a pure function of
|
|
// (bundleId, state), and each runner pass costs ~9 s of pumped fake time. The
|
|
// end-to-end test above proves the gate reaches it at all.
|
|
describe("flowLaunchGateReason", () => {
|
|
it.each([
|
|
// `not_running` is the sharpest: a relaunch is what produced the state
|
|
// being reported, so the remedy must not be one.
|
|
["not_running", /exited after launch/i],
|
|
["stale_process", /re-run the flow to launch again/i],
|
|
["indeterminate", /at most once more/i],
|
|
["connecting", /started after the step's own launch/i],
|
|
["unregistered", /cold start/i],
|
|
] as const)("tells %s something the step has not already done or tried", (state, expected) => {
|
|
expect(flowLaunchGateReason("com.acme.app", state)).toMatch(expected);
|
|
});
|
|
|
|
// The one state whose tool-surface remedy IS the launch step's own action.
|
|
// `launch-app` may still appear — the arm suggests starting it by hand to
|
|
// watch it die — but never as the prescribed retry.
|
|
it("does not tell a crash-on-launch app to simply launch itself again", () => {
|
|
const reason = flowLaunchGateReason("com.acme.app", "not_running");
|
|
|
|
expect(reason).not.toMatch(/Call launch-app/);
|
|
expect(reason).not.toMatch(/restart-app/);
|
|
// What it must say instead: the process is gone because it exited, and
|
|
// re-running reproduces exactly that.
|
|
expect(reason).toMatch(/exited after launch/);
|
|
expect(reason).toMatch(/Re-running the flow repeats the same launch/);
|
|
});
|
|
|
|
// The `not.toContain` guards below compare rendered substrings, so they only
|
|
// mean anything while neither figure is a suffix of the other: 500/1500 would
|
|
// make `"1500 ms".includes("500 ms")` true and fail them for the wrong
|
|
// reason. Pin what those guards need, not merely the ordering.
|
|
it("renders a launch-to-verdict spend the connect wait cannot be read into", () => {
|
|
expect(LAUNCH_TO_VERDICT_MS).toBeGreaterThan(NATIVE_READY_TIMEOUT_MS);
|
|
expect(`${LAUNCH_TO_VERDICT_MS} ms`).not.toContain(`${NATIVE_READY_TIMEOUT_MS} ms`);
|
|
});
|
|
|
|
// `stale_process` has two producers, and one carries THIS endpoint — it
|
|
// predates the listener, not the launchd environment. Blaming the
|
|
// environment would contradict the measured text this wraps.
|
|
it("does not blame the launchd environment for a stale process", () => {
|
|
const reason = flowLaunchGateReason("com.acme.app", "stale_process");
|
|
|
|
expect(reason).toMatch(/predates whatever the relaunch would have given it/);
|
|
// One producer carries THIS endpoint and is merely older than the
|
|
// listener, so the environment is not at fault on a first landing. The
|
|
// message may name it only for a REPEAT, which rules that producer out —
|
|
// so the blame must be conditional AND follow the re-run, both asserted.
|
|
const blame = reason.indexOf("launchd environment");
|
|
const rerun = reason.indexOf("re-run the flow");
|
|
expect(rerun).toBeGreaterThanOrEqual(0);
|
|
expect(blame === -1 || blame > rerun).toBe(true);
|
|
// Unconditional on purpose: `if (blame !== -1)` let the whole
|
|
// repeat-landing clause be deleted with the suite green, leaving a reader
|
|
// who lands here twice with "re-run the flow" and no escape. The wording is
|
|
// pinned too — "It lands here twice, so …" states on a FIRST landing what
|
|
// only a second one supports.
|
|
expect(reason).toMatch(/If it lands here twice, the simulator's launchd environment/);
|
|
});
|
|
|
|
it.each(["not_running", "connecting", "unregistered"] as const)(
|
|
"quotes the whole launch-to-verdict spend in the %s remedy",
|
|
(state) => {
|
|
const reason = flowLaunchGateReason("com.acme.app", state);
|
|
|
|
expect(reason).toContain(`${LAUNCH_TO_VERDICT_MS} ms`);
|
|
expect(reason).not.toContain(`${NATIVE_READY_TIMEOUT_MS} ms`);
|
|
}
|
|
);
|
|
});
|
|
|
|
// The connection can land between the final poll and the measurement. Without
|
|
// the `connected` short-circuit, buildAppStateMessage falls off its switch and
|
|
// a healthy run dies with a literal "undefined" as its reason.
|
|
it("passes when the connection lands between the last poll and the measurement", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "launch", app: "com.acme.app" }],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false, // never connects during the poll…
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => "connected" as const, // …but has by the measurement
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps[0].status).toBe("pass");
|
|
expect(result.ok).toBe(true);
|
|
// Without the short-circuit the step errors with a literal "undefined" —
|
|
// buildAppStateMessage has no `connected` arm to fall to.
|
|
expect(result.steps[0].reason ?? "").not.toMatch(/undefined/);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
// A rejected measurement must not propagate: `runLaunch` has no try/catch, so
|
|
// it would escape `execute()` and the run would return no step reports at all
|
|
// instead of a structured failure.
|
|
it("keeps a rejected measurement inside the step report", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "echo", message: "after" },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => {
|
|
throw new Error("Invalid device: UDID");
|
|
},
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"launch:error",
|
|
"echo:skip",
|
|
]);
|
|
expect(result.steps[0].reason).toContain("could not be inspected");
|
|
expect(result.steps[0].reason).not.toMatch(/Invalid device/);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
});
|
|
|
|
describe("device binding (portability)", () => {
|
|
const reg = (props: Record<string, unknown>) =>
|
|
({ getTool: () => ({ inputSchema: { properties: props } }) }) as unknown as Registry;
|
|
|
|
it("the resolved device wins over a stale stored udid", () => {
|
|
const out = bindDeviceArgs(reg({ udid: {}, x: {}, y: {} }), "gesture-tap", "RESOLVED", {
|
|
udid: "STALE",
|
|
x: 0.5,
|
|
y: 0.5,
|
|
});
|
|
expect(out).toEqual({ udid: "RESOLVED", x: 0.5, y: 0.5 });
|
|
});
|
|
|
|
it("drops a device id entirely for a tool that doesn't declare one", () => {
|
|
const out = bindDeviceArgs(reg({ foo: {} }), "x", "R", { device_id: "STALE", foo: 1 });
|
|
expect(out).toEqual({ foo: 1 });
|
|
});
|
|
|
|
it("stripDeviceKeys removes udid / device_id / device, leaving other args untouched", () => {
|
|
expect(stripDeviceKeys({ udid: "A", device_id: "B", device: "C", x: 1 })).toEqual({ x: 1 });
|
|
});
|
|
|
|
it("stripDeviceKeys KEEPS a `devices` scope, because dropping it changes the step's meaning", () => {
|
|
// A target is stripped so the flow points at no device. A scope is not:
|
|
// `stop-all-simulator-servers` with no `devices` is the machine-wide sweep,
|
|
// so stripping it would record a correctly scoped teardown as a bare step
|
|
// that reaps every device on the machine when hand-run from the YAML — the
|
|
// manual-execution strategy the create-flow skill documents. Replay rebinds
|
|
// it either way (see bindDeviceArgs below).
|
|
expect(stripDeviceKeys({ udid: "A", devices: ["D", "E"], x: 1 })).toEqual({
|
|
devices: ["D", "E"],
|
|
x: 1,
|
|
});
|
|
});
|
|
|
|
it("bindDeviceArgs keeps a recorded scope when the run resolved NO device", () => {
|
|
// A cleanup flow resolves no device when none is unambiguous. Dropping the
|
|
// recorded scope there would widen the teardown from the devices the
|
|
// recording named to every device on the machine — the one direction that
|
|
// costs another agent their session. There is no run target to override.
|
|
expect(
|
|
bindDeviceArgs(reg({ devices: {} }), "stop-all-simulator-servers", "", {
|
|
devices: ["RECORDED"],
|
|
})
|
|
).toEqual({ devices: ["RECORDED"] });
|
|
});
|
|
|
|
it("bindDeviceArgs never forwards a scope to a tool that does not declare it", () => {
|
|
// The schema-blind strip's job: a `.strict()` schema would reject the call.
|
|
expect(
|
|
bindDeviceArgs(reg({ port: {} }), "stop-metro", "RESOLVED", {
|
|
devices: ["RECORDED"],
|
|
port: 8081,
|
|
})
|
|
).toEqual({ port: 8081 });
|
|
});
|
|
|
|
it("rebinds a nested flow-execute onto the run device (issue #607)", () => {
|
|
// `flow-execute`'s own device parameter is named `device`, so before it was
|
|
// a bind key a recorded nested step kept the id it was recorded on and the
|
|
// sub-run drove that device instead of the one the replay was given.
|
|
const out = bindDeviceArgs(
|
|
reg({ name: {}, project_root: {}, device: {} }),
|
|
"flow-execute",
|
|
"RESOLVED",
|
|
{ name: "b", project_root: "/p", device: "STALE" }
|
|
);
|
|
expect(out).toEqual({ name: "b", project_root: "/p", device: "RESOLVED" });
|
|
});
|
|
|
|
it("leaves `platform` alone", () => {
|
|
// Deliberate, and pinned here so a later "symmetry" edit fails loudly. The
|
|
// strip is schema-blind, and `platform` is not device-specific on every tool
|
|
// — react-profiler-analyze declares its own — so stripping it would silently
|
|
// retarget an unrelated recorded step. It is also inert once `device` is
|
|
// bound, because device resolution returns before it is ever read.
|
|
expect(stripDeviceKeys({ platform: "android", x: 1 })).toEqual({ platform: "android", x: 1 });
|
|
});
|
|
|
|
it("injects devices: [resolvedId] for a tool that declares it in its schema", () => {
|
|
// stop-all-simulator-servers' `devices` is a scope, not a single-device
|
|
// target, but it names the recording host's device ids the same way `udid`
|
|
// does — so it gets the same schema-aware rebind, as a one-element list.
|
|
const out = bindDeviceArgs(reg({ devices: {} }), "stop-all-simulator-servers", "RESOLVED", {});
|
|
expect(out).toEqual({ devices: ["RESOLVED"] });
|
|
});
|
|
|
|
it("does not invent a devices key for a tool that doesn't declare it", () => {
|
|
const out = bindDeviceArgs(reg({ foo: {} }), "x", "RESOLVED", { foo: 1 });
|
|
expect(out).toEqual({ foo: 1 });
|
|
expect(out).not.toHaveProperty("devices");
|
|
});
|
|
|
|
it("replaces a stale recorded devices list when the caller NAMED the run device", () => {
|
|
// An explicit `device` is the caller saying which device this run is about,
|
|
// so retargeting the teardown at it is what they asked for — and a flow
|
|
// recorded on one host must not carry that host's ids forward.
|
|
const out = bindDeviceArgs(
|
|
reg({ devices: {} }),
|
|
"stop-all-simulator-servers",
|
|
"RESOLVED",
|
|
{ devices: ["OLD-HOST-ID", "OTHER"] },
|
|
true
|
|
);
|
|
expect(out).toEqual({ devices: ["RESOLVED"] });
|
|
});
|
|
|
|
it("keeps a recorded scope when the run device was only auto-detected", () => {
|
|
// The destructive direction: the flow named one device, exactly one other
|
|
// happens to be booted, and replay would reap THAT one — a device nobody in
|
|
// this run ever named, quite possibly another agent's. This is the
|
|
// cross-agent teardown the `devices` scope exists to prevent, so the
|
|
// recorded ids stand; on another host they reap nothing and come back in
|
|
// `unmatched`, which is the safe direction and a legible one.
|
|
const out = bindDeviceArgs(reg({ devices: {} }), "stop-all-simulator-servers", "AUTO", {
|
|
devices: ["RECORDED-HOST"],
|
|
});
|
|
expect(out).toEqual({ devices: ["RECORDED-HOST"] });
|
|
});
|
|
|
|
it("still narrows an UNSCOPED recorded sweep onto an auto-detected device", () => {
|
|
// Nothing recorded means the step is the machine-wide sweep, so binding can
|
|
// only narrow it. That is why a cleanup flow resolves a device at all.
|
|
const out = bindDeviceArgs(reg({ devices: {} }), "stop-all-simulator-servers", "AUTO", {});
|
|
expect(out).toEqual({ devices: ["AUTO"] });
|
|
});
|
|
|
|
it("binds a scalar and a list device key together when a tool declares both", () => {
|
|
const out = bindDeviceArgs(
|
|
reg({ udid: {}, devices: {} }),
|
|
"hypothetical-tool",
|
|
"RESOLVED",
|
|
{ udid: "STALE", devices: ["OLD"] },
|
|
true
|
|
);
|
|
expect(out).toEqual({ udid: "RESOLVED", devices: ["RESOLVED"] });
|
|
});
|
|
|
|
it("rebinds the TARGET but not the recorded SCOPE on an auto-detected device", () => {
|
|
// The two keys part company here: a stale `udid` must never survive (the
|
|
// step would drive the wrong device), while a stale `devices` must never be
|
|
// retargeted (the step would destroy the wrong device).
|
|
const out = bindDeviceArgs(reg({ udid: {}, devices: {} }), "hypothetical-tool", "AUTO", {
|
|
udid: "STALE",
|
|
devices: ["OLD"],
|
|
});
|
|
expect(out).toEqual({ udid: "AUTO", devices: ["OLD"] });
|
|
});
|
|
});
|
|
|
|
describe("flow validation", () => {
|
|
it("rejects an e2e flow that declares executionPrerequisite", () => {
|
|
expect(() =>
|
|
parseFlow("executionPrerequisite: nope\nsteps:\n - launch: com.acme.app\n")
|
|
).toThrow(/must not declare executionPrerequisite/i);
|
|
});
|
|
|
|
it("a leading echo does not hide the launch step from the e2e check", () => {
|
|
expect(() =>
|
|
parseFlow(
|
|
"executionPrerequisite: nope\nsteps:\n - echo: starting\n - launch: com.acme.app\n"
|
|
)
|
|
).toThrow(/must not declare executionPrerequisite/i);
|
|
});
|
|
|
|
it("a leading script does not hide the launch step from the e2e check", () => {
|
|
expect(() =>
|
|
parseFlow(
|
|
"executionPrerequisite: nope\n" +
|
|
"steps:\n - script: { path: seed.mjs }\n - launch: com.acme.app\n"
|
|
)
|
|
).toThrow(/must not declare executionPrerequisite/i);
|
|
});
|
|
|
|
it("names the rule that classified the flow, not step 1", () => {
|
|
// The two tests above match only the half of the sentence the
|
|
// classification change left true. A launch admitted from BEHIND an
|
|
// `echo:`/`script:` is still the launch this refusal is about, so the
|
|
// sentence must not claim the flow "starts with" one, nor send the author
|
|
// after a "leading launch" they did not write first.
|
|
let message = "";
|
|
try {
|
|
parseFlow(
|
|
"executionPrerequisite: nope\n" +
|
|
"steps:\n - script: { path: seed.mjs }\n - launch: com.acme.app\n"
|
|
);
|
|
} catch (err) {
|
|
message = err instanceof Error ? err.message : String(err);
|
|
}
|
|
expect(message).toContain("first step other than `echo:`/`script:` is a `launch`");
|
|
expect(message).not.toMatch(/starts with a launch step|leading launch/i);
|
|
});
|
|
|
|
it("rejects a path-unsafe snapshot name (no traversal into baseline path)", () => {
|
|
expect(() => parseFlow("steps:\n - snapshot:\n name: ../../etc/evil\n")).toThrow(
|
|
/snapshot name/i
|
|
);
|
|
});
|
|
|
|
it("completes a bare run: name to <name>.yaml", () => {
|
|
// The compatibility spelling for flows written when a run: target was a
|
|
// saved-flow name. Completed at PARSE, so one spelling reaches the runner.
|
|
expect(parseFlow("steps:\n - run: login\n").steps[0]).toEqual({
|
|
kind: "run",
|
|
flow: "login.yaml",
|
|
});
|
|
});
|
|
|
|
it("completes an extensionless run: path, keeping its directory prefix", () => {
|
|
for (const [written, completed] of [
|
|
["shared/login", "shared/login.yaml"],
|
|
["../shared/login", "../shared/login.yaml"],
|
|
["./login", "./login.yaml"],
|
|
]) {
|
|
expect(parseFlow(`steps:\n - run: ${written}\n`).steps[0], written).toEqual({
|
|
kind: "run",
|
|
flow: completed,
|
|
});
|
|
}
|
|
});
|
|
|
|
it("does not complete a run: target the extension cannot rescue", () => {
|
|
// basename() strips a trailing slash, so a completion driven by the
|
|
// SUPPLIED basename would turn "shared/" into the unopenable "shared/.yaml"
|
|
// instead of reporting a target that names no file.
|
|
expect(() => parseFlow("steps:\n - run: shared/\n")).toThrow(/must end in \.yaml/);
|
|
expect(() => parseFlow("steps:\n - run: shared/\n")).not.toThrow(/\.yaml\.yaml/);
|
|
// A wrong extension is a typo to report, not a stem to extend: completing
|
|
// it would run "login.yml.yaml" and bury the mistake.
|
|
expect(() => parseFlow("steps:\n - run: login.yml\n")).toThrow(/must end in \.yaml/);
|
|
// Dots are outside the flow-name charset, so a version-ish stem stays an
|
|
// error rather than becoming "v1.2.yaml".
|
|
expect(() => parseFlow("steps:\n - run: v1.2\n")).toThrow(/must end in \.yaml/);
|
|
});
|
|
|
|
it("reports a missing target instead of a null.yaml completion for a valueless run:", () => {
|
|
for (const body of ["", " ~", " null"]) {
|
|
expect(() => parseFlow(`steps:\n - run:${body}\n`)).toThrow(/`run` has no target/);
|
|
expect(() => parseFlow(`steps:\n - run:${body}\n`)).not.toThrow(/null\.yaml/);
|
|
}
|
|
});
|
|
|
|
it("rejects a non-string run: scalar instead of migrating it to a filename", () => {
|
|
expect(() => parseFlow("steps:\n - run: true\n")).toThrow(/must be a YAML path string/);
|
|
expect(() => parseFlow("steps:\n - run: true\n")).not.toThrow(/true\.yaml/);
|
|
expect(() => parseFlow("steps:\n - run: 123\n")).toThrow(/must be a YAML path string/);
|
|
expect(() => parseFlow("steps:\n - run: 123\n")).not.toThrow(/123\.yaml/);
|
|
});
|
|
|
|
it("rejects a run: path without the .yaml extension", () => {
|
|
expect(() => parseFlow("steps:\n - run: flows/login.yml\n")).toThrow(/must end in \.yaml/);
|
|
// A correctly-typed empty string is a string, so it still reaches the
|
|
// extension check — this boundary is what keeps the non-string rejection
|
|
// above targeted at YAML scalars rather than at "no path here" generally.
|
|
expect(() => parseFlow('steps:\n - run: ""\n')).toThrow(/must end in \.yaml/);
|
|
});
|
|
|
|
it("names the lowercase requirement when only the run: extension's case is wrong", () => {
|
|
expect(() => parseFlow("steps:\n - run: shared/Login.YAML\n")).toThrow(
|
|
/lowercase \.yaml extension/
|
|
);
|
|
});
|
|
|
|
it("rejects an absolute run: path", () => {
|
|
expect(() => parseFlow("steps:\n - run: /anywhere/login.yaml\n")).toThrow(/must be relative/);
|
|
expect(() => parseFlow("steps:\n - run: C:/anywhere/login.yaml\n")).toThrow(
|
|
/must be relative/
|
|
);
|
|
// Drive-relative: not isAbsolute, but resolves against the drive's cwd.
|
|
expect(() => parseFlow("steps:\n - run: C:foo/login.yaml\n")).toThrow(/must be relative/);
|
|
});
|
|
|
|
it("rejects a run: path whose filename is not a safe flow name", () => {
|
|
expect(() => parseFlow("steps:\n - run: ../we!rd.yaml\n")).toThrow(/letters, digits/);
|
|
});
|
|
|
|
it("cites a pattern a rejected .yaml filename could actually match", () => {
|
|
// `my.flow.yaml` IS a .yaml file, so citing the dot-free flow-name pattern
|
|
// would demand a regex forbidding the extension the same check requires.
|
|
let message = "";
|
|
try {
|
|
parseFlow("steps:\n - run: my.flow.yaml\n");
|
|
} catch (err) {
|
|
message = err instanceof Error ? err.message : String(err);
|
|
}
|
|
expect(message).toContain(String(FLOW_FILE_NAME_PATTERN));
|
|
expect(message).not.toContain(String(FLOW_NAME_PATTERN));
|
|
expect(message).toMatch(/letters, digits, underscore, hyphen before the \.yaml/);
|
|
});
|
|
|
|
it("rejects backslashes in a run: path", () => {
|
|
expect(() => parseFlow("steps:\n - run: frag\\login.yaml\n")).toThrow(/forward slashes/);
|
|
});
|
|
|
|
it("round-trips the new step kinds through YAML", () => {
|
|
const flow = {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch" as const, app: "com.acme.app" },
|
|
// Text-only selectors serialize to bare strings, which parse back loose.
|
|
{ kind: "tap" as const, selector: { text: "Login", loose: true } },
|
|
{ kind: "tap" as const, x: 0.5, y: 0.57 },
|
|
{ kind: "type" as const, into: { identifier: "email" }, text: "a@b.com" },
|
|
{
|
|
kind: "assert" as const,
|
|
condition: "visible" as const,
|
|
selector: { text: "Welcome", loose: true },
|
|
},
|
|
{ kind: "snapshot" as const, name: "home", maxMismatch: 0.5 },
|
|
{ kind: "run" as const, flow: "../shared/login.yaml" },
|
|
// Mid-flow relaunch with a per-platform map.
|
|
{ kind: "launch" as const, app: { ios: "com.acme.app", android: "com.acme.android" } },
|
|
],
|
|
};
|
|
const parsed = parseFlow(serializeFlow(flow));
|
|
expect(parsed.steps).toEqual(flow.steps);
|
|
});
|
|
|
|
it("rejects a launch step with an invalid body", () => {
|
|
// An unrecognized platform key is named (strict unknown-key rejection)…
|
|
expect(() => parseFlow("steps:\n - launch: { web: foo }\n")).toThrow(
|
|
/launch has unknown key `web`/
|
|
);
|
|
// …while a non-map, non-string body still gets the shape error.
|
|
expect(() => parseFlow("steps:\n - launch: 42\n")).toThrow(/launch needs an app id/i);
|
|
});
|
|
});
|