mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
5ac6805040
this is urgent
---
## Problem
Every `native-*` tool can get stuck telling the agent to do something
that will never work.
I hit this during an end-to-end pass over 0.17.0. After a mid-session
simulator reboot, `native-devtools-status` reported:
```json
{ "envSetup": true, "appRunning": true, "connected": false,
"requiresRestart": true, "nextLaunchWillBeInjected": true, "injectable": true }
```
and every `native-*` tool returned `restart_required` with "Call
restart-app then retry." Following that advice never helped, and the
message offered nothing else. `flow-execute` was blocked too, since its
`launch` step needs the connection, and flow recording silently degraded
a portable `tap: { selector }` to raw coordinates when it couldn't reach
native devtools.
The app was injected the whole time. All three dylibs were mapped into
the process and the simulator log showed a successful dial on every
launch, while the tool-server held the listening end of that exact
socket and still reported `connected: false`. `argent server stop &&
argent server start` cleared it instantly.
## The real defect is the state, not the wording
`requiresAppRestart` returned `true` for every app without a live
connection:
```ts
async requiresAppRestart(bundleId) {
if (connections.has(bundleId)) return false;
await reverifyEnv();
return true; // <- asserted, never measured
}
```
That is not a measurement. `restart_required` was a constant wearing a
state's clothes, and the advice built on it stays literally true
forever, so an agent obeying it restarts the app indefinitely.
Softening the message would have left that intact. This PR measures the
state instead.
## Fix — derive it from the running process
`restart-app` relaunches into the simulator's **current** launchd
environment, so it can only ever help a process launched under different
terms than a fresh launch would get. That is answerable: read the
process. `launchctl list` already gives its pid; `ps eww -p <pid> -o
etime=,command=` gives the environment it was exec'd with and its age.
Simulator apps are ordinary host processes owned by the same user, so
both are readable.
`appConnectionState` (replacing `requiresAppRestart`) returns:
| state | meaning | remedy |
| --- | --- | --- |
| `connected` | devtools are live | — |
| `not_running` | no process to read | `launch-app` |
| `stale_process` | no bootstrap dylib, a previous run's endpoint, or
older than this service's listener | `restart-app` |
| `unregistered` | injected against **this** endpoint, started after the
listener came up, still silent | restart the **tool-server** |
| `indeterminate` | process could not be read (ios-remote, unreadable
`ps`) | `restart-app`, with the loop warning |
Only `not_running` / `stale_process` / `indeterminate` report
`restart_required`. `unregistered` gets a new `service_stale` precheck
status that names the tool-server restart and explicitly says not to
relaunch the app — the launch a restart would perform has demonstrably
already happened.
The listener-age comparison is load-bearing: after a tool-server restart
the unix path is identical (it is derived from the udid) but the inode
is new, so an injected process older than the listener still needs
relaunching. Without that check this would send people off to restart a
tool-server they had just restarted — a new loop in place of the old
one. Both grace comparisons lean away from `unregistered` - the first
towards `stale_process`, the second towards `indeterminate`, and both of
those ask for a relaunch - so an uncertain read costs at most one wasted
`restart-app`.
Call sites follow the same rule rather than each inventing guidance:
- `describe` withholds `should_restart` for `unregistered` (setting it
would rebuild the restart-app → describe loop through a live service
instead of a system app) and carries the diagnosis as a `hint`, which
still marks the empty read untrustworthy for `await-ui-element`.
- `flow-run` raises the measured diagnosis instead of a blanket
"relaunch it" — both when a flow
resolves a selector against the full hierarchy, and at the `launch`
step's readiness gate, which
previously polled for the connection and then guessed at "a stale or
duplicate argent server".
- `native-devtools-status` reports `state`, and `requiresRestart` is now
false for `unregistered`.
- The six `native-*` tools type their block as the shared
`NativeDevtoolsPrecheckBlock` instead of restating it, so a new precheck
outcome cannot reach an agent through a tool that never mentions it.
## Scope
Why the service can stop registering a connection it is listening for is
still #561 and is not fixed here. What changes is that the state is now
measured, so that defect reports itself accurately instead of
masquerading as an app that needs restarting.
## Verification
**E2E, real iOS 18.6 simulator, Bluesky (RN 0.81.5).** Two tool-servers
were raced onto the same per-udid socket — the #561 shape, where the app
is genuinely injected and connected, just to the newer instance. Both
were then queried on the same app at the same instant:
| | `connected` | `requiresRestart` | `state` | `native-describe-screen`
|
| --- | --- | --- | --- | --- |
| main, stale instance | false | **true** | — | `restart_required` —
"Call restart-app then retry." |
| this branch, stale instance | false | **false** | `unregistered` |
`service_stale` — "Restarting the app cannot change that … restart the
tool-server" |
| healthy instance | true | false | `connected` | `ok`, 44 elements |
Restarting the app in main's row lands on the newer instance again,
leaving the queried one stale forever. That is the loop, reproduced on
demand.
The other states were walked on the same simulator: a pre-existing
process against a fresh listener → `stale_process` (`requiresRestart:
true`); `restart-app` → `connected`; terminate → `not_running` ("Call
launch-app (or restart-app)").
**Unit.** `native-devtools-app-state.test.ts` drives the real blueprint
factory with a stubbed process table and covers all five states plus the
`etime` and env-token parsers. Mutation-checked — dropping the
listener-age comparison, the endpoint match, or the connect grace each
fails exactly one test and nothing else.
**Suite.** `vitest run` in `packages/tool-server`: 296 files, 3113
passed, 1 skipped. `tsc --build` across the workspace, `tsc --noEmit -p
tsconfig.test.json`, `eslint` and `prettier --check` on every changed
file: clean. `origin/main` merged in first.
## Review round
A review pass over the derivation itself, with each finding reproduced
before it was acted on.
**`describe` computed the diagnosis and threw it away.**
`buildAppStateMessage` ran for every
unconnected state, but only `unregistered` carried it out; the rest
returned a bare
`should_restart: true`. That boolean reaches an agent as an undocumented
JSON field — `describe`'s
own description never mentions it — and the single place it is rendered
as prose is
`await-ui-element`'s timeout note, which spells it *"call restart-app
and retry"*. So the two states
whose message carries the escape hatch lost it exactly where it was
needed. `indeterminate` is the
sharp end: on `ios-remote`, whose app processes are out of reach of the
local process table, it is
the **only** reachable unconnected state, so that path had no exit at
all. The diagnosis now rides
out as a `hint` for every state; `should_restart` still marks only the
states a relaunch fixes.
**`native-devtools-status` measured the same device twice.**
`isAppRunning` ran its own
`launchctl list`, and `appConnectionState` ran another one after
re-verifying the launchd env — so
the two snapshots sat seconds apart and could contradict each other
(`appRunning: true` beside
`state: "not_running"`), with `requiresRestart` gated on the staler of
the pair. Both are derived
from one measurement now; only `indeterminate`, which genuinely leaves
running-ness unanswered,
pays for a second probe.
**Coverage.** Mutation testing found the 3 s grace term entirely
unpinned — the fixtures sat 200x
away from the boundary on either side, so both dropping the term and
weakening `>=` to `>` kept the
suite green while flipping a relaunchable process to `unregistered`, the
one verdict the file exists
to withhold. Also unpinned: the `reverifyEnv` call (swapping it for the
latching `ensureEnvReady`
the comment explicitly forbids passed the whole suite) and an
unparseable `ps` age staying "no
evidence" rather than becoming a measurement. All three are now covered,
and each new test was
confirmed to fail on the un-fixed code.
**Verified against the real thing.** The `launchctl` and `ps` contracts
were re-checked on an
iOS 18.6 simulator rather than reasoned about: row layout and pid
column, one-line `ps eww` output
with the argent tokens present in a live app's environment, `etime`
matched against `lstart` to the
second, and the four `etime` format strings read out of the `ps` binary.
Two corrections came out of
it — the environment can reach `kern.argmax`, which is exactly Node's
default `maxBuffer`, so the
probe now sets the same 16 MiB its siblings use; and `ps` suppresses the
environment of a
SIP-protected platform binary, which the docstring's "never as 'not
injected'" promise did not
account for (unreachable here, since `com.apple.*` is rejected as
non-injectable upstream, but the
promise now says so).
**Also corrected:** the `argent-device-interact` skill offered
`restart-app` as the only remedy for
a missing injection; the dylib-platform check in
`download-native-binaries.sh` and a
`requiresAppRestart` reference in `boot-device.ts` described states that
no longer exist; and three
tool descriptions still said *"Fails if native devtools are not
connected"* directly beside the new
sentence explaining that this case returns a status rather than failing.
**Known and deliberately left** (both pre-existing, neither introduced
here):
`describe` with no `bundleId` auto-targets from the connected list, so
an unregistered app throws
`NATIVE_TARGET_NO_CONNECTED_APPS` before any state is measured and the
message is swallowed by the
fallback's catch — surfacing it would newly mark those empty reads blind
for `await-ui-element`'s
`hidden` condition, which wants its own verification pass. And a dylib
that is inserted but silently
skipped by dyld (a platform mismatch) reads as injected-but-silent, so
`service_stale` → tool-server
restart → `stale_process` → `restart-app` is a two-step cycle;
distinguishing it needs a load
confirmation the process table cannot give.
## Review round 2
A second pass over the derivation, run as four independent sweeps
(claims-vs-code, nearest-twin
and reachability, non-happy paths and inputs, and absence). Three of the
four independently landed
on the same defect.
**The flow launch gate was the one caller that never asked whether the
app could be injected at
all.** Every other consumer of `appConnectionState` gates `com.apple.*`
first - the precheck
throws, `describe` returns its terminal hint, `native-devtools-status`
short-circuits,
`flow-ios-tree` only ever auto-resolves a connected app.
`waitForNativeDevtools` measured the raw
launch id. That matters because the launchd environment carrying the
bootstrap dylib is
simulator-wide, so a system app's process inherits the very tokens the
measurement reads.
Confirmed on an iOS 18.6 simulator: `com.apple.Preferences`, launched
under argent, reports
```
71940 0 UIKitApplication:com.apple.Preferences[9a63][rb-legacy]
etime=00:03
DYLD_INSERT_LIBRARIES=.../libArgentInjectionBootstrap.dylib
NATIVE_DEVTOOLS_IOS_CDP_SOCKET=/tmp/argent-nd-39B9C547.sock
```
Run through the shipped algorithm that is `processCarriesInjection:
true` against an older
listener, i.e. **`unregistered`** - so a flow with an `ios:` launch
entry of `com.apple.Preferences`
failed its launch step with *"restart the tool-server"*, for an app no
tool-server restart can ever
help. Library validation means it can never connect, so the gate now
says that terminally.
**`describe` dropped the entire diagnosis when the probe rejected.**
`appConnectionState` re-applies
the launchd env before it can answer anything, so it throws outright on
a sim that goes away
mid-call; that landed in the native fallback's outer `catch` and
returned an empty tree with no
hint. An agent saw "nothing on screen" rather than "could not be read",
and `await-ui-element` had
nothing to append to its timeout note. The two other consumers of the
same call already degraded to
`indeterminate` instead - this one now matches them.
**`native-devtools-status` re-ran the probe that had just failed.** One
route into `indeterminate`
is `inspectRunningApp` rejecting, i.e. `simctl spawn <udid> launchctl
list` failing. The
running-ness fallback then issues that same command unguarded, so a
shut-down simulator surfaced a
raw subprocess error rather than the structured `init_failed` guidance -
which the branch 40 lines
above it handles correctly, for exactly this reason.
**`native-devtools-status` was the only surface withholding the loop
escape.** Every other consumer
carries `buildAppStateMessage`, whose `indeterminate` text ends *"do not
keep restarting the app;
restart the tool-server"*. This tool returned booleans only, and
`requiresRestart: true` is the
whole of what an `indeterminate` app tells you. On ios-remote, where no
app process can ever be
inspected, that is the only state a running app reaches - so the tool
whose stated job is verifying
readiness answered with the restart loop and nothing else. It now
returns `message`.
**Also fixed:** both probes log their own failure (a broken probe
degrades every app to
`indeterminate`, indistinguishable at the tool surface from a genuinely
uninspectable one - the
sibling `ps` probe in `vega-process.ts` logs for the same reason); the
gate skips its
several-second measurement once the run is cancelled; and the gate no
longer repeats the bundle id
its caller's prefix already names.
**Prose that had drifted from the code:** `not_running` is reachable on
ios-remote too, so
`indeterminate` is only the sole unconnected state a *running* app
reaches there; the second grace
comparison leans to `indeterminate`, not `stale_process`; four of the
five states settle
running-ness, not four besides `not_running`; a null pid means a row
that did not parse, not a job
launchd reported as having no process (measured: a `UIKitApplication`
row is removed outright when
the app exits, and `-` appears only on daemon labels);
`waitForNativeDevtools`'s documented return
contract described the branch the change had rewritten; and a rationale
comment was attached to the
test above the one it described.
**Coverage.** The gate's measured reason had none at all - replacing the
whole diagnosis with a
literal left all 542 flow tests green. Eleven mutations were run against
the fixes and every one is
now killed: the injectability guard, the measured reason, the doubled
bundle id, `describe`'s reject
guard, `describe`'s `unregistered` hint merge, the `message` field, the
precheck fallback, the
`appRunning` guard on `requiresRestart`, a state renamed in the
description only, the connect grace
widened from above (the old fixtures all sat 10x clear of it, so it
could have grown to 25 s
unnoticed), and the `indeterminate` guidance line.
---
## Review round 3 — external review findings
Six findings from @hubgan's E2E review, each reproduced before it was
acted on. Fixed in b9d17437
and 3eb59ef7. **This round supersedes several claims made above — they
are listed at the end.**
**The launch gate refused every `launch:` of an Apple system app.** A
coordinate-only flow gave
`["launch:error","tool:skip"]` with `resolveService` called 0 times —
the guard short-circuited
before any measurement — and the reason it emitted said *"Drive it by
coordinate instead"*, from
the gate that had just killed a purely coordinate-driven flow.
`argent-create-flow`'s worked
example (SKILL.md:171, :199-209) is exactly that shape.
The premise underneath it did not survive checking. #453, which
motivated `isInjectableBundleId`,
recorded `connected: false` for `com.apple.Preferences` on an iOS 26.5
sim; the review recorded
`connected: true`, with both dylibs mapped, on 18.5. So injection into a
simulator system app
appears to be runtime-dependent rather than categorically impossible,
and neither reading is
checkable from CI. The gate therefore no longer decides: it polls
exactly as before — where the app
does connect, nothing changes — and only on timeout does a
non-injectable id return "no failure"
instead of a reason. The launch step's job is to start the app; it
succeeded. **Whether
`isInjectableBundleId`'s blanket prefix rule is correct at all is now an
open question, and a
separate one from this PR.**
**The flow tree source could not report it either, because its
measurement was unreachable.**
`resolveNativeTargetApp(api, undefined)` draws candidates from
`listConnectedBundleIds()` — the same
map `appConnectionState` reads to answer `connected` — so a
non-connected app threw the stock
auto-target error first, whose "Launch or restart the app first" is the
loop this PR exists to
break. Driving the real factory: `appConnectionState === "unregistered"`
and
`listConnectedBundleIds() === []` at the same instant. The race between
the two is closed, not
narrow: no macrotask boundary separates them, while the socket `close`
handler is one. The branch's
own test passed only because its stub let the two accessors disagree.
The flow now threads the id
its `launch` step named — which does not come from that map — through
`ActionEnv` → `fetchFlowTree`
→ `queryFullHierarchyTree`; the unreachable branch is gone.
**The gate emitted `buildAppStateMessage` verbatim for every state but
`unregistered`.** Those are
written for the native-* tool surface, whose reader has not launched
anything, so a crash-on-launch
app was answered with "Call launch-app (or restart-app) then retry" —
the action the step had taken
9.5 s earlier. `flowLaunchGateReason` now rewrites each state for a
caller that has just launched,
exhaustively, so a state added later cannot inherit the tool-surface
remedy.
**`describe` without a `bundleId` — the default form — lost the whole
diagnosis.** Auto-targeting
throws before anything is measured and the outer catch returned the tree
bare, so
`await-ui-element`'s blind-read guard (which keys off exactly
`hint`/`should_restart`) let a
`hidden` wait release in 0 ms against an element still on screen, where
the same call *with* a
`bundleId` correctly refused. An empty read the hierarchy could not
corroborate now says so. A
device with **no** native-devtools service at all is deliberately left
unqualified — that is an
absence of a second opinion, not a failed attempt at one, and marking it
blind broke a real
`await-ui-element` case.
**Three of the six `native-*` descriptions named the terminal system-app
rejection and three did
not**, though all six throw the identical error (verified: one
byte-identical message across all
six). The clause is inlined verbatim in all six —
`scripts/extract-tools.mjs` accepts only a single
plain literal and silently drops anything else from the SpiderShield
scan.
**`not_running` cannot tell "stopped" from "not installed"** — both
simply lack a
`UIKitApplication:<id>` row — and for the second reading its only remedy
is refused outright by
`simctl launch`. It now names both readings instead of asserting the one
it cannot see. A real
probe was considered and not taken: `get_app_container` needs a booted
sim and ios-remote has no
such verb, so it would answer `undefined` on exactly the host where
`indeterminate` is already the
only reachable state.
### Claims above that this round supersedes
- *"Library validation means it can never connect, so the gate now says
that terminally."* — the
gate now lets system-app launches through; the premise is contested (see
above).
- *"The flow launch gate was the one caller that never asked whether the
app could be injected at
all"* and *"`flow-ios-tree` only ever auto-resolves a connected app"* —
both inverted here.
- *"Known and deliberately left"* — the `describe` no-`bundleId` path
was closed, with tests, so
that paragraph now over-counts by one.
- The suite figures in **Verification** and **Review round 2** are
pre-round-3 counts.
---
## Review round 4 — reviewing round 3
Three lenses over 856efd8e's two parent commits (claims-vs-code,
non-happy
paths + reachability, absence + mutation). Round 3 did not survive
intact, and
what follows is what was wrong with it.
**Three statements were false in prose an agent acts on.** `connecting`
requires
a process inside the 3 s connect grace, while this gate takes its
verdict 9.5 s
after launching the app — so its remedy claimed the step had launched
the very
process it was measuring, next to a base message reading "it launched
moments
ago". The figure that arm and `not_running` quoted was the connect wait
alone,
omitting the 1.5 s post-launch settle that precedes it
(`LAUNCH_TO_VERDICT_MS`
now names the whole spend). And `stale_process` blamed the launchd
environment,
which is false for the producer reached only when
`processCarriesInjection` is
**true** — that process carries this endpoint and is merely older than
the
listener. The measured text it wraps names both producers, so the two
contradicted each other in one message.
**A `hidden` wait still released in 1 ms.** Round 3 carved out a failed
native-devtools resolution as "no service on this device at all". The
blueprint
is registered unconditionally, so on iOS that rejection only ever means
the
service failed to come up — a socket bind losing to a concurrent
same-udid
server, which this repo documents as the dominant crash source in 0.16.0
telemetry. Reproduced: `describe` → no hint, then `await-ui-element
hidden` →
`{"success":true,"elapsed":1}`. It is now as unexplained as its
siblings.
**The failure-code discrimination was defeatable.** The flow tree source
classified auto-targeting's throw by reading a code off a module-local
`Symbol`,
so a duplicate `@argent/registry` instance would read it as absent and
fall back
to the exact message the change replaces. It tests the empty connected
list
directly now — the same condition, structurally, with no symbol
involved. The
`connected` arm added in round 3 was unreachable for the same reason the
branch
it replaced was, and is gone.
**Coverage the mutation lens found missing.** Three of the four sites
threading
the launched bundle id were unpinned — including the `settleTree` read
every
`tap`/`type`/`scroll-to` uses, so an action step could have silently
lost the
terminal reason with the suite green. Also unpinned: `unexplainedHint`
dropping
a pre-existing degraded hint (where its exact twin fifty lines up **is**
pinned), the tree source's "Flows resolve selectors" half, and both of
the
gate's withholdings.
**Test cost.** The four per-state gate tests drove the whole runner in
pumped
fake time, ~9 s each, which is what pushed this file past vitest's
default under
parallel load. The mapping is a pure function and is tested as one; one
end-to-end test still proves the gate reaches it. 46 s → 16 s, with more
tests
than before. Widening the fake-timer step was tried and is worse (52 s)
— the
pumps are what let the run's disk I/O proceed.
**Also corrected:** comments the round-3 commits had falsified in the
very files
they edited; the `ActionEnv` claim that a fragment never carries a
launch step
(a flow is classified by its FIRST non-echo step, so a later launch is
legal and
does thread its id); `isBlindRead`'s enumeration of what sets its flags;
and
change narration that had leaked into permanent prose.
`argent-create-flow` now
states what a `com.apple.*` target costs a flow — its worked example is
selector-driven, and round 3's justification comment claimed the
opposite.
**Verification.** 17 mutations across both rounds, every one killed by a
specifically-named test. Two consecutive full runs of
`packages/tool-server`:
3181 passed, 1 skipped, and 6 failures in `boot-device-hotboot.test.ts`
that are
pre-existing on this host (it asserts `-gpu darwin`/`swiftshader` where
a Linux
box resolves `-gpu host`; the file is untouched by this PR). `tsc
--build`,
`tsc --noEmit -p tsconfig.test.json`, eslint and prettier clean on every
changed
file; `scripts/extract-tools.mjs` emits all 75 tools with zero warnings.
---
## Review round 5 — reviewing round 4
Round 4 left two of the defects it set out to fix and introduced one. It
also
broke CI.
**ESLint failed on the push.** Two `` \` `` escapes inside a
double-quoted string
(`no-useless-escape`). The pre-push check missed it because it was piped
through
`tail`, so the shell read the pipe's exit status rather than eslint's
and
reported success over a failure. Gates are run unpiped now, and
repo-wide —
`eslint .` is what CI runs, and a changed-files invocation covers a
subset of it.
**`unregistered` still quoted the connect wait alone** — the exact
understatement `LAUNCH_TO_VERDICT_MS` was introduced for, applied in
round 4 to
`not_running` only. And the end-to-end test asserted `8000 ms` while its
sibling
fifty lines down asserted the whole spend, so the file enforced two
contradictory rules and the *correct* figure would have failed CI. The
post-launch settle counts: the connection is read off the live map and
the poll
checks it once before its first sleep, so a dial landing inside those
1.5 s is
caught. All three arms that quote a figure now quote the same one,
pinned
together.
**`stale_process` named the launchd environment as fact** four lines
below a new
comment saying that is false for one of the state's two producers. It is
in fact
sound — but only where the message already scopes it, to a *second*
landing,
whose relaunch makes the process younger than any established listener
and so
rules that producer out (it needs `processAge + grace >= listenerAge`).
The
comment now says which producer and why. The test named
"does not blame the launchd environment" asserted only the phrase it
kept, so it
was green with the blame present; it now pins the ordering it is named
for.
**The `connected` arm is reachable — round 4's "unreachable" claim was
wrong.**
`appConnectionState` re-reads the live connections map *after* its env
re-apply
and process probe (`native-devtools.ts`), several simctl round-trips
past the
empty list that routed the call there, precisely so a dial landing in
that window
is not reported as unregistered. Folding the arm away left that case
throwing a
bare "Flows resolve selectors…" with no app named and no remedy. It is a
branch
again, and says the connection arrived mid-read.
**`unexplainedHint` dropped `screenshot` along with the `bundleId`
suggestion**,
so a caller that passed a bundle id received a hint with no next action
at all —
while `argent-device-interact` tells the agent to follow that hint.
`screenshot`
is unconditional now. Its JSDoc also claimed two of three call sites are
reached
only after an explicit id resolved and measured connected; all three are
reachable with an explicit id, as the file's own comment twenty lines
above says.
**Also:** `argent-create-flow` recommended `tool: gesture-tap` where the
same
file documents the first-class, equally tree-free `tap: { x, y }`; and
change
narration removed from four more comments.
### Standing caveats
- **The `com.apple.*` premise is still unsettled and is not settled by
this PR.**
#453 recorded `connected: false` on iOS 26.5; the review recorded
`connected: true` with both dylibs mapped on 18.5. Every change here is
correct
under either reading, but `isInjectableBundleId` remains a blanket
prefix
predicate and may be wrong on some runtimes. Worth its own issue.
- `boot-electron-spawn-error.test.ts` and `boot-device-hotboot.test.ts`
fail on
the Linux host used for this work; both were confirmed failing at the PR
base
`b2e5c54d` and neither file is touched here.
---
## Review round 6 — reviewing round 5
Round 5's own headline fix was both broken and unpinned.
**The hint stopped parsing.** Moving `screenshot` out of the conditional
clause
without rejoining the sentence produced, on the default no-`bundleId`
form:
> … not evidence that nothing is on screen. Pass `bundleId` to have the
connection state measured, or Take a `screenshot` to see what is there.
That is the exact string `argent-device-interact` instructs an agent to
act on.
Both forms are one sentence again, and both are now pinned by a test
anchored to
the end of the string rather than matching a substring.
**And deleting the action entirely left the suite green.** The defect
round 5
describes — "a caller that passed a bundle id got a hint with no next
action at
all" — could be reintroduced with 635 tests passing. Nothing asserted
the hint
named a next step. It does now.
**Two claims round 5 corrected in the source survived verbatim in
tests.** The
"two of the three call sites are reached only after an explicit id
resolved and
measured connected" partition (all three are reachable with an explicit
id), and
"waited only NATIVE_READY_TIMEOUT_MS" — sitting ten lines above the
assertion
that now forbids that figure.
**A guard test did not test its own name.** `does not blame the launchd
environment for a stale process` pinned only *where* the blame sits.
Restating
it as fact on a first landing — `It lands here twice, so the …` —
passed. It
asserts the conditional now, which is the whole distinction: only a
repeat rules
out the producer that carries this endpoint.
**The `connected` branch still had no coverage** — which is how round 4
came to
delete it as dead in the first place. It is exercised now, including
that its
message still names the app and an action.
**Also:** the terminal system-app reason recommended `tool: gesture-tap`
where
the same round-5 fix in `argent-create-flow` had moved to the
first-class,
equally tree-free `tap: { x, y }`. The twin was missed.
**Verification.** Four mutations against these fixes, all killed:
dropping the
`screenshot` action; reintroducing the broken join; stating the launchd
blame as
fact; folding the `connected` branch away. Gates run unpiped and read by
exit
code — `eslint .` 0, `tsc --build` 0, `tsc --noEmit -p
tsconfig.test.json` 0,
`prettier --check .` 0.
---
## Review round 7 — reviewing round 6
**The gate asserted the contested fact as settled.** Round 3's
justification for
keeping the 8 s wait read "a system app's process inherits the injection
tokens
**and does connect on some runtimes**". Eight other places in this repo
say the
opposite — `blueprints/native-devtools.ts` ("our dylib can never load",
and the
agent-facing precheck throw), `native-devtools-status`, `describe`'s iOS
hint,
and `test/describe-tool.test.ts` ("a com.apple.* app never connects"), a
file
these commits edit. `flow-ios-tree.ts` contradicted *itself*, its
docblock
saying "can never load" above a message saying "cannot be relied on to
inject".
The fact is not settleable here, so nothing asserts it now. The comment
cites
both observations — #453's `connected: false` on iOS 26.5, the review's
`connected: true` with both dylibs mapped on 18.5 — and justifies the
wait
without picking: polling costs this step its timeout under the
pessimistic
reading and is the only thing that makes selectors work under the
optimistic
one, so waiting spends time on an open question where skipping would
bake an
answer into behaviour. The tree source's docblock is hedged to match the
message
it already had.
**This PR does not resolve that inconsistency**, and
`isInjectableBundleId`
remains a blanket `com.apple.` prefix predicate. It deserves its own
issue.
**A guard let the thing it guards be deleted.** Round 6 made the
repeat-landing
assertion conditional (`if (blame !== -1)`), so removing the whole
clause —
leaving a reader who lands there twice with "re-run the flow" and no
escape —
passed 36/36. Unconditional again, wording pinned.
**A rationale that was false in both directions.** "Keeps the
`not.toContain`
guards non-degenerate: they would pass vacuously if the constants were
made
equal" — equal constants make them *fail*, and 500 vs 1500 makes them
fail for
the wrong reason (`"1500 ms".includes("500 ms")`). It pins the substring
property those guards actually need.
**Two more presence-only guards.** "keeps the ax-degraded boot guidance
**ahead
of** the unreadable-hierarchy note" asserted only that both strings
appear, and
passed with them swapped. And round 6's own `tap: { x, y }` twin fix had
no test
at all — the same absence that got the `connected` branch deleted as
dead two
rounds earlier. Both pinned, and the `tap: { x, y }` claim was verified
end-to-end: that step form parses, resolves through `resolveTargetPoint`
with no
tree read, and passes in a flow whose selector step fails with this very
message.
**Two test names promised more than they checked** — a remedy "that is
not the
action the step just took", asserted for states whose remedy is another
relaunch; and a crash-on-launch guard forbidding only the literal `Call
launch-app` while the arm legitimately suggests launch-app for watching
the exit.
**Verification.** Three mutations, all killed: deleting the
repeat-landing
clause, reverting the coordinate form to `tool: gesture-tap`, and
putting the
read-note ahead of the boot guidance. Gates read by exit code, unpiped:
`eslint .` 0, `tsc --build` 0, `tsc --noEmit -p tsconfig.test.json` 0,
`prettier --check .` 0. Suite: 3187 passed, 1 skipped, and the 6
pre-existing
`boot-device-hotboot` failures on this Linux host.
---
## Review round 8 — reviewing round 7
**The gate's own justification was false, and the disproof was already
in this
PR's review.** Round 7's comment claimed polling "is the only thing that
makes
selectors work at all" under the optimistic reading. It is not:
`settleTree`
retries a failed tree read for `SETTLE_TIMEOUT_MS` on top of
`POST_LAUNCH_SETTLE_MS`, so roughly the first 4.5 s of connect latency
is
absorbed with no gate at all — which is exactly why the review's
fragment (no
`launch` step, therefore no gate) resolved selectors 4/4 against
`com.apple.Preferences` on 18.5. Confirmed with an in-repo
counterfactual:
at 2000 ms connect latency the flow passes with or without the gate; at
6000 ms
only with it. What the poll adds is the band from ~4.5 s out to
`LAUNCH_TO_VERDICT_MS`, and the comment now says that.
**Hedging one surface left its twins absolute — the same failure mode, a
fourth
time.** The precheck throw, 230 lines from the message this PR hedged
and in the
same file, still told agents Argent "can never be injected into it"; so
did the
predicate's docblock, `native-devtools-status`'s description ("the dylib
can
never load into them", "TERMINAL state — the app can never be
injected"), and
`describe`'s terminal hint. An agent reading `argent-create-flow` was
told
"depends on the simulator runtime" and the status tool told it "can
never".
Every agent-facing surface now says injection cannot be RELIED on, and a
test
pins that none of them claims impossibility — the six native-*
descriptions, the
status description, the shared recovery text, the device-interact skill,
and
(separately, since it is module-private and reachable only through the
tool's
output) `describe`'s terminal hint. Each surface is asserted non-empty
first, so
a missing one cannot satisfy the negative match vacuously.
**The decision is untouched.** `isInjectableBundleId` still answers "not
injectable", the precheck still throws, and the description still says
`Do NOT restart/retry` — pinned by the same test, because softening the
claim
must not soften the behaviour or #453's restart loop returns. An app
that MIGHT
connect cannot be the basis of a retry loop either, which is now the
predicate's
recorded reason.
**Also:** the round-7 reword left "The flow / flow path" duplicated;
withholding
the verdict before the measurement (promoted to its own paragraph last
round)
was asserted nowhere, so moving it after `appConnectionState` left the
suite
green — a test now proves the measurement is not taken for an app whose
verdict
is withheld either way; and two more test names were narrowed to what
they
actually check.
**Verification.** Mutations run against every decision these commits
added.
Killed: re-asserting "can never load" in the status description;
dropping its
terminal instruction; re-asserting impossibility in `describe`'s hint;
moving
the withholding after the measurement; and the earlier rounds' decisions
re-checked and still pinned. One survivor is unpinnable by construction
— a
claim inside a code comment, which no test can assert. Gates read by
exit code,
unpiped: `eslint .` 0, `tsc --build` 0, `tsc --noEmit -p
tsconfig.test.json` 0
(this one rejected a tuple vitest had accepted), `prettier --check .` 0.
---
## Review round 9 — scope: the skill docs
This is a state-derivation fix. It should not be rewriting skill docs,
and the
earlier rounds had it doing exactly that.
**`argent-create-flow` is back to its base.** Nothing in it was
falsified by this
change. The `com.apple.*` caveat rounds 4-6 grew there documents the
question
this PR explicitly declines to settle — whether `isInjectableBundleId`'s
blanket
prefix rule is right at all — and that belongs to its own issue, not to
a
paragraph in a skill an agent reads before recording a flow.
**`argent-device-interact` only got smaller.** It said *"call
`restart-app` first
if needed"* — the unconditional-restart advice the measured states exist
to
replace — and that clause is deleted. The four-status routing table the
first
round put in its place is gone with it: the remedy already travels on
the tool's
own output and on the six `native-*` descriptions, both pinned by tests,
and a
second copy in a doc no code reads is one more surface to drift.
Tests follow the prose. The two `SKILL.md` reads in
`native-devtools-status.test.ts`
go with the text they pinned, and two comments that cited the reverted
skill text
are corrected.
### Claims above that this round supersedes
- The first round's *"the `argent-device-interact` skill offered
`restart-app` as
the only remedy for a missing injection"* — the clause is deleted, not
replaced.
- Round 4's *"`argent-create-flow` now states what a `com.apple.*`
target costs a
flow"*, round 5's *"`argent-create-flow` recommended `tool:
gesture-tap`"*, and
round 6's twin fix in the skill — all in the reverted paragraph. The
matching
source-side fix in `flow-ios-tree.ts` stands; only its stated twin is
gone.
- Round 8's cross-surface certainty test no longer reads the skill. It
covers the
status description, the six `native-*` descriptions and the shared
recovery text
— the surfaces that make the claim.
**`origin/main` merged in, and #519 needed one test to follow.** It
landed after
this branch last merged main and makes a `getFullHierarchy` reply with
no windows
an error — an untrustworthy read, not a blank screen. `does not measure
when
auto-targeting resolved an app` had been using an empty `windows` as a
don't-care
return, so the call now ended before it could observe anything. The stub
serves a
real window, which is the state the test names. Mutation-checked:
dropping the
empty-connected-list term from the guard makes the measurement fire and
fails
that test alone.
**Verification.** `packages/tool-server`: 304 files, 3242 passed, 1
skipped, on a
macOS host (the `boot-device-hotboot` failures noted in the standing
caveats are
Linux-only and do not reproduce here). Gates read by exit code, unpiped:
`eslint .` 0, `tsc --build` 0, `tsc --noEmit -p tsconfig.test.json` 0,
`prettier --check .` 0.
3298 lines
132 KiB
TypeScript
3298 lines
132 KiB
TypeScript
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
|
|
import * as fs from "node:fs/promises";
|
|
import * as os from "node:os";
|
|
import * as path from "node:path";
|
|
import {
|
|
ArtifactStore,
|
|
FLOW_FILE_NAME_PATTERN,
|
|
FLOW_NAME_PATTERN,
|
|
getFailureSignal,
|
|
type Registry,
|
|
} from "@argent/registry";
|
|
import {
|
|
createRunFlowTool,
|
|
flowLaunchGateReason,
|
|
LAUNCH_TO_VERDICT_MS,
|
|
MAX_RUN_DEPTH,
|
|
NATIVE_READY_TIMEOUT_MS,
|
|
type FlowRunResult,
|
|
} from "../../src/tools/flows/flow-run";
|
|
import { serializeFlow, parseFlow } from "../../src/tools/flows/flow-utils";
|
|
import { bindDeviceArgs, stripDeviceKeys } from "../../src/tools/flows/flow-device";
|
|
import { runSnapshot } from "../../src/tools/flows/flow-visual";
|
|
|
|
// Stub the snapshot differ: the baseline-anchoring test asserts only WHERE the
|
|
// runner points it (root flowsDir + root flow name), not the diffing itself.
|
|
vi.mock("../../src/tools/flows/flow-visual", () => ({
|
|
DEFAULT_MAX_MISMATCH: 0.5,
|
|
runSnapshot: vi.fn(async () => ({ status: "pass", reason: "snapshot stubbed" })),
|
|
}));
|
|
|
|
// Mock the Electron launcher: no test here may legitimately reach it — the
|
|
// chromium-ordering test below pins exactly that — so a misplaced upload guard
|
|
// records a boot attempt instead of spawning a real process.
|
|
const bootElectronApp = vi.fn(async () => ({
|
|
platform: "chromium" as const,
|
|
id: "chromium-cdp-12345",
|
|
port: 12345,
|
|
pid: 4242,
|
|
appPath: "/abs/e2e-app",
|
|
booted: true as const,
|
|
}));
|
|
vi.mock("../../src/tools/devices/boot-electron", () => ({
|
|
bootElectronApp: (...args: unknown[]) =>
|
|
(bootElectronApp as (...a: unknown[]) => unknown)(...args),
|
|
killChromiumByPort: vi.fn(),
|
|
}));
|
|
|
|
// Five tests here drive the launch gate's real 1.5 s settle plus the whole
|
|
// connect wait in fake time, pumping a real event-loop turn between advances so
|
|
// the run's disk I/O can settle. Each pump is a real macrotask, so the cost is
|
|
// the pump count, not the fake duration — under full-suite load it outruns
|
|
// vitest's 5 s default. Widening the advance is NOT the fix: coarser steps
|
|
// starve that I/O and take longer in real time (measured: 1 s steps took the
|
|
// file from 46 s to 52 s). Budget the pumping, and prefer unit-testing the pure
|
|
// message mapping over driving the whole runner once per case.
|
|
vi.setConfig({ testTimeout: 60_000 });
|
|
|
|
/**
|
|
* Ceiling on those pumps. The loop has to cover the settle plus the entire
|
|
* connect wait in fake time — one 250 ms advance each — and still have turns
|
|
* spare for the real-async steps interleaved between them, which multiply as
|
|
* parallel workers contend for the disk.
|
|
*
|
|
* Derived from the connect budget rather than fixed, because running out is
|
|
* silent and mimics nothing else: the run never settles, the loop abandons it
|
|
* pending, and the test hangs on `await pending` until the timeout — so it reads
|
|
* as slowness, and no timeout value fixes it. A settled run leaves the loop
|
|
* immediately, so the headroom costs nothing.
|
|
*/
|
|
const PUMP_LIMIT = Math.ceil(NATIVE_READY_TIMEOUT_MS / 250) * 10;
|
|
|
|
const DEVICE = "00000000-0000-0000-0000-0000000000ab";
|
|
let tmpDir: string;
|
|
|
|
/**
|
|
* `props`, when given, makes `getTool` report that schema for EVERY tool id —
|
|
* so a fixture using it must be a single step and must pass an explicit device,
|
|
* otherwise unrelated dispatches (list-devices) would be handed a bogus schema.
|
|
*/
|
|
function mockRegistry(props?: Record<string, unknown>): Registry {
|
|
return {
|
|
invokeTool: vi.fn(async (id: string) => {
|
|
if (id === "list-devices") return { devices: [] };
|
|
return { ok: true };
|
|
}),
|
|
getTool: vi.fn(() => (props ? { inputSchema: { properties: props } } : undefined)),
|
|
// iOS launch steps gate on a native-devtools connection: report connected
|
|
// so the run proceeds. No selector directives run in these tests, so the
|
|
// flow tree is never fetched.
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => true,
|
|
listConnectedBundleIds: () => [],
|
|
})),
|
|
} as unknown as Registry;
|
|
}
|
|
|
|
async function writeFlow(name: string, yaml: Parameters<typeof serializeFlow>[0]): Promise<void> {
|
|
const dir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(dir, { recursive: true });
|
|
await fs.writeFile(path.join(dir, `${name}.yaml`), serializeFlow(yaml), "utf8");
|
|
}
|
|
|
|
/**
|
|
* Write a straight `run:` chain root → n1 → … → n{links} → tail, one `run:`
|
|
* step per flow. The final hop fires with the root plus every link on the run
|
|
* stack, i.e. a stack of exactly `links + 1` entries — which is what lets the
|
|
* depth-boundary tests below land on the guard's threshold precisely.
|
|
*/
|
|
async function writeRunChain(links: number, tail: string): Promise<void> {
|
|
await writeFlow("root", { executionPrerequisite: "", steps: [{ kind: "run", flow: "n1.yaml" }] });
|
|
for (let i = 1; i <= links; i++) {
|
|
await writeFlow(`n${i}`, {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: i === links ? tail : `n${i + 1}.yaml` }],
|
|
});
|
|
}
|
|
}
|
|
|
|
function asRun(r: FlowRunResult | { notice: string }): FlowRunResult {
|
|
if (!("steps" in r)) throw new Error(`expected a run result, got notice: ${r.notice}`);
|
|
return r;
|
|
}
|
|
|
|
beforeEach(async () => {
|
|
// realpath'd so the path math below matches the runner's canonical anchors:
|
|
// macOS's tmpdir lives behind the /var → /private/var symlink, which would
|
|
// otherwise skew every equality for reasons unrelated to what a test pins.
|
|
tmpDir = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "flow-compose-")));
|
|
});
|
|
afterEach(async () => {
|
|
await fs.rm(tmpDir, { recursive: true, force: true });
|
|
});
|
|
|
|
describe("flow composition (run:)", () => {
|
|
it("expands a referenced fragment's steps inline", async () => {
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "On login screen",
|
|
steps: [
|
|
{ kind: "echo", message: "logging in" },
|
|
{ kind: "tool", name: "tap", args: { x: 0.5 } },
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "login.yaml" },
|
|
{ kind: "echo", message: "done" },
|
|
],
|
|
});
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
|
|
// run marker, login's echo + tap, then main's echo.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"echo:pass",
|
|
"tool:pass",
|
|
"echo:pass",
|
|
]);
|
|
// The expanded steps are attributed to the fragment.
|
|
expect(result.steps[1].flow).toBe("login");
|
|
expect(result.steps[3].flow).toBe("main");
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("composes a bare run: name from the containing flow's own directory", async () => {
|
|
// The legacy layout: both flows saved flat in .argent/flows, main written
|
|
// when a run: target was a saved-flow NAME resolved in that directory. A
|
|
// bare target now anchors to the containing file's directory instead —
|
|
// which for this layout IS that directory, so the flow replays unchanged.
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "On login screen",
|
|
steps: [{ kind: "echo", message: "logging in" }],
|
|
});
|
|
await fs.writeFile(
|
|
path.join(tmpDir, ".argent", "flows", "main.yaml"),
|
|
"steps:\n - run: login\n - echo: done\n",
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"echo:pass",
|
|
"echo:pass",
|
|
]);
|
|
// Parse completed the extension, so the report names the file it opened
|
|
// while attribution keeps the same stem the spelled-out form produces.
|
|
expect(result.steps[0]).toMatchObject({ target: "login.yaml", flow: "login" });
|
|
expect(result.steps[1].flow).toBe("login");
|
|
});
|
|
|
|
it("resolves a bare run: name against a nested fragment's directory", async () => {
|
|
// The one place the two schemes disagree: a bare name inside a fragment
|
|
// that does NOT sit in the flows dir root. `helpers/steps.yaml` composing
|
|
// `login` gets helpers/login.yaml — its own sibling — not the flows-dir
|
|
// copy a name lookup would have found. Both exist here with different
|
|
// messages, so the assertion distinguishes them.
|
|
const helpers = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpers, { recursive: true });
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "FLOWS-DIR-COPY" }],
|
|
});
|
|
await fs.writeFile(
|
|
path.join(helpers, "login.yaml"),
|
|
"steps:\n - echo: HELPERS-SIBLING\n",
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(path.join(helpers, "steps.yaml"), "steps:\n - run: login\n", "utf8");
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/steps.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => s.message)).toContain("HELPERS-SIBLING");
|
|
expect(result.steps.map((s) => s.message)).not.toContain("FLOWS-DIR-COPY");
|
|
});
|
|
|
|
it("refuses a bare run: name whose casing is not the one on disk", async () => {
|
|
// Completing the extension must not open a casing hole: `run: Login`
|
|
// becomes Login.yaml and meets the same on-disk spelling gate as the
|
|
// spelled-out form, so a macOS-green / Linux-red tree is still refused.
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "RAN-THE-LOWERCASE-FILE" }],
|
|
});
|
|
await fs.writeFile(
|
|
path.join(tmpDir, ".argent", "flows", "main.yaml"),
|
|
"steps:\n - run: Login\n",
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
expect(result.steps[0]?.reason ?? "").toMatch(/^mis-cased fragment reference "Login\.yaml": /);
|
|
expect(result.steps.map((s) => s.message)).not.toContain("RAN-THE-LOWERCASE-FILE");
|
|
});
|
|
|
|
it("resolves run: fragments beside an explicit top-level flow_path", async () => {
|
|
const externalDir = path.join(tmpDir, "external-flows");
|
|
const mainPath = path.join(externalDir, "main.yaml");
|
|
await fs.mkdir(externalDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(externalDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "sibling fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
mainPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "login.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute(
|
|
{},
|
|
{ project_root: tmpDir, flow_path: mainPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_path: {
|
|
clientPath: mainPath,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
statVerified: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
);
|
|
|
|
expect(result.flow).toBe("main");
|
|
expect(result.steps.map((step) => `${step.kind}:${step.status}`)).toEqual([
|
|
"run:pass",
|
|
"echo:pass",
|
|
]);
|
|
expect(result.steps[1]).toMatchObject({ flow: "login", message: "sibling fragment" });
|
|
});
|
|
|
|
it("names the lowercase requirement when only the flow_path extension's case is wrong", async () => {
|
|
const upperPath = path.join(tmpDir, "Main.YAML");
|
|
await fs.writeFile(
|
|
upperPath,
|
|
serializeFlow({ executionPrerequisite: "", steps: [{ kind: "echo", message: "hi" }] }),
|
|
"utf8"
|
|
);
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
await expect(
|
|
runFlow.execute(
|
|
{},
|
|
{ project_root: tmpDir, flow_path: upperPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_path: {
|
|
clientPath: upperPath,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
statVerified: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
).rejects.toThrow('flow files must use the lowercase .yaml extension, not ".YAML"');
|
|
});
|
|
|
|
it("resolves a run: path against the containing flow's own directory, not the root's", async () => {
|
|
// Root (in .argent/flows) → ../../shared/login.yaml → helper.yaml, where
|
|
// helper.yaml is login's OWN sibling in shared/ and absent from the root's
|
|
// directory — only per-file anchoring resolves it.
|
|
const sharedDir = path.join(tmpDir, "shared");
|
|
await fs.mkdir(sharedDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedDir, "helper.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "from helper" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(sharedDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helper.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../shared/login.yaml" }],
|
|
});
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"run:pass",
|
|
"echo:pass",
|
|
]);
|
|
// Attribution is the basename stem; the report target keeps the as-written path.
|
|
expect(result.steps[0]).toMatchObject({ flow: "login", target: "../../shared/login.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ flow: "helper", target: "helper.yaml" });
|
|
expect(result.steps[2]).toMatchObject({ flow: "helper", message: "from helper" });
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("does not flag two different files sharing a basename as a cycle", async () => {
|
|
const subDir = path.join(tmpDir, ".argent", "flows", "sub");
|
|
await fs.mkdir(subDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(subDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inner login" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
// Root flow "login" runs sub/login.yaml: same stem, different file.
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "sub/login.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("disambiguates a fragment whose stem collides with the root flow's name", async () => {
|
|
// Root login.yaml composes helpers/login.yaml — two different files, one
|
|
// stem. A bare-stem attribution would make every fragment step carry
|
|
// flow === the report's top-level flow, and renderers that mark fragment
|
|
// steps by that inequality (the CLI's `[fragment]` suffix) would read a
|
|
// failure inside the fragment as a failure of the root flow itself.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inside the shared fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/login.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.flow).toBe("login");
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
// On collision the attribution is the as-written path minus the .yaml
|
|
// extension — never the bare stem, which would equal result.flow — on the
|
|
// run marker and every expanded step alike.
|
|
expect(result.steps[0]).toMatchObject({ flow: "helpers/login", target: "helpers/login.yaml" });
|
|
expect(result.steps[1]).toMatchObject({
|
|
flow: "helpers/login",
|
|
message: "inside the shared fragment",
|
|
});
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("keeps the bare stem attribution when the fragment's stem does not collide with the root's", async () => {
|
|
// The SAME fragment as the collision test above, composed from a root
|
|
// with a different name: attribution stays the documented basename stem,
|
|
// because only a root-stem collision makes the stem ambiguous downstream.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inside the shared fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("checkout", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/login.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "checkout", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.flow).toBe("checkout");
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[0]).toMatchObject({ flow: "login", target: "helpers/login.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ flow: "login", message: "inside the shared fragment" });
|
|
});
|
|
|
|
it("qualifies a nested fragment's bare same-stem spelling as ./<stem>", async () => {
|
|
// Root login.yaml → helpers/steps.yaml → run: login.yaml. The bare
|
|
// spelling resolves against the CONTAINING file's directory, so it names
|
|
// helpers/login.yaml — a genuine fragment, not the root, and no cycle.
|
|
// Stripping the extension off a bare spelling reproduces the stem, which
|
|
// would equal result.flow and drop the renderers' fragment marker; the
|
|
// collision rule qualifies it as ./login — an equivalent spelling of the
|
|
// as-written reference that keeps the inequality.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "steps.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "login.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inside the nested sibling" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/steps.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.flow).toBe("login");
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"run:pass",
|
|
"echo:pass",
|
|
]);
|
|
// The nested run marker and the fragment's expanded step carry ./login —
|
|
// never the bare "login", which would read as the root flow itself.
|
|
expect(result.steps[1]).toMatchObject({ flow: "./login", target: "login.yaml" });
|
|
expect(result.steps[2]).toMatchObject({
|
|
flow: "./login",
|
|
message: "inside the nested sibling",
|
|
});
|
|
expect(result.steps[1]?.flow).not.toBe(result.flow);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("applies the root-stem collision disambiguation to a failed run: report too", async () => {
|
|
// Root login.yaml references helpers/login.yaml, which is MISSING — a
|
|
// moved or deleted fragment is the most common way a run: fails, and the
|
|
// errored marker (execRunStep's fail(), which stamps `flow: display`) is
|
|
// the line that names the missing file. Every other test reaching fail()
|
|
// uses a non-colliding root, so weakening just that line to the bare stem
|
|
// (`flow: runTargetName(target)`) would slip through the rest of the
|
|
// suite — here it would collapse flow to "login" === result.flow, and
|
|
// renderers that mark fragment lines by that inequality (the CLI's
|
|
// `[fragment]` suffix) would read the load failure as the root flow's own.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/login.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.flow).toBe("login");
|
|
expect(result.steps[0]).toMatchObject({
|
|
kind: "run",
|
|
status: "error",
|
|
flow: "helpers/login",
|
|
target: "helpers/login.yaml",
|
|
});
|
|
expect(result.steps[0]?.reason).toMatch(/^could not load fragment "helpers\/login\.yaml": /);
|
|
expect(result.steps[0]?.reason).toContain("ENOENT");
|
|
// The exact inequality renderers key on to mark fragment lines.
|
|
expect(result.steps[0]?.flow).not.toBe(result.flow);
|
|
});
|
|
|
|
it("detects a cycle reached through a different relative spelling", async () => {
|
|
const subDir = path.join(tmpDir, ".argent", "flows", "sub");
|
|
await fs.mkdir(subDir, { recursive: true });
|
|
await writeFlow("a", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "sub/b.yaml" }],
|
|
});
|
|
await fs.writeFile(
|
|
path.join(subDir, "b.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../a.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "a", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const errored = result.steps.find((s) => s.status === "error");
|
|
// The closing hop shares the root's stem (a cycle back to the root always
|
|
// does), so the collision disambiguation renders it as-written — which
|
|
// names the exact edge that closes the loop, rather than the bare "a"
|
|
// that reads like a self-reference from the root's own directory.
|
|
expect(errored?.reason).toMatch(/cyclic flow reference: a → b → \.\.\/a/);
|
|
// Where the as-written path differs from the stem it is the value that
|
|
// locates the reference — `run ../a.yaml`, not a stem-derived fallback.
|
|
expect(errored?.target).toBe("../a.yaml");
|
|
});
|
|
|
|
it("anchors a symlinked root flow's run: at the real file's directory", async () => {
|
|
// .argent/flows/main.yaml is a symlink to shared/flows/main.yaml, which
|
|
// references ../helpers/x.yaml. Only the canonical anchor resolves the
|
|
// shared helper; the un-canonicalized one would hit the project decoy.
|
|
const sharedFlows = path.join(tmpDir, "shared", "flows");
|
|
const sharedHelpers = path.join(tmpDir, "shared", "helpers");
|
|
const decoyHelpers = path.join(tmpDir, ".argent", "helpers");
|
|
await fs.mkdir(sharedFlows, { recursive: true });
|
|
await fs.mkdir(sharedHelpers, { recursive: true });
|
|
await fs.mkdir(decoyHelpers, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedFlows, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../helpers/x.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(sharedHelpers, "x.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "shared helper" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(decoyHelpers, "x.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "project decoy" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.symlink(path.join(sharedFlows, "main.yaml"), path.join(flowsDir, "main.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps[1]).toMatchObject({ kind: "echo", message: "shared helper" });
|
|
});
|
|
|
|
it("collides a fragment's stem against a symlinked root's SPELLED name, not its real file's", async () => {
|
|
// .argent/flows/smoke.yaml is a symlink to shared/flows/main.yaml —
|
|
// renaming on the link side is the ordinary reason to symlink a saved
|
|
// flow — whose only step is run: helpers/smoke.yaml. The fragment's stem
|
|
// "smoke" collides with the SPELLED root name (what result.flow carries),
|
|
// not the real file's stem "main", so the disambiguation must fire.
|
|
// Seeding the runStack with the canonical stem instead — `display:
|
|
// path.basename(canonicalPath, ".yaml")` in execute()'s seed — survives
|
|
// every other fixture, because each one points its symlink at a real file
|
|
// with the SAME basename (the two candidate values are equal); here it
|
|
// collapses the fragment's flow to "smoke" === result.flow, and renderers
|
|
// that mark fragment steps by that inequality (the CLI's `[fragment]`
|
|
// suffix) would print the fragment's steps as the root flow's own.
|
|
const sharedFlows = path.join(tmpDir, "shared", "flows");
|
|
const sharedHelpers = path.join(sharedFlows, "helpers");
|
|
await fs.mkdir(sharedHelpers, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedFlows, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/smoke.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(sharedHelpers, "smoke.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "inside the shared fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.symlink(path.join(sharedFlows, "main.yaml"), path.join(flowsDir, "smoke.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "smoke", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.flow).toBe("smoke");
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
// The run marker and the fragment's expanded step both carry the
|
|
// as-written path minus the extension — never the bare stem.
|
|
expect(result.steps[0]).toMatchObject({ flow: "helpers/smoke", target: "helpers/smoke.yaml" });
|
|
expect(result.steps[1]).toMatchObject({
|
|
flow: "helpers/smoke",
|
|
message: "inside the shared fragment",
|
|
});
|
|
// The exact inequality renderers key on to mark fragment lines.
|
|
expect(result.steps[0]?.flow).not.toBe(result.flow);
|
|
expect(result.steps[1]?.flow).not.toBe(result.flow);
|
|
});
|
|
|
|
it("resolves a run: `..` after a symlinked component with kernel semantics", async () => {
|
|
// .argent/flows/link is a symlink to lex/other, so on disk
|
|
// `link/../frag.yaml` means lex/frag.yaml — `..` names the parent of the
|
|
// link's TARGET. A lexical collapse of the spelling (path.resolve before
|
|
// realpath) instead names the flows-dir sibling frag.yaml; with a decoy
|
|
// planted there, only kernel-faithful resolution runs the file the
|
|
// written path denotes.
|
|
const lexDir = path.join(tmpDir, "lex");
|
|
await fs.mkdir(path.join(lexDir, "other"), { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(lexDir, "frag.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "kernel-resolved fragment" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "lexical decoy" }],
|
|
});
|
|
await writeFlow("root", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "link/../frag.yaml" }],
|
|
});
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.symlink(path.join(lexDir, "other"), path.join(flowsDir, "link"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "root", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[1]).toMatchObject({ kind: "echo", message: "kernel-resolved fragment" });
|
|
expect(result.steps.map((s) => s.message)).not.toContain("lexical decoy");
|
|
});
|
|
|
|
it("errors on a run: `..` through a dangling symlink instead of executing a lexical impostor", async () => {
|
|
// .argent/flows/dangling points at a missing target, so the kernel refuses
|
|
// the spelling `dangling/../frag.yaml` with ENOENT — realpath fails on both
|
|
// the full path and its dirname. A lexical collapse of the spelling would
|
|
// name the flows-dir sibling frag.yaml (planted as a decoy) and run it with
|
|
// run:pass; the fallback must instead surface the kernel's ENOENT for the
|
|
// spelling itself and never execute the decoy.
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "lexical decoy" }],
|
|
});
|
|
await writeFlow("root", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "dangling/../frag.yaml" }],
|
|
});
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.symlink(path.join(tmpDir, "gone"), path.join(flowsDir, "dangling"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "root", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
expect(result.steps[0]?.reason).toMatch(/could not load fragment "dangling\/\.\.\/frag\.yaml"/);
|
|
expect(result.steps[0]?.reason).toMatch(/ENOENT/);
|
|
expect(result.steps.map((s) => s.message)).not.toContain("lexical decoy");
|
|
});
|
|
|
|
it("runs a run: target that reaches above the project root", async () => {
|
|
// There is no path fence on `run:`. A target is reachable when the tool
|
|
// server can read it — the same reach the front door already grants,
|
|
// since flow_path can name any YAML on the host. The old fence made
|
|
// admission depend on project_root, which the CLI sets to the operator's
|
|
// cwd, so the identical file passed from the repo root and failed from a
|
|
// subdirectory. Project root is proj/, the target resolves above it, and
|
|
// it must run.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const flowsDir = path.join(proj, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(tmpDir, "outside.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "executed from outside project root" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(flowsDir, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../../outside.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: proj, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[0]).toMatchObject({ flow: "outside", target: "../../../outside.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ message: "executed from outside project root" });
|
|
});
|
|
|
|
it("reports a MISSING target above the project root as that file's own ENOENT", async () => {
|
|
// With the fence gone, a target that resolves nowhere gets the ordinary
|
|
// per-file load error naming the as-written spelling — the same report an
|
|
// in-project typo produces, wherever it points. The old boundary error
|
|
// ("resolves outside the project root") answered a typo above the root
|
|
// with a policy message about a boundary that no longer exists, sending
|
|
// the author to look at project_root instead of at the spelling.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const flowsDir = path.join(proj, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(flowsDir, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../../nothing-here.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: proj, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
expect(result.steps[0]?.reason).toMatch(
|
|
/could not load fragment "\.\.\/\.\.\/\.\.\/nothing-here\.yaml"/
|
|
);
|
|
expect(result.steps[0]?.reason).toMatch(/ENOENT/);
|
|
});
|
|
|
|
it("gives the same verdict for one flow file under two different project_root values", async () => {
|
|
// The regression the fence caused: admission was decided by
|
|
// `params.project_root`, which `argent flow run` sets to process.cwd(),
|
|
// so `cd packages/app && argent flow run .argent/flows/e2e.yaml` failed
|
|
// while the identical file passed from the repo root — green locally, red
|
|
// in CI, for a reason nothing in the flow file expresses. The two roots
|
|
// here STRADDLE the target: shared/login.yaml sits under tmpDir, which the
|
|
// old fence's project-root zone admitted, and outside proj, which neither
|
|
// that zone nor the containing file's directory (proj/.argent/flows)
|
|
// covered — so the fence passed the first and refused the second. Same
|
|
// flow_path, same sideways `run:` target: the two must now reach the same
|
|
// verdict and execute the fragment.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const flowsDir = path.join(proj, ".argent", "flows");
|
|
const sharedDir = path.join(tmpDir, "shared");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.mkdir(sharedDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "shared login" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const mainPath = path.join(flowsDir, "main.yaml");
|
|
await fs.writeFile(
|
|
mainPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../../shared/login.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
// The flow_path route with a boundary-resolved, stat-verified client path
|
|
// is exactly what `argent flow run` sends (see packages/argent-cli/src/flow.ts).
|
|
const runFrom = async (projectRoot: string): Promise<FlowRunResult> =>
|
|
asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ project_root: projectRoot, flow_path: mainPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_path: {
|
|
clientPath: mainPath,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
statVerified: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
);
|
|
|
|
const fromAboveProject = await runFrom(tmpDir);
|
|
const fromProject = await runFrom(proj);
|
|
|
|
for (const result of [fromAboveProject, fromProject]) {
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[1]).toMatchObject({ flow: "login", message: "shared login" });
|
|
}
|
|
});
|
|
|
|
it("runs a symlinked-out vendored tree's own subtree fragments", async () => {
|
|
// The vault/vendored layout with the real tree OUTSIDE the project root:
|
|
// proj/.argent/flows/main.yaml is a symlink to vendor/flows/main.yaml,
|
|
// which composes helpers/x.yaml below its own real directory. Every path
|
|
// here is anchored on the containing file's CANONICAL directory, so the
|
|
// fragment is looked up beside the real main.yaml in vendor/flows, not
|
|
// beside the symlink in the project — which is what makes the
|
|
// SKILL-recommended shared-tree layout work at all.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const projFlows = path.join(proj, ".argent", "flows");
|
|
const vendorFlows = path.join(tmpDir, "vendor", "flows");
|
|
await fs.mkdir(projFlows, { recursive: true });
|
|
await fs.mkdir(path.join(vendorFlows, "helpers"), { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "helpers", "x.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "vendored helper" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/x.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.symlink(path.join(vendorFlows, "main.yaml"), path.join(projFlows, "main.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: proj, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[1]).toMatchObject({ kind: "echo", message: "vendored helper" });
|
|
});
|
|
|
|
it("lets a nested fragment in a symlinked-out tree reach a sibling of its root flow", async () => {
|
|
// Same vendored layout, one level deeper: main.yaml → helpers/b.yaml →
|
|
// ../sibling.yaml, where sibling.yaml sits beside main.yaml in the vendor
|
|
// tree. This is the case the review filed. The old fence narrowed on
|
|
// every descent — from helpers/ the admitted zone was helpers/ itself,
|
|
// and the vendor tree is by definition not under the project root — so a
|
|
// fragment could reach that sibling from main.yaml but not from one
|
|
// directory down, capping a shared flows tree at a single level. Nothing
|
|
// narrows now: each hop anchors on its own containing file's canonical
|
|
// directory and the target is read if it is there.
|
|
const proj = path.join(tmpDir, "proj");
|
|
const projFlows = path.join(proj, ".argent", "flows");
|
|
const vendorFlows = path.join(tmpDir, "vendor", "flows");
|
|
await fs.mkdir(path.join(vendorFlows, "helpers"), { recursive: true });
|
|
await fs.mkdir(projFlows, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "sibling.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "vendored sibling" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "helpers", "b.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../sibling.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(vendorFlows, "main.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "helpers/b.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.symlink(path.join(vendorFlows, "main.yaml"), path.join(projFlows, "main.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: proj, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"run:pass",
|
|
"echo:pass",
|
|
]);
|
|
expect(result.steps[1]).toMatchObject({ flow: "sibling", target: "../sibling.yaml" });
|
|
expect(result.steps[2]).toMatchObject({ message: "vendored sibling" });
|
|
});
|
|
|
|
it("detects a cycle through a symlinked spelling", async () => {
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await writeFlow("a", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "alias.yaml" }],
|
|
});
|
|
await fs.symlink(path.join(flowsDir, "a.yaml"), path.join(flowsDir, "alias.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "a", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
// Exact-match on the whole report, not a substring: the symlink-aware
|
|
// guard trips on the FIRST hop (realpath equates alias.yaml with the
|
|
// running a.yaml), one error step and nothing expanded. Without realpath
|
|
// the run recurses a level before the resolved-path comparison catches
|
|
// it — "a → alias → alias" after an expanded pass — which a substring
|
|
// match on the shorter chain would still accept.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}:${s.depth ?? 0}`)).toEqual([
|
|
"run:error:0",
|
|
]);
|
|
expect(result.steps[0]?.reason).toBe("cyclic flow reference: a → alias");
|
|
});
|
|
|
|
it("names a cycle that closes exactly at the run-depth limit as a cycle", async () => {
|
|
// The closing hop back to root fires with a run stack of exactly
|
|
// MAX_RUN_DEPTH — the same hop the depth guard would reject. It is still a
|
|
// loop, and the chain is what tells the author which edge to cut, so the
|
|
// cycle must win here; ordering the depth guard first swallows both.
|
|
const links = MAX_RUN_DEPTH - 1;
|
|
await writeRunChain(links, "root.yaml");
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "root", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
// The closing hop is the bare spelling `root.yaml`, whose stem is the
|
|
// root's own name — the collision rule qualifies a bare same-stem
|
|
// spelling as `./<stem>`, so the chain's final entry renders as ./root.
|
|
const chain = ["root", ...Array.from({ length: links }, (_, i) => `n${i + 1}`), "./root"];
|
|
const errored = result.steps.find((s) => s.status === "error");
|
|
expect(errored?.reason).toBe(`cyclic flow reference: ${chain.join(" → ")}`);
|
|
expect(errored?.reason).not.toContain("max run depth");
|
|
});
|
|
|
|
it("reports a non-cyclic chain past the run-depth limit as excessive depth", async () => {
|
|
// Same shape and same threshold as the cycle above, but every flow is
|
|
// distinct — this is the depth guard's remaining reachable case, and it
|
|
// must keep firing now that the cycle guard precedes it.
|
|
const links = MAX_RUN_DEPTH - 1;
|
|
await writeRunChain(links, `n${MAX_RUN_DEPTH}.yaml`);
|
|
await writeFlow(`n${MAX_RUN_DEPTH}`, {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "one hop too deep to reach" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "root", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const errored = result.steps.find((s) => s.status === "error");
|
|
expect(errored?.reason).toBe("max run depth exceeded");
|
|
expect(errored?.flow).toBe(`n${MAX_RUN_DEPTH}`);
|
|
// The as-written path rides the depth error too — the report line must
|
|
// name the reference that overflowed, not just the stem.
|
|
expect(errored?.target).toBe(`n${MAX_RUN_DEPTH}.yaml`);
|
|
});
|
|
|
|
it("reports a missing run: target as a step error, not a tool-level rejection", async () => {
|
|
// A typo'd or moved fragment path is the most common way a run: fails,
|
|
// and it must land in the per-step report — resolved, ok: false, with the
|
|
// underlying cause — never reject the whole tool call. The catch around
|
|
// the fragment read is the only thing between the two.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "gone.yaml" },
|
|
{ kind: "echo", message: "never reached" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps[0]).toMatchObject({
|
|
kind: "run",
|
|
status: "error",
|
|
flow: "gone",
|
|
target: "gone.yaml",
|
|
});
|
|
expect(result.steps[0]?.reason).toMatch(/^could not load fragment "gone\.yaml": /);
|
|
expect(result.steps[0]?.reason).toContain("ENOENT");
|
|
// The failure hard-stops the flow like any other step error.
|
|
expect(result.steps[1]).toMatchObject({ kind: "echo", status: "skip" });
|
|
});
|
|
|
|
it("reports a malformed run: target as a step error carrying the parse failure", async () => {
|
|
// The same catch covers parseFlow, so a fragment that exists but does not
|
|
// parse degrades identically — the parse diagnostic reaches the step
|
|
// report instead of rejecting the run.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "broken.yaml" }],
|
|
});
|
|
// Written raw — serializeFlow could never produce an invalid step.
|
|
await fs.writeFile(
|
|
path.join(tmpDir, ".argent", "flows", "broken.yaml"),
|
|
"steps:\n - frobnicate: nope\n",
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps[0]).toMatchObject({
|
|
kind: "run",
|
|
status: "error",
|
|
flow: "broken",
|
|
target: "broken.yaml",
|
|
});
|
|
expect(result.steps[0]?.reason).toMatch(/^could not load fragment "broken\.yaml": /);
|
|
expect(result.steps[0]?.reason).toContain("unrecognized step kind");
|
|
});
|
|
|
|
it("bounds the reason when a run: target is in-project YAML but not a flow", async () => {
|
|
// A mistyped run: path can select any YAML file inside the project root —
|
|
// here a CI config whose sole top-level key happens to be `steps`. Its
|
|
// first entry lands in the parse diagnostic, and the reason ships verbatim
|
|
// to `argent flow run` stdout and into the MCP content block, so badEntry's
|
|
// cap must keep the file's values from surviving to those surfaces in full.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../ci/deploy.yaml" }],
|
|
});
|
|
await fs.mkdir(path.join(tmpDir, "ci"), { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(tmpDir, "ci", "deploy.yaml"),
|
|
`steps:\n - db_password: "hunter2-${"x".repeat(4000)}-PROD-TAIL"\n`,
|
|
"utf8"
|
|
);
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps[0]).toMatchObject({ kind: "run", status: "error" });
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^could not load fragment "\.\.\/\.\.\/ci\/deploy\.yaml": /);
|
|
expect(reason).toContain("…(+");
|
|
expect(reason).not.toContain("PROD-TAIL");
|
|
expect(reason.length).toBeLessThan(500);
|
|
});
|
|
|
|
it("refuses a mis-cased run: target instead of running the case-folded file", async () => {
|
|
// These tests run on macOS, whose case-insensitive APFS is precisely what
|
|
// makes this reachable: `run: Frag.yaml` opens the file really named
|
|
// frag.yaml, the fragment's steps run, and every one of them is attributed
|
|
// to "Frag" — a name no directory entry carries. The same tree on a
|
|
// case-sensitive checkout (Linux CI) dies with ENOENT, so the slip must be
|
|
// refused where it is authored, not where it is replayed. Nothing of the
|
|
// fragment may execute: the refusal precedes the read.
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "RAN-THE-LOWERCASE-FILE" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "Frag.yaml" },
|
|
{ kind: "echo", message: "never reached" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error", "echo:skip"]);
|
|
expect(result.steps[0]).toMatchObject({ kind: "run", flow: "Frag", target: "Frag.yaml" });
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^mis-cased fragment reference "Frag\.yaml": /);
|
|
expect(reason).toContain(`no directory entry is named "Frag.yaml"`);
|
|
expect(reason).toContain(`case-insensitively to "frag.yaml"`);
|
|
// The recovery quotes a target the run: gate would itself accept.
|
|
expect(reason).toContain(`reference it as "frag.yaml"`);
|
|
// Not the load error: the file was never read, so nothing of it ran.
|
|
expect(reason).not.toContain("could not load fragment");
|
|
expect(result.steps.map((s) => s.message)).not.toContain("RAN-THE-LOWERCASE-FILE");
|
|
});
|
|
|
|
it("runs a run: target whose mixed-case spelling is the one really on disk", async () => {
|
|
// The gate compares byte-for-byte against the listing, not against a
|
|
// lowercase convention: an on-disk Frag.yaml is a perfectly portable
|
|
// fragment and `run: Frag.yaml` must compose it, with the mixed-case stem
|
|
// keying its steps. A gate that merely lowercased the target would fail
|
|
// this and break every camelCase flow tree.
|
|
await writeFlow("Frag", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "from the mixed-case file" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "Frag.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[0]).toMatchObject({ flow: "Frag", target: "Frag.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ flow: "Frag", message: "from the mixed-case file" });
|
|
});
|
|
|
|
it("runs a symlinked run: spelling whose real file is named differently", async () => {
|
|
// The listing consulted must be the directory the target is SPELLED in,
|
|
// compared against the SPELLED basename. Comparing against
|
|
// path.basename(canonical) instead — realpath rewrites a symlink to its
|
|
// target — would see "a.yaml" for `run: alias.yaml` and refuse a link that
|
|
// every directory entry backs and that the cycle guard already treats as a
|
|
// first-class spelling.
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await writeFlow("a", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "reached through the link" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "alias.yaml" }],
|
|
});
|
|
await fs.symlink(path.join(flowsDir, "a.yaml"), path.join(flowsDir, "alias.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "echo:pass"]);
|
|
expect(result.steps[0]).toMatchObject({ flow: "alias", target: "alias.yaml" });
|
|
expect(result.steps[1]).toMatchObject({ message: "reached through the link" });
|
|
});
|
|
|
|
it("reports a target absent from the listing as ENOENT, not as a casing slip", async () => {
|
|
// Only a case-FOLDED verdict may refuse. A mixed-case target with no
|
|
// case-insensitive neighbour at all is an ordinary missing fragment, and
|
|
// the per-file ENOENT from the read names the spelling and the reason far
|
|
// better than a casing complaint could — a gate that fired on `absent` too
|
|
// would answer every typo with advice about capitalization.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "Gone.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^could not load fragment "Gone\.yaml": /);
|
|
expect(reason).toContain("ENOENT");
|
|
expect(reason).not.toContain("case-insensitively");
|
|
});
|
|
|
|
it("refuses a mis-cased run: target in a subdirectory", async () => {
|
|
// A multi-segment target is checked in whatever directory it lands in, not
|
|
// only in the containing flow's own — the phantom spelling is exactly as
|
|
// unportable one level down, and a check anchored on the anchor directory
|
|
// alone would list the wrong directory and wave it through.
|
|
const subDir = path.join(tmpDir, ".argent", "flows", "sub");
|
|
await fs.mkdir(subDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(subDir, "frag.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "RAN-THE-SUBDIR-FILE" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "sub/Frag.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^mis-cased fragment reference "sub\/Frag\.yaml": /);
|
|
expect(reason).toContain(`case-insensitively to "frag.yaml"`);
|
|
// The hint keeps the target's own directory prefix, so it is the line the
|
|
// author can paste back into the flow.
|
|
expect(reason).toContain(`reference it as "sub/frag.yaml"`);
|
|
expect(result.steps.map((s) => s.message)).not.toContain("RAN-THE-SUBDIR-FILE");
|
|
});
|
|
|
|
it("asks for a rename when the on-disk spelling no run: target could reach", async () => {
|
|
// The on-disk name here is frag.YAML, which the run: extension gate
|
|
// (parseRunTarget) refuses outright — quoting it back as a replacement
|
|
// target would send the author to an error one layer down. The only way
|
|
// out is the rename, so that is what the recovery asks for.
|
|
await fs.mkdir(path.join(tmpDir, ".argent", "flows"), { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(tmpDir, ".argent", "flows", "frag.YAML"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "RAN-THE-SHOUTY-FILE" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "frag.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(false);
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:error"]);
|
|
const reason = result.steps[0]?.reason ?? "";
|
|
expect(reason).toMatch(/^mis-cased fragment reference "frag\.yaml": /);
|
|
expect(reason).toContain(`rename "frag.YAML" to "frag.yaml"`);
|
|
expect(reason).not.toContain("reference it as");
|
|
expect(result.steps.map((s) => s.message)).not.toContain("RAN-THE-SHOUTY-FILE");
|
|
});
|
|
|
|
it("rejects run: composition when the root flow was uploaded (no shared filesystem)", async () => {
|
|
// A remote client's flow arrives as content and is materialized to a temp
|
|
// file — the files its run: paths reference stayed on the client, and a
|
|
// same-named file on the server must never be read in their place. The
|
|
// rejection is a preflight contract error, so no step (e.g. a leading
|
|
// launch or tap) executes before it fires.
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "echo", message: "before" },
|
|
{ kind: "run", flow: "login.yaml" },
|
|
{ kind: "echo", message: "after" },
|
|
],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(tmpDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "server-local file" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
await expect(
|
|
createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
).rejects.toThrow(/co-located/i);
|
|
// Preflight, not mid-run: nothing was dispatched to the device.
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects an uploaded flow whose run: hides behind a when: block that would not fire", async () => {
|
|
// Without the preflight walking when: children, this flow reports green on
|
|
// iOS and only errors when the android guard first fires (e.g. in CI).
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n - when:\n platform: android\n steps:\n - run: login.yaml\n",
|
|
"utf8"
|
|
);
|
|
|
|
await expect(
|
|
createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
).rejects.toThrow(/co-located/i);
|
|
});
|
|
|
|
it("rejects an uploaded flow whose run: sits two when: blocks deep", async () => {
|
|
// Pins walkSteps' recursion DEPTH, which the single-level test above
|
|
// cannot: a one-level peek (`yield* inner` in place of
|
|
// `yield* walkSteps(inner)`) still passes that neighbor but never reaches
|
|
// this run:, so the flow reports ok: true with the run line inside a block
|
|
// skip — the silent-green CI outcome the preflight exists to prevent. The
|
|
// parser nests when: to MAX_BLOCK_DEPTH (20), so two levels is ordinary
|
|
// authorable YAML, not an edge case.
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n" +
|
|
" - when:\n" +
|
|
" platform: android\n" +
|
|
" steps:\n" +
|
|
" - when:\n" +
|
|
" platform: android\n" +
|
|
" steps:\n" +
|
|
" - run: login.yaml\n",
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
const err = await createRunFlowTool(registry)
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
() => null,
|
|
(e: unknown) => e
|
|
);
|
|
expect(err).toBeInstanceOf(Error);
|
|
expect((err as Error).message).toContain('run: composition ("run: login.yaml")');
|
|
expect(getFailureSignal(err)?.failure_stage).toBe("flow_upload_run_composition");
|
|
// Preflight, not mid-run: nothing was dispatched to the device.
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects an uploaded chromium e2e flow's run: before booting the launch's Electron app", async () => {
|
|
// The guard's docstring promises the rejection fires "before anything
|
|
// executes" — a POSITIONAL contract its mere existence doesn't keep. For
|
|
// an uploaded chromium e2e flow that also carries a run:, the guard
|
|
// sitting above resolveRunDevice is what decides whether an Electron
|
|
// instance is booted (and orphaned by the throw, which lands before the
|
|
// teardown finally) just to deliver a contract error. The absolute app
|
|
// path is deliberate — legal for an upload — so nothing else stands
|
|
// between the boot and the error: move the guard call below
|
|
// `const device = resolved.device;` and the boot fires first.
|
|
bootElectronApp.mockClear();
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n - launch: { chromium: /abs/e2e-app }\n - run: login.yaml\n",
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
await expect(
|
|
createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
).rejects.toThrow(/run: composition/);
|
|
// Preflight, not post-boot: no Electron instance was spawned and no
|
|
// device was resolved before the rejection.
|
|
expect(bootElectronApp).not.toHaveBeenCalled();
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects an uploaded fragment's run: instead of answering the prerequisite handshake", async () => {
|
|
// The other half of the guard's "before anything executes" position: it
|
|
// sits above the executionPrerequisite early return. A prerequisite is
|
|
// fragments-only (no leading launch) and fragments can carry run:, so an
|
|
// uploaded fragment can declare both. With the blocks
|
|
// swapped, the un-acknowledged call gets the prerequisite notice back —
|
|
// sending the agent off to do manual device setup for a flow the very
|
|
// next call rejects. The contract error must win.
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "Signed into the demo account",
|
|
steps: [{ kind: "run", flow: "login.yaml" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const outcome = await createRunFlowTool(mockRegistry())
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
(r) => r,
|
|
(e: unknown) => e
|
|
);
|
|
// A rejection — not a { notice, executionPrerequisite } return: on a
|
|
// failure this prints the notice that leaked through instead.
|
|
expect(outcome).toBeInstanceOf(Error);
|
|
expect((outcome as Error).message).toContain('run: composition ("run: login.yaml")');
|
|
expect(getFailureSignal(outcome)?.failure_stage).toBe("flow_upload_run_composition");
|
|
});
|
|
|
|
it("rejects an uploaded flow quoting the first run:'s as-written directory-qualified path", async () => {
|
|
// The guard's quote is the author's remediation pointer: with two
|
|
// same-stem steps (`run: ios/login.yaml`, then `run: android/login.yaml`)
|
|
// the throw fires on the FIRST offender, and only the as-written
|
|
// directory says which of the two lines it means. Reducing the target to
|
|
// its basename stem (runTargetName — what the report attribution sites
|
|
// use, and what `step.flow` carried before targets kept their spelling)
|
|
// would print `run: login`, a string appearing nowhere in the flow and
|
|
// ambiguous between the two steps. The bare-spelling rejections above
|
|
// pin only the `.yaml` suffix (there the as-written path and the stem
|
|
// differ by nothing else), so this test is what holds the directory
|
|
// component — the contract content.ts pins for the MCP renderer
|
|
// ("`run ios/login.yaml` and `run android/login.yaml` must render
|
|
// distinctly, not as one stem"); this error path is that same reasoning
|
|
// on the tool-server surface.
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n - run: ios/login.yaml\n - run: android/login.yaml\n",
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
const err = await createRunFlowTool(registry)
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
() => null,
|
|
(e: unknown) => e
|
|
);
|
|
expect(err).toBeInstanceOf(Error);
|
|
expect((err as Error).message).toContain('run: composition ("run: ios/login.yaml")');
|
|
expect(getFailureSignal(err)?.failure_stage).toBe("flow_upload_run_composition");
|
|
// Preflight, not mid-run: nothing was dispatched to the device.
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("allows run: composition for a co-located flow_file resolved in place", async () => {
|
|
// The everyday co-located client: the flow_file boundary resolves the
|
|
// exact ${project_root}/.argent/flows/${name}.yaml path on a shared
|
|
// filesystem (presentOnHost, NOT an upload). This return is what carries
|
|
// the whole feature — misclassifying it as an upload would reject every
|
|
// local run: composition with the co-located contract error, so the
|
|
// upload-rejection tests above need this inverse pin.
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "composed fragment ran" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "login.yaml" }],
|
|
});
|
|
const flowFile = path.join(tmpDir, ".argent", "flows", "main.yaml");
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: flowFile, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: flowFile,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps[1]).toMatchObject({
|
|
kind: "echo",
|
|
status: "pass",
|
|
message: "composed fragment ran",
|
|
});
|
|
});
|
|
|
|
it("rejects a snapshot step when the root flow was uploaded (no durable baseline dir)", async () => {
|
|
// Baselines anchor beside the flow's file, and an uploaded flow
|
|
// materializes to a fresh temp directory each call — a plain snapshot can
|
|
// only fail on a missing baseline, and updateBaselines would write PNGs no
|
|
// later run can find. Rejected preflight, like uploaded run: composition.
|
|
vi.mocked(runSnapshot).mockClear();
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "echo", message: "before" },
|
|
{ kind: "snapshot", name: "home", maxMismatch: 0.5 },
|
|
],
|
|
}),
|
|
"utf8"
|
|
);
|
|
|
|
const registry = mockRegistry();
|
|
const err = await createRunFlowTool(registry)
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
() => null,
|
|
(e: unknown) => e
|
|
);
|
|
expect(err).toBeInstanceOf(Error);
|
|
expect((err as Error).message).toContain('snapshot step ("snapshot: home")');
|
|
expect(getFailureSignal(err)?.failure_stage).toBe("flow_upload_snapshot_baseline");
|
|
// Preflight, not mid-run: nothing was dispatched to the device and the
|
|
// differ was never pointed at the temp materialization dir.
|
|
expect(registry.invokeTool).not.toHaveBeenCalled();
|
|
expect(vi.mocked(runSnapshot)).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("rejects an uploaded flow whose snapshot hides behind a when: block that would not fire", async () => {
|
|
// Same preflight walk as run: — a guard-gated snapshot must not report
|
|
// green on one platform and only surface the contract error where the
|
|
// guard first fires (e.g. in CI).
|
|
const uploadedPath = path.join(tmpDir, "materialized-upload.yaml");
|
|
await fs.writeFile(
|
|
uploadedPath,
|
|
"steps:\n - when:\n platform: android\n steps:\n - snapshot:\n name: home\n",
|
|
"utf8"
|
|
);
|
|
|
|
const err = await createRunFlowTool(mockRegistry())
|
|
.execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: uploadedPath, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: "/client/.argent/flows/main.yaml",
|
|
presentOnHost: false,
|
|
viaUpload: true,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
.then(
|
|
() => null,
|
|
(e: unknown) => e
|
|
);
|
|
expect(getFailureSignal(err)?.failure_stage).toBe("flow_upload_snapshot_baseline");
|
|
});
|
|
|
|
it("allows a snapshot step for a co-located flow_file resolved in place", async () => {
|
|
// The inverse pin for the snapshot upload rejection above: the everyday
|
|
// co-located client (presentOnHost, NOT an upload) keeps its durable
|
|
// baseline directory beside the flow file, so the snapshot path still runs.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
});
|
|
const flowFile = path.join(tmpDir, ".argent", "flows", "main.yaml");
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, flow_file: flowFile, device: DEVICE },
|
|
{
|
|
artifacts: new ArtifactStore(),
|
|
fileInputs: {
|
|
flow_file: {
|
|
clientPath: flowFile,
|
|
presentOnHost: true,
|
|
viaUpload: false,
|
|
},
|
|
},
|
|
}
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(result.steps[0]).toMatchObject({ kind: "snapshot", status: "pass" });
|
|
expect(vi.mocked(runSnapshot)).toHaveBeenCalledWith(
|
|
expect.anything(),
|
|
expect.objectContaining({
|
|
flowsDir: path.join(tmpDir, ".argent", "flows"),
|
|
flowName: "main",
|
|
})
|
|
);
|
|
});
|
|
|
|
it("attributes a hard-stop-skipped run: step to its target stem, like every other path", async () => {
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "never runs" }],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: { android: "com.acme.app" } }, // DEVICE is iOS → errors
|
|
{ kind: "run", flow: "login.yaml" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "run:skip"]);
|
|
// Same attribution as an executed/errored run marker: the target stem,
|
|
// with the as-written path in target — not the enclosing flow.
|
|
expect(result.steps[1]).toMatchObject({ flow: "login", target: "login.yaml" });
|
|
});
|
|
|
|
it("applies the root-stem collision disambiguation on the hard-stop skip path too", async () => {
|
|
// Same setup as above, but the root shares the fragment's stem — the skip
|
|
// report must carry the same disambiguated name an executed run marker
|
|
// would, or attribution would depend on whether the step ever ran.
|
|
const helpersDir = path.join(tmpDir, ".argent", "flows", "helpers");
|
|
await fs.mkdir(helpersDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(helpersDir, "login.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "never runs" }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: { android: "com.acme.app" } }, // DEVICE is iOS → errors
|
|
{ kind: "run", flow: "helpers/login.yaml" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "login", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "run:skip"]);
|
|
expect(result.steps[1]).toMatchObject({ flow: "helpers/login", target: "helpers/login.yaml" });
|
|
});
|
|
|
|
it("attributes a fragment's when marker and skipped block steps to the fragment, not the root", async () => {
|
|
// A platform guard is static (no tree fetch) and android never matches the
|
|
// iOS DEVICE, so the block skips with the fragment on the run stack. The
|
|
// marker line (execWhenStep) and the authored-step skip line
|
|
// (reportBlockSkipped → stepFlow's non-run branch) each derive attribution
|
|
// separately — all of them must name the fragment, or the CLI's
|
|
// `[fragment]` suffix would tag these lines with the wrong flow.
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "echo", message: "inside fragment" },
|
|
{
|
|
kind: "when",
|
|
condition: { kind: "platform", platform: "android" },
|
|
steps: [{ kind: "echo", message: "android only" }],
|
|
},
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "frag.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"echo:pass",
|
|
"when:skip",
|
|
"echo:skip",
|
|
]);
|
|
// Executed echo, unmet-guard marker, and block-skip line: all "frag".
|
|
expect(result.steps[1]).toMatchObject({ flow: "frag", message: "inside fragment" });
|
|
expect(result.steps[2]).toMatchObject({ flow: "frag", depth: 1 });
|
|
expect(result.steps[3]).toMatchObject({ flow: "frag", depth: 2, message: "android only" });
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("attributes hard-stop skips inside a fragment to the fragment, and the root's own to the root", async () => {
|
|
// The fragment's leading launch declares no iOS app id, so it errors and
|
|
// hard-stops the run with the fragment still on the stack. Every post-stop
|
|
// skip line inside the fragment — plain step, when marker, and the when
|
|
// block's expansion — must stay attributed to the fragment, while the
|
|
// root's trailing step flips back to the root: attribution follows the
|
|
// run stack, not where the stop happened.
|
|
await writeFlow("frag", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: { android: "com.acme.app" } }, // DEVICE is iOS → errors
|
|
{ kind: "echo", message: "fragment echo" },
|
|
{
|
|
kind: "when",
|
|
condition: { kind: "platform", platform: "ios" },
|
|
steps: [{ kind: "echo", message: "guarded echo" }],
|
|
},
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "frag.yaml" },
|
|
{ kind: "echo", message: "root echo" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"launch:error",
|
|
"echo:skip",
|
|
"when:skip",
|
|
"echo:skip",
|
|
"echo:skip",
|
|
]);
|
|
expect(result.steps[2]).toMatchObject({ flow: "frag", message: "fragment echo" });
|
|
expect(result.steps[3]).toMatchObject({ flow: "frag", kind: "when" });
|
|
expect(result.steps[4]).toMatchObject({ flow: "frag", message: "guarded echo" });
|
|
expect(result.steps[5]).toMatchObject({ flow: "main", message: "root echo" });
|
|
expect(result.ok).toBe(false);
|
|
});
|
|
|
|
it("keys and stores a composed fragment's snapshot with the root flow", async () => {
|
|
const sharedDir = path.join(tmpDir, "shared");
|
|
await fs.mkdir(sharedDir, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedDir, "visual.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "../../shared/visual.yaml" }],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
// The snapshot ran from shared/visual.yaml but stays anchored to the ROOT
|
|
// flow: baselines under the root flow's directory, keyed by the root name.
|
|
expect(vi.mocked(runSnapshot)).toHaveBeenCalledWith(
|
|
expect.anything(),
|
|
expect.objectContaining({
|
|
flowsDir: path.join(tmpDir, ".argent", "flows"),
|
|
flowName: "main",
|
|
})
|
|
);
|
|
});
|
|
|
|
it("anchors a symlinked root flow's snapshot baselines beside the real file", async () => {
|
|
// .argent/flows/visual.yaml is a symlink to shared/flows/visual.yaml. The
|
|
// baseline anchor must be the real file's directory — the same canonical
|
|
// anchor `run:` targets resolve against — not the symlink's spelling, or
|
|
// one root file reached through two spellings would keep two baseline sets.
|
|
const sharedFlows = path.join(tmpDir, "shared", "flows");
|
|
await fs.mkdir(sharedFlows, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(sharedFlows, "visual.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.symlink(path.join(sharedFlows, "visual.yaml"), path.join(flowsDir, "visual.yaml"));
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "visual", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
// The real directory, not the spelling the run was addressed by. (The
|
|
// link and its target share a basename here, so this fixture cannot tell
|
|
// the two candidate KEYS apart — the vault tests below do that.)
|
|
expect(vi.mocked(runSnapshot)).toHaveBeenCalledWith(
|
|
expect.anything(),
|
|
expect.objectContaining({ flowsDir: sharedFlows, flowName: "visual" })
|
|
);
|
|
});
|
|
|
|
/**
|
|
* The baseline store is `<flowsDir>/__baselines__/<flowName>`, and flowsDir
|
|
* is the CANONICAL root flow's directory — so the key has to name the
|
|
* canonical file too, or the pair identifies no single file. These pin that
|
|
* agreement from both sides: two flows that share an anchor must not share a
|
|
* store, and a flow that is its own real file must not move.
|
|
*/
|
|
it("gives two projects' same-named symlinks into one vault separate baseline stores", async () => {
|
|
// The shared-vault layout: each project has its own legitimately named
|
|
// .argent/flows/smoke.yaml, both symlinked into one vault at differently
|
|
// named real files. The canonical anchor is vault/ for both, so keying by
|
|
// the as-written stem would give them ONE vault/__baselines__/smoke/:
|
|
// whichever project ran --update-baselines last would silently replace the
|
|
// other's reviewed, committed PNG — reported as "baseline updated", as
|
|
// though it had refreshed its own — and the other project's next run would
|
|
// then fail against a screen it never captured. Nothing is printed either
|
|
// way, so only the store paths can catch this.
|
|
const vault = path.join(tmpDir, "vault");
|
|
await fs.mkdir(vault, { recursive: true });
|
|
const snapshotFlow = serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
});
|
|
await fs.writeFile(path.join(vault, "a-smoke.yaml"), snapshotFlow, "utf8");
|
|
await fs.writeFile(path.join(vault, "b-smoke.yaml"), snapshotFlow, "utf8");
|
|
const vaultedProject = async (proj: string, target: string): Promise<string> => {
|
|
const flows = path.join(tmpDir, proj, ".argent", "flows");
|
|
await fs.mkdir(flows, { recursive: true });
|
|
await fs.symlink(path.join(vault, target), path.join(flows, "smoke.yaml"));
|
|
return path.join(tmpDir, proj);
|
|
};
|
|
const projA = await vaultedProject("projA", "a-smoke.yaml");
|
|
const projB = await vaultedProject("projB", "b-smoke.yaml");
|
|
|
|
vi.mocked(runSnapshot).mockClear();
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const a = asRun(
|
|
await runFlow.execute({}, { name: "smoke", project_root: projA, device: DEVICE })
|
|
);
|
|
const b = asRun(
|
|
await runFlow.execute({}, { name: "smoke", project_root: projB, device: DEVICE })
|
|
);
|
|
|
|
expect([a.ok, b.ok]).toEqual([true, true]);
|
|
// The caller-visible identity is untouched — both runs still report as the
|
|
// flow the user asked to run. Only the baseline key follows the real file.
|
|
expect([a.flow, b.flow]).toEqual(["smoke", "smoke"]);
|
|
|
|
const stores = vi
|
|
.mocked(runSnapshot)
|
|
.mock.calls.map(([, opts]) => path.join(opts.flowsDir, "__baselines__", opts.flowName));
|
|
expect(stores).toEqual([
|
|
path.join(vault, "__baselines__", "a-smoke"),
|
|
path.join(vault, "__baselines__", "b-smoke"),
|
|
]);
|
|
});
|
|
|
|
it("leaves a regular (non-symlinked) root flow's baseline store exactly where it was", async () => {
|
|
// The no-regression pin for every existing user: when the root flow is a
|
|
// real file its canonical stem IS the as-written one, so deriving the key
|
|
// from the canonical path must not move the store anyone has already
|
|
// seeded and committed under .argent/flows/__baselines__/<flow>/.
|
|
await writeFlow("checkout", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
});
|
|
|
|
vi.mocked(runSnapshot).mockClear();
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "checkout", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.ok).toBe(true);
|
|
const [, opts] = vi.mocked(runSnapshot).mock.calls[0]!;
|
|
expect(path.join(opts.flowsDir, "__baselines__", opts.flowName)).toBe(
|
|
path.join(tmpDir, ".argent", "flows", "__baselines__", "checkout")
|
|
);
|
|
});
|
|
|
|
it("keys a fragment's snapshot to a symlinked root's REAL file, not the fragment or the link", async () => {
|
|
// The root-anchoring pin above, extended to the symlinked-root case the
|
|
// key change touches: the snapshot is authored in vault/frag.yaml, so
|
|
// three names are in play — the fragment's stem ("frag"), the spelling the
|
|
// run was addressed by ("smoke"), and the root's real file ("a-smoke").
|
|
// Only the last agrees with the anchor the store sits in.
|
|
const vault = path.join(tmpDir, "vault");
|
|
await fs.mkdir(vault, { recursive: true });
|
|
await fs.writeFile(
|
|
path.join(vault, "a-smoke.yaml"),
|
|
serializeFlow({ executionPrerequisite: "", steps: [{ kind: "run", flow: "frag.yaml" }] }),
|
|
"utf8"
|
|
);
|
|
await fs.writeFile(
|
|
path.join(vault, "frag.yaml"),
|
|
serializeFlow({
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "snapshot", name: "home", maxMismatch: 0.5 }],
|
|
}),
|
|
"utf8"
|
|
);
|
|
const flowsDir = path.join(tmpDir, ".argent", "flows");
|
|
await fs.mkdir(flowsDir, { recursive: true });
|
|
await fs.symlink(path.join(vault, "a-smoke.yaml"), path.join(flowsDir, "smoke.yaml"));
|
|
|
|
vi.mocked(runSnapshot).mockClear();
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "smoke", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["run:pass", "snapshot:pass"]);
|
|
expect(vi.mocked(runSnapshot)).toHaveBeenCalledWith(
|
|
expect.anything(),
|
|
expect.objectContaining({ flowsDir: vault, flowName: "a-smoke" })
|
|
);
|
|
// Report attribution is unchanged by the key: the root still reports under
|
|
// the name it was run as, the expanded step under the fragment's.
|
|
expect(result.flow).toBe("smoke");
|
|
expect(result.steps[1]?.flow).toBe("frag");
|
|
});
|
|
|
|
it("stamps nesting depth on expanded fragment steps (omitted at top level)", async () => {
|
|
await writeFlow("inner", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "echo", message: "deepest" }],
|
|
});
|
|
await writeFlow("login", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "tool", name: "tap", args: { x: 0.5 } },
|
|
{ kind: "run", flow: "inner.yaml" },
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "run", flow: "login.yaml" },
|
|
{ kind: "echo", message: "done" },
|
|
],
|
|
});
|
|
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
|
|
// Each run marker sits at its enclosing depth; the fragment it expands runs
|
|
// one deeper. Top-level steps omit the field entirely, so a flow with no
|
|
// nesting steps reports byte-identically to the pre-depth shape.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.depth ?? 0}`)).toEqual([
|
|
"run:0",
|
|
"tool:1",
|
|
"run:1",
|
|
"echo:2",
|
|
"echo:0",
|
|
]);
|
|
expect(result.steps[0].depth).toBeUndefined();
|
|
expect(result.steps[4].depth).toBeUndefined();
|
|
});
|
|
|
|
it("expands a referenced e2e flow inline, launch step and all", async () => {
|
|
await writeFlow("other-e2e", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "echo", message: "in nested e2e" },
|
|
],
|
|
});
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "other-e2e.yaml" }],
|
|
});
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
// run marker, then the nested e2e's launch + echo expanded inline.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"run:pass",
|
|
"launch:pass",
|
|
"echo:pass",
|
|
]);
|
|
expect(result.steps[1].flow).toBe("other-e2e");
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("runs a nested flow-execute against the run device, not the recorded one (issue #607)", async () => {
|
|
// The raw `tool: flow-execute` form is what the recorder falls back to when
|
|
// the target is not a resolvable sibling — and what a remote recording always
|
|
// produces. Its device parameter is named `device`, which was not a bind key,
|
|
// so the sub-run drove the id baked in at record time. Here the flow carries
|
|
// a device that does not exist while the run is given a real one.
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{
|
|
kind: "tool",
|
|
name: "flow-execute",
|
|
args: { name: "b-only", project_root: "/elsewhere", device: "STALE-ID" },
|
|
},
|
|
],
|
|
});
|
|
|
|
// Single step + explicit device, per mockRegistry's contract.
|
|
const registry = mockRegistry({ name: {}, project_root: {}, device: {} });
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(registry.invokeTool).toHaveBeenCalledWith(
|
|
"flow-execute",
|
|
expect.objectContaining({ device: DEVICE })
|
|
);
|
|
expect(registry.invokeTool).not.toHaveBeenCalledWith(
|
|
"flow-execute",
|
|
expect.objectContaining({ device: "STALE-ID" })
|
|
);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("detects a cyclic run reference", async () => {
|
|
await writeFlow("a", { executionPrerequisite: "", steps: [{ kind: "run", flow: "b.yaml" }] });
|
|
await writeFlow("b", { executionPrerequisite: "", steps: [{ kind: "run", flow: "a.yaml" }] });
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "run", flow: "a.yaml" }],
|
|
});
|
|
const runFlow = createRunFlowTool(mockRegistry());
|
|
const result = asRun(
|
|
await runFlow.execute({}, { name: "main", project_root: tmpDir, device: DEVICE })
|
|
);
|
|
const errored = result.steps.find((s) => s.status === "error");
|
|
expect(errored?.reason).toMatch(/cyclic/i);
|
|
// The chain renders the human-readable stems, not canonical paths.
|
|
expect(errored?.reason).toContain("main → a → b → a");
|
|
// The error report carries the as-written path: on a failed resolution it
|
|
// is what locates the bad reference (`run ../shared/x.yaml`, not a bare
|
|
// stem), and stepLabel in argent-mcp falls back to the stem without it.
|
|
expect(errored?.target).toBe("a.yaml");
|
|
// The cycle is detected two fragments down; its error marker keeps that
|
|
// depth (the fail() path stamps depthOf(scope) like the success marker),
|
|
// so the error line renders inside the block that caused it.
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}:${s.depth ?? 0}`)).toEqual([
|
|
"run:pass:0",
|
|
"run:pass:1",
|
|
"run:error:2",
|
|
]);
|
|
});
|
|
|
|
it("executes a leading launch step from scratch (restart-app) and reports it", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "echo", message: "running" },
|
|
],
|
|
});
|
|
const registry = mockRegistry();
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:pass", "echo:pass"]);
|
|
// e2e contract: terminate + relaunch, so a running copy can't leak state.
|
|
expect(registry.invokeTool).toHaveBeenCalledWith("restart-app", { bundleId: "com.acme.app" });
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
it("errors the launch step when no app id is declared for the platform", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: { android: "com.acme.app" } }, // DEVICE is iOS
|
|
{ kind: "echo", message: "should never run" },
|
|
],
|
|
});
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "echo:skip"]);
|
|
expect(result.steps[0].reason).toMatch(/no app id declared for platform/i);
|
|
expect(result.ok).toBe(false);
|
|
});
|
|
|
|
it("errors the launch step when native devtools never connects on iOS", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "echo", message: "should never run" },
|
|
],
|
|
});
|
|
// Registry whose native-devtools service is unavailable: the launch step
|
|
// must fail rather than let selectors silently fall back to the AX tree.
|
|
// (An unresolvable service fails fast; a resolvable-but-never-connected
|
|
// one hits the same guard after the connect timeout.)
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => {
|
|
throw new Error("native-devtools unavailable");
|
|
}),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "echo:skip"]);
|
|
expect(result.steps[0].reason).toMatch(/could not connect to native devtools/i);
|
|
// Every reason names the bundle id, so the prefix must not — doubled, it
|
|
// reads as two separate failures. Asserted here and on the measured and
|
|
// system-app shapes below, since the rule is worth nothing partially held.
|
|
expect(result.steps[0].reason?.match(/com\.acme\.app/g)).toHaveLength(1);
|
|
// Resolution fails for reasons the flow author can act on and cannot
|
|
// otherwise see — a socket already bound, a device of the wrong platform —
|
|
// and the step's reason is the only place any of them surfaces.
|
|
expect(result.steps[0].reason).toContain("native-devtools unavailable");
|
|
expect(result.ok).toBe(false);
|
|
});
|
|
|
|
// An Apple system app may never load the dylib (a platform binary with
|
|
// library validation), so its hierarchy may never become readable — which is
|
|
// not a reason to fail the LAUNCH. The step started the app, and a flow that
|
|
// taps by coordinate needs nothing else; the impossibility belongs where a
|
|
// selector actually needs the hierarchy.
|
|
it("lets a system-app launch through so a coordinate-driven flow still runs", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:pass", "tool:pass"]);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
// A blocked precheck comes back RESOLVED, and returns before the terminate and
|
|
// the launch — so this is a step whose app was never started. The gate's
|
|
// remedies are all written for one that was: read as a launch, `not_running`
|
|
// becomes "it exited after launch", which sends the author after a crash that
|
|
// never happened and drops the message that named the real cause.
|
|
it("fails a launch whose restart-app was blocked before it started anything", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) => {
|
|
if (id === "list-devices") return { devices: [] };
|
|
if (id === "restart-app") {
|
|
return {
|
|
status: "init_failed",
|
|
attempts: 3,
|
|
message:
|
|
`Native devtools failed to initialize for ${DEVICE} after 3 attempts. ` +
|
|
`Last error: Native devtools dylib not found: /x/libArgentInjectionBootstrap.dylib. ` +
|
|
`Try shutting down and re-booting the simulator, or restart CoreSimulatorService.`,
|
|
};
|
|
}
|
|
return { ok: true };
|
|
}),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
// What the gate would measure for an app nothing launched.
|
|
appConnectionState: async () => "not_running" as const,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:error", "tool:skip"]);
|
|
expect(result.steps[0].reason).toContain("dylib not found");
|
|
expect(result.steps[0].reason).not.toContain("exited after launch");
|
|
});
|
|
|
|
// The connected case above never reaches the gate's verdict. This is the one
|
|
// that matters: a system app that never connects at all. The launch must STILL
|
|
// pass, where every measured state would have failed it — and failed it with a
|
|
// remedy that cannot apply to such an app.
|
|
it("passes a system-app launch that never connects at all", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
// The launchd env carrying the bootstrap dylib is simulator-wide, so a
|
|
// system app's process inherits the injection tokens and scores as a
|
|
// live app the service merely never registered.
|
|
appConnectionState: async () => "unregistered" as const,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"launch:pass",
|
|
"tool:pass",
|
|
]);
|
|
expect(result.ok).toBe(true);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
// The verdict is withheld BEFORE the state is measured: that measurement is
|
|
// several uninterruptible simctl round-trips, and no arm consults it for such
|
|
// an app — pure latency on every system-app launch step.
|
|
it("does not measure a state it will not report", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "launch", app: "com.apple.Preferences" }],
|
|
});
|
|
const appConnectionState = vi.fn(async () => "unregistered" as const);
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps[0].status).toBe("pass");
|
|
expect(appConnectionState).not.toHaveBeenCalled();
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
// The gate consults the service before withholding its verdict, so an
|
|
// unresolvable service was the one remaining way a system-app launch could
|
|
// fail — on a failure irrelevant to an app it was never going to serve.
|
|
it("passes a system-app launch even when native-devtools cannot resolve", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => {
|
|
throw new Error("listen EADDRINUSE: address already in use /tmp/argent-nd-00000000.sock");
|
|
}),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual(["launch:pass", "tool:pass"]);
|
|
expect(result.ok).toBe(true);
|
|
});
|
|
|
|
// The control for the test above: an injectable app in the identical state
|
|
// must still fail, or the pass-through would be excusing every service
|
|
// failure rather than the one it reasons about.
|
|
it("still fails an injectable launch when native-devtools cannot resolve", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "tool", name: "gesture-tap", args: { x: 0.5, y: 0.35 } },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => {
|
|
throw new Error("listen EADDRINUSE: address already in use /tmp/argent-nd-00000000.sock");
|
|
}),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps[0].status).toBe("error");
|
|
expect(result.steps[0].reason).toMatch(/native-devtools service is unavailable/);
|
|
});
|
|
|
|
// The other half of letting the launch through: a SELECTOR step against the
|
|
// same app has to say why it cannot resolve, terminally. Also the only
|
|
// end-to-end proof that the launched bundle id reaches the tree source —
|
|
// without it the author gets the stock "Launch or restart the app first"
|
|
// auto-target text, the restart loop this measurement exists to break.
|
|
it("gives a selector step against a system app the terminal reason, not the auto-target text", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "assert", selector: { text: "General" }, condition: "visible" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps[0].status).toBe("pass");
|
|
const reason = result.steps[1].reason ?? "";
|
|
expect(reason).toMatch(/Apple system app/);
|
|
expect(reason).toMatch(/com\.apple\.Preferences/);
|
|
// The auto-target text is what auto-resolution alone can produce here, and
|
|
// its remedy is the loop.
|
|
expect(reason).not.toMatch(/auto-targeting/);
|
|
expect(reason).not.toMatch(/Launch or restart the app first/);
|
|
});
|
|
|
|
// `assert`/`await` read the tree through `waitForCondition`; `tap`, `type`,
|
|
// `scroll-to` and `long-press` reach it through `settleTree`, a separate call
|
|
// site. Pinning only the first left the id droppable at the second — the one
|
|
// every action directive uses — with the suite green.
|
|
it("threads the launched id to the settleTree read an action directive uses", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tap", selector: { text: "General" } },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const reason = result.steps[1].reason ?? "";
|
|
expect(reason).toMatch(/Apple system app/);
|
|
expect(reason).not.toMatch(/Launch or restart the app first/);
|
|
});
|
|
|
|
// `launch-app` / `restart-app` are in FOREGROUND_CHANGING_TOOLS, which drops
|
|
// the launched id — but unlike the rest of that set they NAME the app they
|
|
// switched to. Dropping it there sends every read from that step onward back
|
|
// to the auto-target text, in the middle of a run that has a launched app.
|
|
it.each(["restart-app", "launch-app"])(
|
|
"keeps the launched id across a %s tool step that names the same app",
|
|
async (name) => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name, args: { bundleId: "com.apple.Preferences" } },
|
|
{ kind: "assert", selector: { text: "General" }, condition: "visible" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const reason = result.steps[2].reason ?? "";
|
|
expect(reason).toMatch(/Apple system app/);
|
|
expect(reason).not.toMatch(/Launch or restart the app first/);
|
|
}
|
|
);
|
|
|
|
// The control that keeps the restore scoped: `button` can send the app to the
|
|
// background with nothing naming what replaced it, so there the id really is
|
|
// unknown and auto-targeting is the honest fallback.
|
|
it("still drops the launched id across a foreground-changing tool that names no app", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.apple.Preferences" },
|
|
{ kind: "tool", name: "button", args: { name: "home" } },
|
|
{ kind: "assert", selector: { text: "General" }, condition: "visible" },
|
|
],
|
|
});
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(mockRegistry()).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
expect(result.steps[2].reason ?? "").toMatch(/Launch or restart the app first/);
|
|
});
|
|
|
|
// The measured half says what is wrong with the app; without this half a flow
|
|
// author is told to restart a tool-server with no mention that a selector
|
|
// needed a hierarchy.
|
|
it("says why the tree was being read, not just what is wrong with the app", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "assert", selector: { text: "General" }, condition: "visible" },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => true,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => "unregistered" as const,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
const result = asRun(
|
|
await createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
)
|
|
);
|
|
|
|
const reason = result.steps[1].reason ?? "";
|
|
expect(reason).toMatch(/argent server stop && argent server start --detach/);
|
|
expect(reason).toMatch(/Flows resolve selectors against the full view hierarchy/);
|
|
});
|
|
|
|
// Replacing the whole diagnosis with a literal left every flow test green.
|
|
// `unregistered` is the case that matters — the remedy inverts, and this gate
|
|
// runs right after a launch, so "re-run to relaunch" is the loop one level up.
|
|
it("reports the measured reason when the connection times out", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "launch", app: "com.acme.app" }],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => "unregistered" as const,
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
// Leave setImmediate real: the run reads the flow off disk between sleeps,
|
|
// and that I/O needs actual event-loop turns. Pumping one between advances
|
|
// elapses the 1.5 s settle and 8 s connect wait in fake time.
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps[0].status).toBe("error");
|
|
expect(result.steps[0].reason).toContain(
|
|
"argent server stop && argent server start --detach"
|
|
);
|
|
expect(result.steps[0].reason).not.toMatch(/restart-app/);
|
|
expect(result.steps[0].reason?.match(/com\.acme\.app/g)).toHaveLength(1);
|
|
// This gate is the one caller whose wait was bounded: a cold start slower
|
|
// than LAUNCH_TO_VERDICT_MS reads identically to a genuinely unregistered
|
|
// app, so "restarting cannot help" must not be the last word here.
|
|
expect(result.steps[0].reason).toMatch(/cold start/i);
|
|
expect(result.steps[0].reason).toMatch(/re-run the flow/i);
|
|
// The figure sizing that judgement is the whole wait the step performed:
|
|
// the poll reads the live map once before its first sleep, so the
|
|
// post-launch settle counts and quoting the poll alone understates it.
|
|
expect(result.steps[0].reason).toContain(`${LAUNCH_TO_VERDICT_MS} ms`);
|
|
expect(result.steps[0].reason).not.toContain(`${NATIVE_READY_TIMEOUT_MS} ms`);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
// `buildAppStateMessage` is written for a reader who has not launched
|
|
// anything, so "call launch-app (or restart-app)" is the missing step there
|
|
// and the step's own action here — emitted verbatim it hands the flow author
|
|
// back the relaunch that just ran, which re-runs into the identical state.
|
|
//
|
|
// Exercised directly rather than through the runner: it is a pure function of
|
|
// (bundleId, state), and each runner pass costs ~9 s of pumped fake time. The
|
|
// end-to-end test above proves the gate reaches it at all.
|
|
describe("flowLaunchGateReason", () => {
|
|
it.each([
|
|
// `not_running` is the sharpest: a relaunch is what produced the state
|
|
// being reported, so the remedy must not be one.
|
|
["not_running", /exited after launch/i],
|
|
["stale_process", /re-run the flow to launch again/i],
|
|
["indeterminate", /at most once more/i],
|
|
["connecting", /started after the step's own launch/i],
|
|
["unregistered", /cold start/i],
|
|
] as const)("tells %s something the step has not already done or tried", (state, expected) => {
|
|
expect(flowLaunchGateReason("com.acme.app", state)).toMatch(expected);
|
|
});
|
|
|
|
// The one state whose tool-surface remedy IS the launch step's own action.
|
|
// `launch-app` may still appear — the arm suggests starting it by hand to
|
|
// watch it die — but never as the prescribed retry.
|
|
it("does not tell a crash-on-launch app to simply launch itself again", () => {
|
|
const reason = flowLaunchGateReason("com.acme.app", "not_running");
|
|
|
|
expect(reason).not.toMatch(/Call launch-app/);
|
|
expect(reason).not.toMatch(/restart-app/);
|
|
// What it must say instead: the process is gone because it exited, and
|
|
// re-running reproduces exactly that.
|
|
expect(reason).toMatch(/exited after launch/);
|
|
expect(reason).toMatch(/Re-running the flow repeats the same launch/);
|
|
});
|
|
|
|
// The `not.toContain` guards below compare rendered substrings, so they only
|
|
// mean anything while neither figure is a suffix of the other: 500/1500 would
|
|
// make `"1500 ms".includes("500 ms")` true and fail them for the wrong
|
|
// reason. Pin what those guards need, not merely the ordering.
|
|
it("renders a launch-to-verdict spend the connect wait cannot be read into", () => {
|
|
expect(LAUNCH_TO_VERDICT_MS).toBeGreaterThan(NATIVE_READY_TIMEOUT_MS);
|
|
expect(`${LAUNCH_TO_VERDICT_MS} ms`).not.toContain(`${NATIVE_READY_TIMEOUT_MS} ms`);
|
|
});
|
|
|
|
// `stale_process` has two producers, and one carries THIS endpoint — it
|
|
// predates the listener, not the launchd environment. Blaming the
|
|
// environment would contradict the measured text this wraps.
|
|
it("does not blame the launchd environment for a stale process", () => {
|
|
const reason = flowLaunchGateReason("com.acme.app", "stale_process");
|
|
|
|
expect(reason).toMatch(/predates whatever the relaunch would have given it/);
|
|
// One producer carries THIS endpoint and is merely older than the
|
|
// listener, so the environment is not at fault on a first landing. The
|
|
// message may name it only for a REPEAT, which rules that producer out —
|
|
// so the blame must be conditional AND follow the re-run, both asserted.
|
|
const blame = reason.indexOf("launchd environment");
|
|
const rerun = reason.indexOf("re-run the flow");
|
|
expect(rerun).toBeGreaterThanOrEqual(0);
|
|
expect(blame === -1 || blame > rerun).toBe(true);
|
|
// Unconditional on purpose: `if (blame !== -1)` let the whole
|
|
// repeat-landing clause be deleted with the suite green, leaving a reader
|
|
// who lands here twice with "re-run the flow" and no escape. The wording is
|
|
// pinned too — "It lands here twice, so …" states on a FIRST landing what
|
|
// only a second one supports.
|
|
expect(reason).toMatch(/If it lands here twice, the simulator's launchd environment/);
|
|
});
|
|
|
|
it.each(["not_running", "connecting", "unregistered"] as const)(
|
|
"quotes the whole launch-to-verdict spend in the %s remedy",
|
|
(state) => {
|
|
const reason = flowLaunchGateReason("com.acme.app", state);
|
|
|
|
expect(reason).toContain(`${LAUNCH_TO_VERDICT_MS} ms`);
|
|
expect(reason).not.toContain(`${NATIVE_READY_TIMEOUT_MS} ms`);
|
|
}
|
|
);
|
|
});
|
|
|
|
// The connection can land between the final poll and the measurement. Without
|
|
// the `connected` short-circuit, buildAppStateMessage falls off its switch and
|
|
// a healthy run dies with a literal "undefined" as its reason.
|
|
it("passes when the connection lands between the last poll and the measurement", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [{ kind: "launch", app: "com.acme.app" }],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false, // never connects during the poll…
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => "connected" as const, // …but has by the measurement
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps[0].status).toBe("pass");
|
|
expect(result.ok).toBe(true);
|
|
// Without the short-circuit the step errors with a literal "undefined" —
|
|
// buildAppStateMessage has no `connected` arm to fall to.
|
|
expect(result.steps[0].reason ?? "").not.toMatch(/undefined/);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
// A rejected measurement must not propagate: `runLaunch` has no try/catch, so
|
|
// it would escape `execute()` and the run would return no step reports at all
|
|
// instead of a structured failure.
|
|
it("keeps a rejected measurement inside the step report", async () => {
|
|
await writeFlow("main", {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch", app: "com.acme.app" },
|
|
{ kind: "echo", message: "after" },
|
|
],
|
|
});
|
|
const registry = {
|
|
invokeTool: vi.fn(async (id: string) =>
|
|
id === "list-devices" ? { devices: [] } : { ok: true }
|
|
),
|
|
getTool: vi.fn(() => undefined),
|
|
resolveService: vi.fn(async () => ({
|
|
isConnected: () => false,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => {
|
|
throw new Error("Invalid device: UDID");
|
|
},
|
|
})),
|
|
} as unknown as Registry;
|
|
|
|
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
|
|
try {
|
|
const pending = createRunFlowTool(registry).execute(
|
|
{},
|
|
{ name: "main", project_root: tmpDir, device: DEVICE }
|
|
);
|
|
let settled = false;
|
|
void pending.then(() => (settled = true));
|
|
for (let i = 0; i < PUMP_LIMIT && !settled; i++) {
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
await vi.advanceTimersByTimeAsync(250);
|
|
}
|
|
const result = asRun(await pending);
|
|
|
|
expect(result.steps.map((s) => `${s.kind}:${s.status}`)).toEqual([
|
|
"launch:error",
|
|
"echo:skip",
|
|
]);
|
|
expect(result.steps[0].reason).toContain("could not be inspected");
|
|
expect(result.steps[0].reason).not.toMatch(/Invalid device/);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
});
|
|
|
|
describe("device binding (portability)", () => {
|
|
const reg = (props: Record<string, unknown>) =>
|
|
({ getTool: () => ({ inputSchema: { properties: props } }) }) as unknown as Registry;
|
|
|
|
it("the resolved device wins over a stale stored udid", () => {
|
|
const out = bindDeviceArgs(reg({ udid: {}, x: {}, y: {} }), "gesture-tap", "RESOLVED", {
|
|
udid: "STALE",
|
|
x: 0.5,
|
|
y: 0.5,
|
|
});
|
|
expect(out).toEqual({ udid: "RESOLVED", x: 0.5, y: 0.5 });
|
|
});
|
|
|
|
it("drops a device id entirely for a tool that doesn't declare one", () => {
|
|
const out = bindDeviceArgs(reg({ foo: {} }), "x", "R", { device_id: "STALE", foo: 1 });
|
|
expect(out).toEqual({ foo: 1 });
|
|
});
|
|
|
|
it("stripDeviceKeys removes udid / device_id / device, leaving other args untouched", () => {
|
|
expect(stripDeviceKeys({ udid: "A", device_id: "B", device: "C", x: 1 })).toEqual({ x: 1 });
|
|
});
|
|
|
|
it("stripDeviceKeys KEEPS a `devices` scope, because dropping it changes the step's meaning", () => {
|
|
// A target is stripped so the flow points at no device. A scope is not:
|
|
// `stop-all-simulator-servers` with no `devices` is the machine-wide sweep,
|
|
// so stripping it would record a correctly scoped teardown as a bare step
|
|
// that reaps every device on the machine when hand-run from the YAML — the
|
|
// manual-execution strategy the create-flow skill documents. Replay rebinds
|
|
// it either way (see bindDeviceArgs below).
|
|
expect(stripDeviceKeys({ udid: "A", devices: ["D", "E"], x: 1 })).toEqual({
|
|
devices: ["D", "E"],
|
|
x: 1,
|
|
});
|
|
});
|
|
|
|
it("bindDeviceArgs keeps a recorded scope when the run resolved NO device", () => {
|
|
// A cleanup flow resolves no device when none is unambiguous. Dropping the
|
|
// recorded scope there would widen the teardown from the devices the
|
|
// recording named to every device on the machine — the one direction that
|
|
// costs another agent their session. There is no run target to override.
|
|
expect(
|
|
bindDeviceArgs(reg({ devices: {} }), "stop-all-simulator-servers", "", {
|
|
devices: ["RECORDED"],
|
|
})
|
|
).toEqual({ devices: ["RECORDED"] });
|
|
});
|
|
|
|
it("bindDeviceArgs never forwards a scope to a tool that does not declare it", () => {
|
|
// The schema-blind strip's job: a `.strict()` schema would reject the call.
|
|
expect(
|
|
bindDeviceArgs(reg({ port: {} }), "stop-metro", "RESOLVED", {
|
|
devices: ["RECORDED"],
|
|
port: 8081,
|
|
})
|
|
).toEqual({ port: 8081 });
|
|
});
|
|
|
|
it("rebinds a nested flow-execute onto the run device (issue #607)", () => {
|
|
// `flow-execute`'s own device parameter is named `device`, so before it was
|
|
// a bind key a recorded nested step kept the id it was recorded on and the
|
|
// sub-run drove that device instead of the one the replay was given.
|
|
const out = bindDeviceArgs(
|
|
reg({ name: {}, project_root: {}, device: {} }),
|
|
"flow-execute",
|
|
"RESOLVED",
|
|
{ name: "b", project_root: "/p", device: "STALE" }
|
|
);
|
|
expect(out).toEqual({ name: "b", project_root: "/p", device: "RESOLVED" });
|
|
});
|
|
|
|
it("leaves `platform` alone", () => {
|
|
// Deliberate, and pinned here so a later "symmetry" edit fails loudly. The
|
|
// strip is schema-blind, and `platform` is not device-specific on every tool
|
|
// — react-profiler-analyze declares its own — so stripping it would silently
|
|
// retarget an unrelated recorded step. It is also inert once `device` is
|
|
// bound, because device resolution returns before it is ever read.
|
|
expect(stripDeviceKeys({ platform: "android", x: 1 })).toEqual({ platform: "android", x: 1 });
|
|
});
|
|
|
|
it("injects devices: [resolvedId] for a tool that declares it in its schema", () => {
|
|
// stop-all-simulator-servers' `devices` is a scope, not a single-device
|
|
// target, but it names the recording host's device ids the same way `udid`
|
|
// does — so it gets the same schema-aware rebind, as a one-element list.
|
|
const out = bindDeviceArgs(reg({ devices: {} }), "stop-all-simulator-servers", "RESOLVED", {});
|
|
expect(out).toEqual({ devices: ["RESOLVED"] });
|
|
});
|
|
|
|
it("does not invent a devices key for a tool that doesn't declare it", () => {
|
|
const out = bindDeviceArgs(reg({ foo: {} }), "x", "RESOLVED", { foo: 1 });
|
|
expect(out).toEqual({ foo: 1 });
|
|
expect(out).not.toHaveProperty("devices");
|
|
});
|
|
|
|
it("replaces a stale recorded devices list when the caller NAMED the run device", () => {
|
|
// An explicit `device` is the caller saying which device this run is about,
|
|
// so retargeting the teardown at it is what they asked for — and a flow
|
|
// recorded on one host must not carry that host's ids forward.
|
|
const out = bindDeviceArgs(
|
|
reg({ devices: {} }),
|
|
"stop-all-simulator-servers",
|
|
"RESOLVED",
|
|
{ devices: ["OLD-HOST-ID", "OTHER"] },
|
|
true
|
|
);
|
|
expect(out).toEqual({ devices: ["RESOLVED"] });
|
|
});
|
|
|
|
it("keeps a recorded scope when the run device was only auto-detected", () => {
|
|
// The destructive direction: the flow named one device, exactly one other
|
|
// happens to be booted, and replay would reap THAT one — a device nobody in
|
|
// this run ever named, quite possibly another agent's. This is the
|
|
// cross-agent teardown the `devices` scope exists to prevent, so the
|
|
// recorded ids stand; on another host they reap nothing and come back in
|
|
// `unmatched`, which is the safe direction and a legible one.
|
|
const out = bindDeviceArgs(reg({ devices: {} }), "stop-all-simulator-servers", "AUTO", {
|
|
devices: ["RECORDED-HOST"],
|
|
});
|
|
expect(out).toEqual({ devices: ["RECORDED-HOST"] });
|
|
});
|
|
|
|
it("still narrows an UNSCOPED recorded sweep onto an auto-detected device", () => {
|
|
// Nothing recorded means the step is the machine-wide sweep, so binding can
|
|
// only narrow it. That is why a cleanup flow resolves a device at all.
|
|
const out = bindDeviceArgs(reg({ devices: {} }), "stop-all-simulator-servers", "AUTO", {});
|
|
expect(out).toEqual({ devices: ["AUTO"] });
|
|
});
|
|
|
|
it("binds a scalar and a list device key together when a tool declares both", () => {
|
|
const out = bindDeviceArgs(
|
|
reg({ udid: {}, devices: {} }),
|
|
"hypothetical-tool",
|
|
"RESOLVED",
|
|
{ udid: "STALE", devices: ["OLD"] },
|
|
true
|
|
);
|
|
expect(out).toEqual({ udid: "RESOLVED", devices: ["RESOLVED"] });
|
|
});
|
|
|
|
it("rebinds the TARGET but not the recorded SCOPE on an auto-detected device", () => {
|
|
// The two keys part company here: a stale `udid` must never survive (the
|
|
// step would drive the wrong device), while a stale `devices` must never be
|
|
// retargeted (the step would destroy the wrong device).
|
|
const out = bindDeviceArgs(reg({ udid: {}, devices: {} }), "hypothetical-tool", "AUTO", {
|
|
udid: "STALE",
|
|
devices: ["OLD"],
|
|
});
|
|
expect(out).toEqual({ udid: "AUTO", devices: ["OLD"] });
|
|
});
|
|
});
|
|
|
|
describe("flow validation", () => {
|
|
it("rejects an e2e flow that declares executionPrerequisite", () => {
|
|
expect(() =>
|
|
parseFlow("executionPrerequisite: nope\nsteps:\n - launch: com.acme.app\n")
|
|
).toThrow(/must not declare executionPrerequisite/i);
|
|
});
|
|
|
|
it("a leading echo does not hide the launch step from the e2e check", () => {
|
|
expect(() =>
|
|
parseFlow(
|
|
"executionPrerequisite: nope\nsteps:\n - echo: starting\n - launch: com.acme.app\n"
|
|
)
|
|
).toThrow(/must not declare executionPrerequisite/i);
|
|
});
|
|
|
|
it("rejects a path-unsafe snapshot name (no traversal into baseline path)", () => {
|
|
expect(() => parseFlow("steps:\n - snapshot:\n name: ../../etc/evil\n")).toThrow(
|
|
/snapshot name/i
|
|
);
|
|
});
|
|
|
|
it("completes a bare run: name to <name>.yaml", () => {
|
|
// The compatibility spelling for flows written when a run: target was a
|
|
// saved-flow name. Completed at PARSE, so one spelling reaches the runner.
|
|
expect(parseFlow("steps:\n - run: login\n").steps[0]).toEqual({
|
|
kind: "run",
|
|
flow: "login.yaml",
|
|
});
|
|
});
|
|
|
|
it("completes an extensionless run: path, keeping its directory prefix", () => {
|
|
for (const [written, completed] of [
|
|
["shared/login", "shared/login.yaml"],
|
|
["../shared/login", "../shared/login.yaml"],
|
|
["./login", "./login.yaml"],
|
|
]) {
|
|
expect(parseFlow(`steps:\n - run: ${written}\n`).steps[0], written).toEqual({
|
|
kind: "run",
|
|
flow: completed,
|
|
});
|
|
}
|
|
});
|
|
|
|
it("does not complete a run: target the extension cannot rescue", () => {
|
|
// basename() strips a trailing slash, so a completion driven by the
|
|
// SUPPLIED basename would turn "shared/" into the unopenable "shared/.yaml"
|
|
// instead of reporting a target that names no file.
|
|
expect(() => parseFlow("steps:\n - run: shared/\n")).toThrow(/must end in \.yaml/);
|
|
expect(() => parseFlow("steps:\n - run: shared/\n")).not.toThrow(/\.yaml\.yaml/);
|
|
// A wrong extension is a typo to report, not a stem to extend: completing
|
|
// it would run "login.yml.yaml" and bury the mistake.
|
|
expect(() => parseFlow("steps:\n - run: login.yml\n")).toThrow(/must end in \.yaml/);
|
|
// Dots are outside the flow-name charset, so a version-ish stem stays an
|
|
// error rather than becoming "v1.2.yaml".
|
|
expect(() => parseFlow("steps:\n - run: v1.2\n")).toThrow(/must end in \.yaml/);
|
|
});
|
|
|
|
it("reports a missing target instead of a null.yaml completion for a valueless run:", () => {
|
|
for (const body of ["", " ~", " null"]) {
|
|
expect(() => parseFlow(`steps:\n - run:${body}\n`)).toThrow(/`run` has no target/);
|
|
expect(() => parseFlow(`steps:\n - run:${body}\n`)).not.toThrow(/null\.yaml/);
|
|
}
|
|
});
|
|
|
|
it("rejects a non-string run: scalar instead of migrating it to a filename", () => {
|
|
expect(() => parseFlow("steps:\n - run: true\n")).toThrow(/must be a YAML path string/);
|
|
expect(() => parseFlow("steps:\n - run: true\n")).not.toThrow(/true\.yaml/);
|
|
expect(() => parseFlow("steps:\n - run: 123\n")).toThrow(/must be a YAML path string/);
|
|
expect(() => parseFlow("steps:\n - run: 123\n")).not.toThrow(/123\.yaml/);
|
|
});
|
|
|
|
it("rejects a run: path without the .yaml extension", () => {
|
|
expect(() => parseFlow("steps:\n - run: flows/login.yml\n")).toThrow(/must end in \.yaml/);
|
|
// A correctly-typed empty string is a string, so it still reaches the
|
|
// extension check — this boundary is what keeps the non-string rejection
|
|
// above targeted at YAML scalars rather than at "no path here" generally.
|
|
expect(() => parseFlow('steps:\n - run: ""\n')).toThrow(/must end in \.yaml/);
|
|
});
|
|
|
|
it("names the lowercase requirement when only the run: extension's case is wrong", () => {
|
|
expect(() => parseFlow("steps:\n - run: shared/Login.YAML\n")).toThrow(
|
|
/lowercase \.yaml extension/
|
|
);
|
|
});
|
|
|
|
it("rejects an absolute run: path", () => {
|
|
expect(() => parseFlow("steps:\n - run: /anywhere/login.yaml\n")).toThrow(/must be relative/);
|
|
expect(() => parseFlow("steps:\n - run: C:/anywhere/login.yaml\n")).toThrow(
|
|
/must be relative/
|
|
);
|
|
// Drive-relative: not isAbsolute, but resolves against the drive's cwd.
|
|
expect(() => parseFlow("steps:\n - run: C:foo/login.yaml\n")).toThrow(/must be relative/);
|
|
});
|
|
|
|
it("rejects a run: path whose filename is not a safe flow name", () => {
|
|
expect(() => parseFlow("steps:\n - run: ../we!rd.yaml\n")).toThrow(/letters, digits/);
|
|
});
|
|
|
|
it("cites a pattern a rejected .yaml filename could actually match", () => {
|
|
// `my.flow.yaml` IS a .yaml file, so citing the dot-free flow-name pattern
|
|
// would demand a regex forbidding the extension the same check requires.
|
|
let message = "";
|
|
try {
|
|
parseFlow("steps:\n - run: my.flow.yaml\n");
|
|
} catch (err) {
|
|
message = err instanceof Error ? err.message : String(err);
|
|
}
|
|
expect(message).toContain(String(FLOW_FILE_NAME_PATTERN));
|
|
expect(message).not.toContain(String(FLOW_NAME_PATTERN));
|
|
expect(message).toMatch(/letters, digits, underscore, hyphen before the \.yaml/);
|
|
});
|
|
|
|
it("rejects backslashes in a run: path", () => {
|
|
expect(() => parseFlow("steps:\n - run: frag\\login.yaml\n")).toThrow(/forward slashes/);
|
|
});
|
|
|
|
it("round-trips the new step kinds through YAML", () => {
|
|
const flow = {
|
|
executionPrerequisite: "",
|
|
steps: [
|
|
{ kind: "launch" as const, app: "com.acme.app" },
|
|
// Text-only selectors serialize to bare strings, which parse back loose.
|
|
{ kind: "tap" as const, selector: { text: "Login", loose: true } },
|
|
{ kind: "tap" as const, x: 0.5, y: 0.57 },
|
|
{ kind: "type" as const, into: { identifier: "email" }, text: "a@b.com" },
|
|
{
|
|
kind: "assert" as const,
|
|
condition: "visible" as const,
|
|
selector: { text: "Welcome", loose: true },
|
|
},
|
|
{ kind: "snapshot" as const, name: "home", maxMismatch: 0.5 },
|
|
{ kind: "run" as const, flow: "../shared/login.yaml" },
|
|
// Mid-flow relaunch with a per-platform map.
|
|
{ kind: "launch" as const, app: { ios: "com.acme.app", android: "com.acme.android" } },
|
|
],
|
|
};
|
|
const parsed = parseFlow(serializeFlow(flow));
|
|
expect(parsed.steps).toEqual(flow.steps);
|
|
});
|
|
|
|
it("rejects a launch step with an invalid body", () => {
|
|
// An unrecognized platform key is named (strict unknown-key rejection)…
|
|
expect(() => parseFlow("steps:\n - launch: { web: foo }\n")).toThrow(
|
|
/launch has unknown key `web`/
|
|
);
|
|
// …while a non-map, non-string body still gets the shape error.
|
|
expect(() => parseFlow("steps:\n - launch: 42\n")).toThrow(/launch needs an app id/i);
|
|
});
|
|
});
|