mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
fix/flow-dir-export-throw
27 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jakub Piasecki
|
d4a712c401 |
refactor(gesture): rename the momentum-free flag to momentum (#732)
`settle: true` collided with the runner's other `settle` — waiting for the tree to quiesce — which is the very thing a swipe step does after every dispatch, whichever way the flag is set. One sentence of the flow skill used both senses at once. This names the physics instead: `momentum: false` is the momentum-free gesture, and the natural fling is the default the prose already described. ### Renamed | surface | before | after | | --- | --- | --- | | `swipe` flow directive | `settle: true` | `momentum: false` | | `gesture-swipe` param | `settle` | `momentum` | | `gesture-drag` param | `settle` | `momentum` | The directive passes the flag straight through to the tools, so all three move together. `momentum: true` is the default and normalizes to absent, mirroring what `settle: false` did — parse/serialize stay exact inverses. ### Also - Recording summary label `(settle)` → `(momentum-free)` - Internal constants → `MOMENTUM_FREE_EASE_EXPONENT`, `MOMENTUM_FREE_MIN_STEPS` - `flow-run` tool description and `argent-create-flow` skill updated ### Testing `tsc -b`, eslint (`--max-warnings 0`) and prettier clean; `test/flows` + `test/tools` pass 671/671, including a new case covering the legacy-spelling rejection. |
||
|
Jakub Piasecki
|
e314b3ef4a |
feat(flow): swipe directive (#520)
Add one semantic finger flick for dismissing cards, paging carousels, opening drawers, and other gesture-driven actions distinct from goal-seeking scroll-to. Support bare directions plus from with exactly one of direction, to, or by, optional settle, and positive finite duration. Resolve selector or normalized-point anchors through the shared target path. Direction uses the Maestro geometry presets and now rejects anchored gestures that would collapse or travel opposite to the requested direction. Relative by deltas saturate supplied axes to [0, 1] and reject any effective zero or sign reversal. Resolved starts must remain on-screen, while explicit endpoints stay author-controlled. Keep parse and serialization invariants aligned for programmatic steps, improve unknown-key hints and selector diagnostics, surface swipe targets in run reports and recording summaries, share positive-millisecond validation, and reuse the common flow test harness. Dispatch gesture-swipe on touch platforms and gesture-drag on Chromium, reject touch directives on Vega, and document when to prefer scroll-to or a raw system-edge gesture. |
||
|
Hubert Gancarczyk
|
50c9580e30 |
fix(tools): make a schema failure name the parameter that was sent (#718)
> Stacked on #717. ## The problem An invalid tool call returned Zod's raw issue array. Someone who wrote `flow_name` instead of `name` got back, depending on the tool: ``` Invalid params for tool "flow-start-recording": [{"expected":"string","code":"invalid_type","path":["name"]}] Invalid params for tool "flow-execute": [{"code":"custom","path":["flow_path"],"message":"Pass exactly one flow source: name or flow_path."}] ``` The first names the parameter the tool **wanted** and never the one the caller **sent**. The second — flow-execute's, whose `name` is optional so `flow_path` can stand in — names neither: it is anchored on the one source field the caller had no reason to send. Either way the mistake is invisible, and working it out costs a whole turn. ## What changes **1. `describeParamIssues` — one sentence per bad parameter.** Missing fields read as `` `name` is required (string) and was not provided`` rather than as a type error about `undefined`. Unknown keys are named and path-qualified, so a key nested in `selector` is not reported as a bare name that contradicts the top-level list printed beside it. The message closes with the caller's own keys — `You sent: \`flow_name\`, \`project_root\`.` — which is what makes the mistake self-evident. Key names only, never values: this string reaches logs, telemetry and the agent transcript, and a params object can carry a secret. Wired into both entry points: the registry's `invokeTool` and the HTTP boundary. **2. `flow-execute` accepts `flow_name` as an alias for `name`.** The tool is called `flow-execute`, so "the flow's name" spells itself that way; this is a mistake worth absorbing rather than rejecting. The name is resolved in code rather than by the schema — a Zod `required` failure would once again name only the field it wanted — and the error text names the spellings that are silently discarded (`flowName`, `flow`, `flow_file`), because by then Zod has already stripped them and the caller has no other way to learn their key was dropped. **3. A flow DIRECTIVE passed as `command` points at the tool that records it.** `command: "echo"` used to come back as a bare "Tool not found". It now names `flow-add-echo` — and says to call it **directly**, since routing it through the recorder would write the echo *and* a `tool: flow-add-echo` step that fails on every replay. The hint distinguishes directives the recorder **rewrites** from those it stores **raw** for the polish pass, because promising a rewrite that does not happen sends the author looking for a directive that is not in the file. `wait` and `long-press` get their own answers: neither has a recording tool at all. Only a genuine not-found is rewritten — a tool that ran and failed still reports its own error. **4. A recorder tool as `command` is refused.** `flow-add-echo`, `flow-add-step`, `flow-start-recording`, `flow-finish-recording` each mutate the recording itself, and nesting one reported success either way. Each is refused for its own reason: `flow-add-echo` would write the echo **and** a replay-breaking `tool:` step, `flow-start-recording` would **erase** this flow at replay, `flow-finish-recording` would end the take it is a step in, and `flow-add-step` cannot record itself. Now nothing is executed and nothing is written. --------- Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com> |
||
|
Hubert Gancarczyk
|
f21811d069 |
fix(flow): re-probe a recorded wait against the tree the runner reads (#717)
> Stacked on #715. ## The problem `await-ui-element` evaluates against the tree it reads — the **accessibility hierarchy** on iOS and Android, the **CDP DOM** on Chromium, the **toolkit page source** on Vega. The `await:` / `assert:` directive that the polish pass converts a recorded wait into is evaluated against the **runner's** tree instead. They overlap, but outside Vega neither contains the other — an id present in one can be absent from the other, and on iOS even the role vocabularies are disjoint. Both directions have been hit in practice: an id visible to the runner but not the recorder, and text-field values visible to the recorder but not the runner. The consequence is that a check can pass live and fail once converted. That makes the core promise — *each step is executed live so you verify it works before it's recorded* — untrue at exactly the point it is relied on. ## What changes After a recorded `await-ui-element` succeeds, the recorder re-runs the same condition against the tree the runner resolves directives against, and reports the result on the step it just wrote. **It warns; it does not refuse.** What gets written is a raw `tool: await-ui-element` step, and at replay that tool reads the same tree it just passed against — so "this would fail every run" was never true of the form actually recorded. What the probe genuinely reports is whether the **polish-time conversion** is safe, which is why the warning says to act on it *then* rather than re-record now (re-recording would duplicate the step). Because polish begins after `flow-finish-recording`, each verdict is also carried there: it is repeated on its own step's `summary` line, and `message` counts how many steps carry one. A warning raised at step 7 of a 40-step recording is otherwise long out of reach by the time it is actionable. ## Three outcomes, kept distinct | Runner tree says | Reported as | | --- | --- | | condition holds | nothing — step recorded clean | | condition does not hold | recorded + warning naming the causes to rule out before converting | | tree unreadable, or too slow to answer | recorded + warning that the conversion is **UNKNOWN, not known-bad** | The middle row is deliberately not "the conversion will fail". The identical verdict comes back when the screen simply moved on between the live wait and the re-probe, and on `text` checks when the selector matches several elements and the two sides elect different ones — so the consequence stays conditional on the cause, and the message names the causes in the order worth ruling out. The third row matters: nothing was compared, so calling it a divergence would send the author off to rewrite a selector that may be perfectly good. It is also the injection-free case the flow skills explicitly sanction, so refusing there would block a supported way of working. On iOS the quoted reason ends "provide bundleId explicitly" — advice aimed at the native tools. The warning corrects it, because no directive carries a bundleId and forwarding one would make the probe answer a question the runner never asks. ## Platform-specific explanation The *why* differs per platform, and stating the iOS story everywhere makes the message wrong where someone is trying to act on it. iOS gets AX-vs-native-hierarchy; Chromium gets "the flow tree keeps only addressable nodes"; Android gets the importance trim; Vega gets "the trees agree, so the screen changed". The named tool for reading the runner's side follows the same rule — pointing an Android author at `native-find-views` points at a tool they cannot call. The remedy is per **condition** as well as per platform: on `hidden` a determinate verdict means the runner's tree still *has* the element, so "retarget at an id the full hierarchy carries" is the opposite of what helps. ## A failed live wait is reported by cause `await-ui-element` returns `{ success: false }` for three different things: a condition read and found false, a tree source that never answered, and a cancelled wait. Only the first judges the condition, so only the first is narrated as "the wait never held" and prescribes re-recording or deleting the step. The other two observed nothing and are reported as UNKNOWN — the same unknown-vs-known-bad distinction the probe draws for the runner's tree. A wait nested inside a recorded `run-sequence` gets neither warning: that tool reports its own shape, which carries no `success` key. The skill says so, and says to read `toolResult` there. --------- Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com> |
||
|
filip131311
|
6177409c7c |
fix(tools): drop top-level JSON Schema combinators from tool input schemas (#782)
Fixes #773. ## The bug `gesture-rotate` (`anyOf`), `flow-execute` and `flow-read-prerequisite` (`oneOf`) hand-write an `inputSchema` that spreads a top-level combinator onto the derived schema. The Anthropic Messages API rejects that outright: ``` tools.N.custom.input_schema: input_schema does not support oneOf, allOf, or anyOf at the top level ``` The 400 fails the **whole request — every tool in it** — so any client forwarding our schemas verbatim as `custom` tools is bricked entirely, not just for these three tools. Reported from opencode. ## Why we never hit it **Claude Code silently repairs it before sending.** Verified live: it strips the combinator and re-expresses the rule as a prose line prepended to the description. All of our E2E/QA runs through Claude Code, so the defect was invisible from inside our own test loop. Secondary reasons: the code is recent (#546, 2026-07-24 for `gesture-rotate`; #568, 2026-08-05 for the two flow tools — before late July no argent tool had a top-level combinator), and nothing in CI asserted the shape of any tool's `inputSchema`. ## Why removing it is safe The combinators never enforced anything. Nothing in argent validates against the JSON Schema — the zod `.refine()`/`.superRefine()` does all the rejecting, on every dispatch path (HTTP 400, MCP `isError`, CLI flag error, internal registry dispatch), and always did. #546's stated rationale was "to keep invalid calls rejected at the schema layer", but that layer does not exist; the only JSON Schema evaluator in the repo was a 5-line helper inside that PR's own test. Verified equivalence before deleting: the removed `anyOf` accepts exactly the same 8 radius shapes as the two refinements, and `oneOf` the same 4 source combinations as the superRefine. ## What changed - The three hand-written `inputSchema` consts are gone. No tool hand-writes one now, so `Registry.registerTool` is the single emission point for all 76 schemas. - Each rule moves into the tool description, **worded differently per tool** — `tool-description-quality.yml` triggers on these paths and scores `disambiguation_score` (W=1.5), so identical prose on two tools would work against the gate. - `ToolDefinition.inputSchema` documents why it should stay underived. - New per-tool contract test over the whole catalog: no top-level `oneOf`/`allOf`/`anyOf` (API-rejected) and no top-level `not`/`if`/`then`/`else`/`$ref`/`$schema` (blocked pre-emptively — the natural way to re-encode a rejected combinator). Nested combinators stay legal and in use: `tv-remote`'s `button` is a `z.union`. - Catalog walk + expected count extracted to `test/helpers/catalog.ts`, shared with `interaction-messages` so a new tool needs the count bumped once, not twice. - Six existing tests read `.inputSchema` off unregistered definitions (populated only at registration) and now derive it. ## Not done by design **No throw in `registerTool`.** `createRegistry()` runs at tool-server bootstrap (`index.ts:182`), where a throw becomes an uncaught exception → `crashShutdown` → `exit 1`: a total outage, and `update-argent` is itself a tool so it could not run to repair it. The catalog is static and compiled in, so the contract test catches the same regressions before merge — and it also covers `paste` and the Lens tools, which never pass through `registerTool` at all. **Deliberately a denylist, not an allowlist** of permitted top-level keys: `additionalProperties` legitimately appears on schemas derived from `.strict()` objects (`screenshot-diff`, `stop-all-simulator-servers`). ## Follow-up left out `gesture-rotate`'s refinements carry `path: []`, and `deriveInvalidParams` drops empty-path issues — so every cross-field rejection is telemetered with an empty `invalid_params`. Fixing it changes CLI wording and starts populating that field, so it is not riding in this PR. ## Verification - 75 registered tools probed live: **zero** top-level combinators; the three now advertise `type, properties, required` only. - Negative test: re-introducing an `anyOf` fails the gate with `gesture-rotate declares top-level anyOf`. - `packages/registry` 90/90 pass; `packages/tool-server` 3952 pass with **7 pre-existing failures** in `bind-failure-telemetry`, `lens-relay-telemetry`, `startup-telemetry` and `tool-fail-invalid-params-emission` — all failing identically on pristine `fe5976857` (they die in `attachRegistryEventLogger`, unrelated to schemas). - Build, eslint, prettier clean; knip identical to baseline (60 unused exports / 187 unused exported types both before and after). Reaches users on the next npm publish; current version is 0.20.0. |
||
|
Hubert Gancarczyk
|
8623965b8a |
perf(flow): stop the recorder returning the whole flow file per step (#715)
## What changes `flow-add-step` and `flow-add-echo` returned the **entire growing YAML** on every call. Across measured recording sessions that was the majority of all tool-result text, and it grows quadratically with step count. - Both tools now return `stepCount` — how many steps the flow has — instead of `flowFile`. - `flow-add-step` additionally returns `recorded`: the one summary line that was appended. - The full file still comes back once, from `flow-finish-recording`. ## Why The cost is not just tokens. With the whole file echoed per step, context pressure during a long walkthrough visibly changes the artifact being produced — checks get trimmed from a test to buy back room. Returning a line per step removes that pressure without removing any information the author needs *at that moment*. ## Supporting change `summarizeStep` is lifted out of `flow-finish-recording`'s per-file loop so the recorder can render its one line exactly the way the final summary does — one spelling, not two that can drift. That line now also carries `times`, `duration` and `delayMs`. They change what replays, and since the recorder no longer returns the YAML, this line is the author's only per-step view of what landed in the file. ## Compatibility This is a **breaking change to the tool result shape**: `flowFile` is gone from `flow-add-step` and `flow-add-echo`. `savedTo` is unchanged and is still the only field naming the destination in remote-client mode. `flow-finish-recording`'s result **shape** is unchanged, but its `summary` strings are not: taps now render `×N`, long-presses `for Nms`, and tool steps `(after Nms)`, so the two surfaces spell a step the same way. Nothing outside the agent reads `summary`, so that is not a break. ## Tests Recorder tests now assert against the flow **as persisted on disk** rather than a returned string, which is the surface that actually matters. --------- Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com> |
||
|
Hubert Gancarczyk
|
96ea94f87a |
feat(flow): add an idle condition that waits for the screen to stop moving (#728)
> Stacked on #574. Replaces the route-fingerprint approach that #727 and #729 carried; both are closed. > > Rebased off the #715-#726 chain so it can merge on #574 alone. That chain owns `DirectiveOutcome.warning` / `StepReport.warning` and the `indeterminate` scoring rule; the small amount of each that this condition needs is carried here, scoped to `idle`, and will conflict textually with #724/#725 for whichever lands second. ## One check a selector cannot express ```yaml - await: { idle: true } ``` Has the screen stopped moving? Flows were substituting fixed `wait:` steps for it, which either wait too long on every run or too little on the run that mattered. ## Why it reads pixels as well as the tree Each signal is blind to what the other sees. The tree cannot see presentation-layer motion. An iOS push or modal dismissal commits its hierarchy up front and then animates a layer for a few hundred milliseconds; a cross-fade or a scrim moves no node at all. Measured on a real Android emulator: a page animating a large element continuously reads as a single static `WebView` node, and the existing `await-screen-idle` tool called it settled in **465 ms, five times out of five**. On a deterministic Chromium page whose DOM is byte-identical across the animation, that tool returned `settled: true` in **407 ms** while this condition failed 8/8. Pixels alone would not do either: they cannot see a tree still churning behind an unchanged surface, and anything animated forever (a video, a shimmer) would make a pixel-only settle unsatisfiable on a screen the tree calls ready. Unlike that tool, this cannot be silently stepped over. Readiness is not an acceptance criterion — the verdict belongs to the identity and outcome checks around it — so a screen that never settles still **passes**, carrying a `warning` that says what the green actually bought. That is what makes it safe to persist in a flow: it reports on every run without ever failing one over a video or a shimmer. The only thing that stops a run is a tree source that cannot be read. ## The rule the design follows The absence of evidence is never evidence. - A capture that did not arrive cannot complete the hold. - Neither can a single agreeing pair. Every animation that reverses has a turning point, and two samples straddling it come back identical while the screen is moving. On a live 3 s cross-fade that passed a default-shaped step on roughly **one run in three**; settling now takes two consecutive still intervals, so `minStableMs: 0` still means three reads. - A round is never started without the budget to observe it with. A round begun with nothing left neither captures nor reads, and both absences were otherwise recorded as facts about the device. - No verdict comes from a latch. A screen that settles and then moves again, or goes blank, has not settled. ## `timeout:` is a real bound No describe path takes an abort signal, so the tree read is raced against what is left of the budget. Without that, a wedged ViewInspector RPC ran 2.25 s past an 8000 ms budget, and a busy Android screen ran 5.4-6.0 s against a 3000 ms one. A read that runs out of step budget is the step ending, not the source failing, and only the latter is reported as an environment problem. ## Failure modes stay apart They call for opposite responses. A screen that never stopped moving is a verdict about the app. A tree that could not be read is not, and it names the foreground check first, because a backgrounded app reads identically to an uninstrumented one and relaunching is the wrong repair. A run whose captures never produced a comparable pair still settles on the tree and says so in a warning, rather than passing off half a proof as the whole one. ## Capture routing and tolerance Captures route exactly as the `screenshot` tool routes them, so tvOS and Vega, which have no simulator-server backend but are perfectly screenshottable through `xcrun` and the emulator console, are covered rather than written off. The pixel comparison owns its tolerance rather than borrowing `screenshot-diff`'s. That one holds a baseline stored across sessions, machines and OS versions against a live capture and must absorb real drift; this one holds two captures one poll apart from a single session, where a static screen reads back byte-identical (measured: zero changed pixels across five consecutive pairs on an idle simulator). The margin is load-bearing, because uniform change is all-or-nothing: at the baseline tolerance any cross-fade slower than ~2 s counted zero changed pixels and read as settled mid-animation. ## Not a screen check A dropped tap leaves the source screen perfectly idle. It belongs after the element check that names the destination, never instead of one. #730 documents the pairing. ## Verification - 3307 tests pass; typecheck, `typecheck:tests`, lint and prettier clean. Eleven mutations applied to the source, all eleven caught by tests. - Driven end to end against a real iOS simulator, a real Android emulator, and a Chromium app through a tool-server built from this branch: static screens pass with no warning (proving captures ran), animating screens pass carrying the warning that names the motion, and the pre-existing tool disagrees exactly where it should. --------- Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com> |
||
|
Hubert Gancarczyk
|
afd722924f |
feat(flow): support concurrent flow recordings (#574)
## Problem
Only one flow could be recorded at a time, anywhere on the machine.
Recording state was three module globals in `flow-utils.ts` — harmless
in a per-client process, but the tool-server is a **host-wide singleton
per install bundle**. Every MCP client, every subagent sharing a
parent's connection, every `argent` CLI call and every project on the
machine attaches to the same process, the same `Registry`, and those
same three globals. HTTP requests are served concurrently with no
serialization.
Three failures followed, all reproduced against the pre-change code:
1. **A second recording clobbered the first.** `startRecordingSession`
just overwrote, so the first agent's next `flow-add-step` silently
appended into the *other* agent's flow file — in a different project.
2. **Any agent could finish any agent's recording.**
`flow-finish-recording`'s schema was literally `z.object({})`.
3. **Replay stomped recording.** `resolveFlowFilePath` called
`setActiveProjectRoot`, so a `flow-execute` in project B rebound the
global root mid-recording of project A. Already worked around, locally
only, in `flow-add-step`.
## Approach
Recording state is now a `Map` keyed by the resolved flow file path,
`<project_root>/.argent/flows/<name>.yaml` — the identity of the
artifact being built, so "same key" means "same output file" and two
sessions on one key are a genuine collision rather than an accident of
scoping. `flow-add-step`, `flow-add-echo` and `flow-finish-recording`
each take `name` + `project_root`, so every call is self-contained;
required rather than optional-with-fallback, since a fallback would
succeed while an agent is alone and fail only once a second recording
exists.
`resolveFlowFilePath` is pure, which fixes defect 3 and retires
`FLOW_PROJECT_ROOT_REQUIRED` (the state it guarded can no longer exist).
Rejected alternatives — keying by device udid (flows are deliberately
device-portable, and `flow-add-echo`/`flow-finish-recording` have no
device at all), an opaque `recordingId` (an agent that loses the token
cannot recover), and per-caller identity from the transport
(`ToolContext` carries no caller identity, and subagents share their
parent's MCP connection, so it would fail the exact scenario being
fixed).
### Locking, and why readers needed more than a lock
Every mutation of one flow file runs under a lock keyed by that path —
appends, `flow-start-recording`'s truncate-and-register, and
`flow-finish-recording`'s read-summarize-clear. Keyed by file rather
than owned by the session, because a restart *replaces* the session: a
session-owned lock cannot exclude the operation that supersedes it. Per
file, not global, so two recordings never queue behind each other. An
append whose session was finished, restarted or evicted meanwhile fails
with `FLOW_NO_ACTIVE_RECORDING` rather than writing into a different
take.
The lock serializes writers only, and no reader of a flow YAML can join
it — `flow-execute`'s own load, its `run:` fragment load,
`flow-read-prerequisite`, `flow-add-step`'s sibling-fragment check, and
the `argent` CLI reading from another process entirely, where an
in-process lock cannot reach. Both writers used a plain `fs.writeFile`,
which opens `O_TRUNC`, so a reader could land between the truncate and
the write. That case is silent: `parseFlow("")` returns `{ steps: [] }`
with no error and `summarize` derives `ok` from "no failures", so a
`flow-execute` racing an append reports a **top-level PASS having
replayed zero steps**. Both writes now go through a sibling temp file
and a rename, so a reader sees either the whole old file or the whole
new one — and unlike the lock, that also holds cross-process.
### `stop-all-simulator-servers` gets a `devices` scope
Every agent is told to call this at session end, and unscoped it walked
the whole registry — so agent A wrapping up tore down agent B's devtools
mid-recording, degrading B's flow to brittle coordinate taps. It now
takes an optional `devices` scope and reports `unmatched` ids, so a
mistyped id doesn't read as a clean machine.
Which namespaces count as "owned by a device" turned out to be the
load-bearing part, because nothing cascades to most of them and
`unmatched` turns any omission into an actively wrong diagnosis — a
correct device id reported as a typo. The set now covers `AXService`
(the in-sim ax daemon, spawned `--timeout 3600`; an iOS session that
only ran boot/launch/describe owns this and nothing else),
`ScreenRecordingSession` (an ffmpeg child plus the touch-visualizer
overlay it enabled on the device), `NativeProfilerSession` (an xctrace
child, or an on-device perfetto process and its trace file) and
`JsRuntimeDebugger` (a bound loopback server, the CDP socket to Metro, a
log handle). The debugger URNs interpose the Metro port —
`<ns>:<port>:<deviceId>` — so the matcher understands both shapes,
consuming only the first colon so a wireless adb serial after the port
still compares whole.
`stop-simulator-server` shared none of this and had drifted: it looked
URNs up with an exact, case-sensitive `services.get()`, so a lower-cased
UDID silently no-op'd there while the scoped stop-all reaped it. Both
tools now share one matcher. Their namespace *sets* stay deliberately
different, documented where they are defined — `stop-simulator-server`
is also the documented recovery for a wedged transport, and widening it
to devtools/AX would make a routine retry drop the native-devtools
connection another agent's recording depends on, which is the hazard the
`devices` scope exists to prevent.
## Verification
**End-to-end, three devices, one shared tool-server** (iPhone 16 Pro,
iPhone 17 Pro, Pixel 3a): three interleaved recordings, plus an
unrelated `flow-execute` in a fourth project while all three were live.
Both iOS devices held their own live native-devtools concurrently (two
per-UDID sockets, `connected: true`). Each YAML contained exactly its
own steps in order, taps captured as selectors rather than coordinates,
and neither picked up another's or the third root's path. All three
replayed green — also through the **released 0.17.0 CLI** against the
modified server, which checks wire compatibility. A device-scoped
teardown of sim 1 left sim 2 and the emulator running.
Re-run against the final build:
- Two interleaved recordings across two project roots on one device,
with a **separate process** polling one of the flow files as fast as it
could for the duration. Zero empty or truncated observations; each YAML
held exactly its own steps; both replayed green (6/6 and 5/5). Against
the pre-change in-place write the same reader observes zero-step reads
interleaved among the real ones — the silent-PASS input.
- A scoped teardown of the iPhone reaped `AXService` and
`NativeDevtools` for that UDID only, reported no `unmatched`, and left
the second sim and the emulator running. Lower-casing the UDID matched;
a genuinely bogus id was still reported.
- With a screen recording live on the Pixel 3a, a scoped teardown reaped
`ScreenRecordingSession` and the **ffmpeg child actually exited** (1 →
0). Before the fix it survived to the 180 s cap while the tool called
the correct serial a typo.
The pre-change repros were re-run afterwards to confirm observed
behavior changed, not just the tests.
`npx vitest run` in `packages/tool-server` (295 files, 3154 tests) and
`packages/argent-cli` (280), `tsc --build`, `tsc --noEmit` on both test
projects, eslint and prettier — all green.
## Review notes
Several rounds of review found real defects in this change; each was
reproduced before fixing and re-verified after. Worth knowing when
reading the diff, since the tests exist to pin them:
- `flow-start-recording` truncated the `.yaml` outside any lock, so a
step from the take being discarded landed in the freshly reset file and
was reported as success (26-29 of 200 runs).
- `flow-finish-recording` never took the lock; its `await fs.readFile`
was a yield during which a concurrent append committed, leaving the
reported summary disagreeing with disk (11-16 of 200).
- A failed finish destroyed the recording. Hand-editing the `.yaml`
mid-recording is a documented workflow, so `parseFlow` can legitimately
throw — and the error told the agent to call `flow-start-recording`,
which truncates the very take it would recover.
- `unmatched` reported a device this session had just stopped, because
`disposeService` returns a node to IDLE and keeps it.
- A bare IP claimed every wireless-adb device at that address, since an
adb serial is itself `ip:port`.
- The eviction backstop stamped with `Date.now()`; ties at millisecond
resolution let it drop the session that was just used.
- `requireRecordingSession` ended with "Call flow-start-recording first"
— reached for a key that was finished, superseded or evicted as well as
one never started, and on those branches the file on disk is fully
populated while that call truncates it and reports no `restarted`.
- The atomic write regressed twice on its own terms: its scratch name
was derived from the flow file's basename, which has no length cap, so a
long flow name that appended fine before failed `ENAMETOOLONG`; and the
cleanup guard began after the temp file was created, leaking a scratch
file into a committed directory on any write that failed after opening.
- Two tests passed against implementations they claimed to reject: the
`stop-simulator-server` narrowness guard used a udid that classifies as
*android*, where the iOS-only services it guards can never appear, so
widening the iOS branch kept it green; and the client-mode finish
asserted only the *shape* of `savedTo`, never its content, though that
directive is the only thing that lands the file in client mode.
## Notes for reviewers
- Client/remote persist mode is the least-exercised path, and now has
its own concurrency tests (overlapping appends, and a restart
superseding an in-flight one) — it previously had none. The
`project_root` key is byte-stable across the file boundary (`kind:
"probe"` passes it through unchanged), which is what keeps a remote
recording addressable from `flow-add-step`, which declares no file
input.
- Two spellings of one flow file still mint two sessions — and two
independent locks — over one file, which bypasses every guarantee here.
Two ways in: the root spelled two ways (a symlink, or a case variant on
APFS), and the likelier one, the *name* cased two ways on a
case-insensitive volume. Neither is normalized away because the correct
normalization is the filesystem's: case-folding the key would wrongly
merge two distinct flows on ext4, and resolving symlinks is impossible
in client mode, where the root does not exist on this host. Documented
at `getFlowPath`.
- No recording tool can detect that another agent took its key.
`flow-add-step`, `flow-add-echo` and `flow-finish-recording` all
re-resolve the key on every call, so after a takeover they act on
whichever take is now live and report success: an append lands in the
other agent's file, and a finish finishes and clears it. The only case
that fails loudly is the narrow one the liveness check covers - an
append already IN FLIGHT when the restart lands. There is no caller
identity to key ownership on (see the rejected alternatives above), so
this is spelled out for the agent under "Pick a name unique to your
task" in `argent-create-flow` rather than mechanized.
- Skew hazard in one direction only: a *new* client against an *older*
tool-server. Zod strips the undeclared `name`/`project_root`, and the
old server falls back to its module globals — so concurrent recordings
silently collapse onto one session. The other direction (released 0.17.0
CLI against this server) was verified end-to-end.
- `packages/argent-private/docs/reference.md` still describes
`flow-add-step` as appending to "the current recording", and names two
tool ids that do not exist (`flow-insert-echo`/`flow-run` are file
names; the ids are `flow-add-echo`/`flow-execute`). It is a submodule,
so the fix cannot ride on this branch.
- A restart reported `discardedSteps` from the superseded session's
in-memory flow while it truncates the *file*. Hand-editing the `.yaml`
mid-recording is documented, so the two diverge: four steps wiped, one
reported. It now counts the file, and reports no number at all when that
file cannot be read or parsed - 0 is the answer a genuinely empty take
gives. Client mode still counts from memory (this host cannot see the
client's file), and the description now says which mode it is promising.
- `ChromiumJsRuntimeDebugger` cascades from `ChromiumCdp` and was
therefore already being torn down - silently, while `NetworkInspector`
and `ReactProfilerSession` cascade the same way and were named.
`stopped` is documented as the services that were live and got shut
down, so it is listed now too.
- The client-mode case named for a superseded in-flight append restarted
*between* calls, so the next append re-resolved the key and succeeded -
the guard was never reached, and neutering it left the whole file green.
It now parks an append in its live sub-tool call across the restart.
- `stop-tools.test.ts`' mock handed the tool the live service map where
the real `getSnapshot` copies, so a cascade rewrote the state the sweep
was still reading and the answer depended on map insertion order. The
mock copies and recurses like `_teardown` now, and the cascade case runs
under both orders.
- Prose the code contradicted, each verified against source before
rewriting: Vega owns `JsRuntimeDebugger`/`NetworkInspector` once the
debugger has run (and can match anything at all through an ERROR node);
"nothing here cascades" was denied twice in its own file; an iOS
boot/launch/describe session owns `NativeDevtools` too; `:tcp` is
mintable but unreached in production; the tool-server is a singleton
*per install bundle*, not per machine; `argent flow list` does enumerate
`.argent/flows`; the stop-tools comments narrated a "before" that only
existed on this branch.
Found while verifying, out of scope, worth their own issues: a `tool:
flow-execute` self-reference recurses past both cycle guards (`run:` is
guarded, a nested `flow-execute` reseeds the stack); react-profiler
artifacts collide host-wide on a second-resolution filename;
`stop-metro` is port-scoped and kills another agent's bundler;
`debugger-reload-metro`'s HTTP fallback broadcasts to every attached
client. A tool's `capability` is enforced only by the HTTP layer, not by
`registry.invokeTool`, so a call that reaches the registry another way -
`flow-add-step` takes `command` as a bare string - can resolve a service
the tool declares it does not support, leaving an ERROR node behind
(e.g. `SimulatorServer:<vega serial>`).
---------
Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com>
|
||
|
Jakub Piasecki
|
cc2f907ce9 |
feat(flows): resolve run: directives from explicit YAML paths (#567)
A `run:` value is now a YAML path resolved against the directory of the flow file that contains the step, matching the CLI cutover: ```yaml - run: login.yaml - run: ../shared/login.yaml # cross-directory composition now works ``` Bare names fail at parse with a migration hint (`run: login` → `run: login.yaml`). The filename stem stays the fragment's report attribution; reports also carry the as-written path as `target`. Cycle detection compares realpath-canonical paths (same stem in two dirs ≠ cycle; same file via symlink/spelling = cycle). Snapshot baselines are unchanged — keyed by and stored beside the root flow. An uploaded root flow with `run:` steps is now rejected up front with a contract error instead of a mid-run temp-dir ENOENT (its referenced files stayed on the client, and relative resolution in the temp dir could reach host files); same for a relative chromium app path. The recorder emits the sibling form (`run: <name>.yaml`). **Breaking:** existing `run: <name>` directives fail at parse until renamed to `run: <name>.yaml`. Snapshot baselines for a saved flow whose YAML is a symlink now anchor beside the real file (the symlink target's directory) — move any committed `.argent/flows/__baselines__/<flow>/` set beside the real file once, or the next run fails with a missing baseline. A `run:` target must also resolve inside the project root or beside/below the flow file that references it (arbitrary host YAML is no longer reachable), and an uploaded flow with a `snapshot` step is rejected up front (its baselines cannot anchor to the per-call temp dir). |
||
|
Jakub Piasecki
|
a644d52d39 |
feat(flows): run CLI flows from explicit paths (#568)
`argent flow run` now takes an explicit YAML path instead of a saved-flow name: ``` argent flow run .argent/flows/checkout.yaml --platform ios ``` The path may point anywhere on the local filesystem; the filename stem names the run's report/artifacts, so it must be `[A-Za-z0-9_-]+` with a lowercase `.yaml` extension. A bare name fails with the exact replacement command, and `argent flow list` prints paths instead of names. Because sibling `run:` files and `__baselines__` resolve beside the YAML, the CLI now requires the auto-started local tool server and refuses `ARGENT_TOOLS_URL` / `argent link` routing. The `flow-execute` tool stays remotely callable via `name` + `project_root`, and gains a `flow_path` parameter (exactly one of `name`/`flow_path`) that is only honored when the file-input boundary verified it as a real host path — uploads and boundary bypasses are rejected, so it can't execute arbitrary server YAML. Also: the flow-name pattern is now a single shared export from `@argent/registry`, and the recorder composes a `flow_path` call into a `run:` step when it targets a sibling of the recording. |
||
|
filip131311
|
5a64e4e7de |
fix(flows): stop a recorded sub-flow pinning the device it was recorded on (#696)
Fixes #607. ## The bug `DEVICE_BIND_KEYS` covered `udid` and `device_id`, but `flow-execute`'s own device parameter is named `device`. So a recorded `tool: flow-execute` step kept the record-time id in the YAML, and on replay the runner never rebound it. Reproduced live on 0.18.1. Recording a cross-project `flow-execute` step: ```yaml steps: - tool: flow-execute args: name: b-only project_root: /…/projB device: BC0026C7-AAE0-490E-8BD4-A345CF65FB1B # survived stripDeviceKeys ``` Then editing that id to one that does not exist — what a flow committed on one machine looks like on another — and replaying against a real booted device: ``` flow-execute { name: pinned, device: BC0026C7-… } [1] ✓ flow-execute { "device": "00000000-0000-0000-0000-000000000000", <-- baked in, not rebound "ok": false, "errored": 1, … "reason": "…[SimulatorServer:00000000-…] simulator-server exited…" } PASS — 1 passed, 0 failed, 0 errored ``` This contradicted three things the code and docs already claimed: that flows store no device id and are portable, that the runner is authoritative on device and drops any id stored in a step, and that device ids are stripped at record time. It also made the raw `tool:` form behave differently from `run:` composition, which has always inherited the run device. That matters more than it looks: the recorder falls back to the raw form whenever the target is not a resolvable sibling, **and always uses it for a remote recording**, since `run:` composition is host-resolved. Remote recordings are therefore the main producer of the affected artifact. ## The fix Add `device` to `DEVICE_BIND_KEYS`. That fixes both halves at once — `stripDeviceKeys` keeps it out of new recordings, and `bindDeviceArgs` replaces an id already committed to a flow file. Safe because `bindDeviceArgs` only injects keys the target tool declares, and across all 75 registered tools **exactly one** declares `device` — `flow-execute` itself. (24 use `device_id`, the rest `udid`; both were already bound.) ### `platform` is deliberately not stripped There is a test pinning this so a later symmetry edit fails loudly. Two independent reasons: 1. It is read **only when no device was given** — `resolveFlowDevice` returns on `opts.device` before touching it, and the chromium boot spec is gated on `!params.device`. Once `device` is bound it is inert. 2. The strip is schema-blind, and `platform` is *not* device-specific on every tool: `react-profiler-analyze` declares its own. Stripping it would silently rewrite a recorded Android profile analysis into an iOS one. ## Verified live ``` replay, flow bakes in the all-zeros id, run given a real device: before -> nested device 00000000-…, ok false, errored 1 after -> nested device BC0026C7-…, ok true, errored 0 record leg -> recorded YAML has no `device:` key ``` Five new tests, each confirmed to fail before the change and pass after: the two unit cases, an integration case through the runner, the record-side case (using a **non-resolvable** target — a resolvable sibling records as `run:`, which carries no args and so could never show this), and the remote-persist case. Full tool-server suite 3091 passing; lint, prettier, `typecheck:tests` and the tool-description gate clean. ## Behaviour changes worth knowing - **Flows with a baked-in `device` stop pinning** and follow the run device. That is the fix and the documented contract, but it is a silent semantic change for anyone who noticed the pinning and relied on it. No migration is possible or desirable — the baked-in id is host-specific and is exactly what made those flows non-portable. - **A nested `flow-execute` can no longer target a device other than its parent's.** Verified nothing in the repo does this (no tracked `.argent` flows, no flow references in the e2e scripts or workflows), and `run:` already behaved this way. The loud case is a nested Chromium flow that used to boot its own Electron instance; the quiet case is a nested step with a different `platform`, which now silently runs on the parent's device. ## Related - #606 (a nested flow's failure does not fail the outer step) is what let the transcript above report `PASS`. Independent in code — #606 is in the result inspection after dispatch, this is in the args before it — so they do not conflict. Worth knowing when reviewing #606 that the "nested run errored" symptom there had a second, separate cause, fixed here. - `extractDeviceArg` in `http.ts` keeps its own parallel `udid`/`device_id` list and also omits `device`, so a top-level `flow-execute` call gets no platform attribution in telemetry. Not touched here — it is telemetry-only, and unifying it would create an import direction that does not exist today. - `project_root` / `flow_file` are absolute host paths baked into recorded steps — the same portability family, but they cannot simply be stripped (the nested call needs them) and rebinding needs a design decision. Filing separately. Co-authored-by: Filip131311 <f.kaminski2000@gmail.com> |
||
|
Jakub Piasecki
|
3f602a7a9b |
feat(flow): conditional when blocks (#498)
## What
A `when:` block runs a step list only when one condition holds — a UI
condition (`exists`/`visible`/`hidden`/`text`, the await/assert shapes)
or a static `platform` test. **No else**, by design: a block exists to
restore determinism (dismiss an interstitial, reconverge), never to
test two paths. Tap-if-present is a one-step block:
`when: { visible: "Got it" }` + `steps: [tap: "Got it"]`.
No per-step `optional:` — `when:` covers every directive once. The key
is rejected at parse with a pointer to `when:`, not silently dropped (a
Maestro habit that would leave a "can't fail" step hard-stopping the
flow).
## How
- The guard reuses the assert engine at the 1s grace (`timeout` is a
parse error), so a skipped block barely costs a clean run.
- Verdicts are evidence-based: no trustworthy read in the window →
indeterminate → the step **errors** ("could not evaluate" is not
"condition false"); a transient read failure after trusted reads showed
the condition false stays a clean skip.
- `platform` folds `ios-remote` into `ios` — the parser rejects that
spelling, so the fold is the only way a guard matches a remote sim.
- A skipped block reports one line per authored step, same shape for
unmet guard, hard stop, or cancellation; everything goes through
`pushReport`, so when lines stream live. Failures inside an entered
block hard-stop as usual.
- Results carry an explicit `aborted` flag; `ok` keys off
fail/error/cancel, not skip lines.
- Nesting is capped at 20, so a cyclic YAML alias is a structured parse
error, not a stack overflow.
## Testing
`vitest` 2546 pass (17 when-specific); `tsc --noEmit` (tool-server,
argent-mcp, argent-cli), eslint, prettier clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Jakub Piasecki
|
1a4f89cd5f |
feat(flow): allow to compose all flows via run: (#507)
## Summary `run:` accepted **fragments only** — an e2e flow (one starting with `launch:`) was rejected as a `run:` target. This lifts that: an e2e flow can now be nested via `run:`, its `launch` running inline (a real terminate + relaunch on native). Composition becomes uniform — any flow can be a reusable building block. ## What changed - **`flow-run.ts`**: dropped the `isE2eFlow` rejection in `execRunStep`; cycle / `MAX_RUN_DEPTH` guards still bound recursion. - **`flow-add-step.ts`**: the recorder now captures a recorded `flow-execute` of an e2e sibling as a portable `run: <name>` directive (was: raw `tool:` step). - **`flow-utils.ts`, `argent-create-flow/SKILL.md`**: updated docs/skill that said e2e flows can't be `run:` targets. ## Known limitation — Chromium Chromium `launch` boots an Electron app from a *path*, wired to the top-level flow only — so a nested **chromium** e2e flow doesn't boot its own app. Native nesting is correct; chromium nested e2e is unsupported. ## Testing Rewrote the two affected tests (composition + recorder). Full flows suite green (349 tests); `tsc --noEmit` clean. |
||
|
Jakub Piasecki
|
3d3b82a076 |
feat(flow): regex matching via matches in assertions and selectors (#503)
## What
- Add `matches` as a case-sensitive JavaScript regex comparator for
`await` and `assert` text conditions.
- Support `{ text: { matches: ... } }` in every flow selector slot,
combined with `id` and `role`.
- Preserve regex matchers through parsing, YAML serialization, recording
summaries, and run reports.
## Correctness
- Prevent empty-matchable patterns from matching unlabeled nodes or
empty assertion text.
- Reject unknown raw `await-ui-element` selector keys instead of
silently dropping constraints.
- Reject selector combinations YAML cannot round-trip without changing
their semantics.
- Document native label aggregation accurately and keep action ranking
consistent.
## Maintenance
- Share selector-field and regex-syntax validation across both matcher
parse paths.
- Compile selector regexes once per tree walk and full-match regexes
once per ranking pass.
- Make comparator serialization exhaustive, extract the duplicated flow
test harness, and scope slow-test timeouts narrowly.
## Verification
- Full tool-server suite: 2,588 tests across 263 files.
- Source and test typechecks, ESLint, Prettier, and skill-copy
validation pass.
|
||
|
Jakub Piasecki
|
551c781fb0 |
feat(flow): support running end-to-end flows without agent (#429)
# Flow E2E
This PR turns flows from replayable tool-call recordings into a
declarative, portable e2e test system that runs headlessly via `argent
flow run <name>` — no LLM in the loop, non-zero exit on failure,
suitable for CI.
Agents can still replay flows via `flow-execute` and get the full output
from each step after it finishes.
## Flow format
Flows are YAML files in `.argent/flows/`, now built from **directives**
interpreted by the runner rather than raw tool calls:
- `launch` — terminate + relaunch the app under test (per-platform app
id map supported); its presence marks a flow as **e2e** (controls its
own start state) vs. a **fragment** (runs against current state,
declares an `executionPrerequisite`, composable via `run:` with
cycle/depth guards).
- `tap`, `type` (focus handshake + optional submit), `scroll-to`
(momentum-free, increments by half the container, `within:` for nested
scrollers), `await`/`assert` (`visible`/`exists`/`hidden`/`text`
conditions), `wait`, `snapshot`, `echo`, and raw `tool:` as escape hatch
— still fully supported, so previously recorded flows keep working
unchanged.
- **Selectors** are `{ text, identifier, role }`; a bare string is a
loose selector resolving identifier-first, then visible text. Any
directive failure hard-stops the flow; later steps report `skip`.
## Execution engine
- Flows store no device id — the runner binds a device at replay
(explicit `--device`/`--platform` or the single booted one).
- Selector resolution runs against **platform-unified UI trees**: iOS
native view hierarchy (full testID coverage, gated on native-devtools
connecting), Android full accessibility hierarchy (full resource-id
coverage), Chromium CDP DOM, and the Vega automation toolkit. A shared
flatten/text-hoisting pass lets `text` assertions read what an
identified container visibly shows. Vega is remote-driven, so touch
directives fail with guidance to use `tv-remote`/`keyboard`.
- Steps wait for the tree to **settle** (two consecutive identical
reads) before acting; `launch` additionally gates on platform
tree-source readiness instead of silently degrading.
## Visual snapshots
`snapshot` diffs a full-resolution screenshot against baselines in
`.argent/flows/__baselines__/<flow>/`, keyed by platform + resolution,
with configurable `maxMismatch` and `--update-baselines`. The status bar
is pinned for the whole run (iOS `simctl status_bar`, Android demo mode)
so clocks/battery don't drive diffs. Baseline/current/diff images are
returned as artifacts on failure (and the written baseline on first
capture/update); a clean pass carries none, keeping reports and agent
context lean.
## Recording
The record-live workflow (`flow-start-recording` → `flow-add-step` →
`flow-finish-recording`) now emits directives automatically: coordinate
taps are rewritten to portable selector `tap:` steps (selector captured
from the pre-tap tree), a leading `restart-app` becomes `launch:`,
`flow-execute` of a sibling fragment becomes `run:`, and device ids are
stripped. The skill instructs the agent to polish the saved YAML
afterwards (fold tap+keyboard into `type:`, swipes-to-reach into
`scroll-to:`, awaits into sugar form).
## Also included
- `argent flow run` / `argent flow list` CLI with per-step report
rendering, artifact materialization, and `--json` output.
- New structured failure codes for flow validation, device resolution,
composition, and devtools readiness.
- Extensive unit test coverage (tree adapters, selector matching,
scrolling, composition, type focus, snapshots, status bar).
|
||
|
Hubert Gancarczyk
|
0edefebe06 |
fix(telemetry): orchestator sub-tools invocations was missing ai_client and platform attributes (#400)
## Summary Fixes telemetry attribution for tools dispatched by orchestrators. `run-sequence`, `flow-execute`, and `flow-add-step` run their steps via `registry.invokeTool()` directly, which minted a fresh, unregistered invocation id per nested call — so the `ai_client` / `platform` the HTTP layer captured for the outer request never reached the nested gestures, and they were recorded as **anonymous** in telemetry. Remove `opt_out` event that is a leftover. ## Changes - Add an optional `recordChildInvocation` recorder to `InvokeToolOptions` / `ToolContext`, bound by the HTTP layer to the outer request's attribution. - New `invokeSubTool` helper: registers each sub-tool under a minted id inheriting the request's AI client, with the platform re-derived from the sub-tool's **own** device arg (`udid` / `device_id` / `avdName`); forwards the recorder so propagation survives arbitrary nesting (e.g. `flow-execute → run-sequence → gesture-tap`). - Route the three orchestrators' sub-dispatch through - Refactor platform inference in the HTTP layer into a shared `platformFromArgs` helper (also hardened to not throw on a malformed device arg). - Remove opt_out event leftover Why re-derive platform per child: an orchestrator like `flow-execute` has no platform of its own and a single flow can target several devices, so the child's device arg is the only correct platform source; the parent's platform is the fallback. |
||
|
Hubert Gancarczyk
|
de0dbbda66 |
feat(tools): implement screenshot-diff tool (#229)
## Summary Adds a new `screenshot-diff` tool for comparing mobile screenshots during UI QA and visual regression checks. The tool can compare two saved PNGs, or compare one saved PNG against one live full-resolution device capture, then returns a compact text summary and generated diff artifacts. ## Context Mobile UI validation often needs more than a single screenshot: small layout, color, typography, copy, clipping, and regression changes are easy to miss manually. This feature gives agents a deterministic before/after comparison path while keeping `screenshot-diff` as supporting evidence. ## What Changed - Registered a new `screenshot-diff` tool in the tool server. - Supports saved PNG comparisons and one-sided live capture via `captureBaseline` or `captureCurrent`. - Adds PNG diffing with fixed internal defaults, changed-region detection, status-bar-band ignoring, full-size diff output, and downscaled context diff output. - Adds optional OCR/text analysis for moved, appeared, disappeared, changed, and restyled text. - Updates MCP content rendering to attach the downscaled context diff image plus the text summary. 1. Capture a stable full-resolution baseline. 2. Perform an interaction or apply a UI change. 3. Compare the current screen against the baseline. 4. Review a summarized diff plus generated artifacts. |
||
|
Ignacy Łątka
|
11df817616 |
test: drop real-time timeouts from the slowest unit tests (#236)
The slowest unit tests were waiting on real wall-clock timers, not doing
work. This makes them deterministic and fast with **no production
behavior change and no production test seams**.
## Speedup
**Suite-level**
| Scope | Before | After | Faster |
|---|---|---|---|
| tool-server — wall clock | 2.72s | ~1.3s | ~2× |
| tool-server — aggregate test time | 9.70s | 4.03s | 2.4× |
| argent-mcp — aggregate test time | 0.90s | 0.28s | 3.2× |
**Per file**
| File | Before | After | Faster |
|---|---|---|---|
| simulator-server-blueprint.test.ts | 2.55s | 0.05s | ~50× |
| boot-device-hardening.test.ts | 2.17s | 0.18s | 12× |
| boot-device-spawn-error.test.ts | 1.11s | 0.005s | ~200× |
| boot-device-hotboot.test.ts | 1.01s | 0.06s | 16× |
| fetch-timeout.test.ts | 0.81s | 0.18s | 4.5× |
| flow-tools.test.ts (delayMs test) | 0.31s | 0.04s | ~7× |
**Worst individual tests, before → after**
| Test | Before | After |
|---|---|---|
| rejects the boot promise…when spawn emits error | 1.06s | ~1ms |
| reports the emulator crash error… | 1.00s | ~2ms |
| falls back to cold boot when hot-boot child exits early | 1.00s | ~1ms
|
| reports a signal-terminated emulator (SIGSEGV)… | 1.00s | ~1ms |
| 5× simulator-server-blueprint tests | ~0.50s ea | ~0.001–0.02s ea |
| throws after retries when POST never responds | 0.60s | 0.14s |
| sleeps the step's delayMs | 0.31s | 0.04s |
No test exceeds ~0.7s anymore (the slowest,
`workspace-reader-integration`, is genuine subprocess I/O — `node
--version` etc. — left as-is on purpose; it is not a timeout).
## What changed
- **`boot-device.ts` (only production change):** the three hardcoded
poll cadences are extracted into a single immutable
`BOOT_POLL_INTERVALS_MS` constant (values unchanged: 1500/1000/500ms)
and referenced at the four `setTimeout` sites. Pure readability
extraction — **no mutable state, no exported test hooks, no runtime
behavior change.**
- **Boot-device tests:** the timing-sensitive tests now use vitest fake
timers (`vi.useFakeTimers()` + `vi.advanceTimersByTimeAsync`).
`boot-device-hardening` / `-spawn-error` add the same partial
`vi.mock("../src/utils/adb")` already used by
`boot-device-hotboot.test.ts`, so pre-flight does no real filesystem I/O
and the timers drive deterministically.
- **`simulator-server-blueprint.test.ts`:** `signalReady` now emits
`stream_ready` then `api_ready` (matching the real streaming server's
ordering) so the blueprint resolves immediately instead of waiting out
the 500ms `STREAM_GRACE_MS` fallback. Adds a dedicated fake-timer test
that explicitly covers the non-streaming grace path (coverage gained,
not lost).
- **`fetch-timeout.test.ts` / `flow-tools.test.ts`:** smaller timing
knobs; same behaviors asserted.
## No regressions
- 1011/1011 tests pass (tool-server 656, argent-mcp 53, argent 6,
registry 64, tools-client 1, installer 231); `tsc --build` and Prettier
clean. The converted boot-device, simulator-server-blueprint, and
flow-tools tests are deterministic under fake timers (verified across
stress runs).
- No production behavior change; `simulator-server.ts` is byte-identical
to `main`; net +1 test.
- Heads-up: `argent-mcp/test/fetch-timeout.test.ts` has a
**pre-existing** macOS flake (~10–20% rate; `connect EADDRNOTAVAIL
127.0.0.1:<port> - Local (0.0.0.0:0)` — kernel source-port exhaustion
under rapid-fire loopback fetches). Stress-testing showed it on `main`'s
pristine file too (2/10 failures vs 1/10 on this branch), so this PR
does not introduce or worsen it. Fixing it (e.g., EADDRNOTAVAIL retry,
or moving the test off real loopback fetch) is worth a follow-up.
|
||
|
Ignacy Łątka
|
2180dd8e6a |
ci: typecheck test files in CI (#202)
We've had lots of type errors in our tests, scripts and other non-core
code despite all CI being green. This PR fixes that.
This PR also removes some redundant features like `@ts-check` inline
directives which have been replaced by proper ts config files.
<details>
## AI Summary
Audit of every CI workflow against every package's `package.json` and
`tsconfig` surfaced several gaps where CI did partial coverage. This PR
closes them and fixes every pre-existing type error the new gates
surfaced.
## Gaps closed
| # | Gap | Fix |
|---|---|---|
| 1 | `tsc --build` only covers `src/**`; test files (`test/**`,
`tests/**`) never typecheck. Vitest transforms tests with esbuild —
strips types instead of checking them. | New `tsconfig.test.json` per
test-bearing package (`composite: false`, `noEmit`, `rootDir: "."`,
includes `src/**` + `test/**` (or `tests/**`) + `vitest.config.ts`). New
`typecheck:tests` npm script per package. New CI step `npm run
typecheck:tests --workspaces --if-present`. |
| 2 | Build/publish scripts (`scripts/*.{cjs,mjs}`,
`packages/argent/scripts/*.cjs`, `packages/skills/scripts/install.js`) —
JS, never typechecked. `sync-readme.cjs` runs in `prepack`;
`postinstall.cjs` runs on every install; `bundle-tools.cjs` is
build-critical. A typo only surfaces at runtime. | Root
`tsconfig.scripts.json` with `allowJs` + `checkJs` + `noImplicitReturns`
covers all script paths. Root `typecheck:scripts` npm script. New CI
step. Per-file `// @ts-check` comments removed — tsconfig drives
inclusion. |
| 3 | `argent-cli` had no `typecheck:tests` script. When tests get added
there they'd silently bypass the gate. | Added `tsconfig.test.json` +
`typecheck:tests` script. `--if-present` would have skipped it anyway,
so this is future-proofing. |
| 4 | Local `prettier --check .` walked into the `argent-private`
submodule and reported violations there. CI didn't see this because
`actions/checkout@v4` doesn't init submodules — so local `prettier
--check` did not match CI behavior. | New `.prettierignore` excludes the
submodule plus `dist/`, `node_modules/`, lock files, `tsbuildinfo`.
No-op in CI; aligns local with remote. |
## Test type errors fixed
These were already present. The `npm test` passed because esbuild
stripped the offending types:
- **argent-installer**: widen `scope` literal type for
dead-code-elimination guard; non-null `addAllowlist!`/`removeAllowlist!`
(optional methods on adapter).
- **argent-mcp**: add `.js` extensions for NodeNext module resolution.
- **registry**: broaden `StaticBlueprintResult` blueprint api from `{
id; deps? }` to `Record<string, unknown>` so existing tests can pass
arbitrary api shapes.
- **tool-server**:
- non-null `zodSchema!` on tool defs that always carry one
(`launchAppTool`, `restartAppTool`, etc.)
- cast sentinel `"ignored"` strings to `DeviceInfo` in factory-rejection
tests
- match the 4-arg `dispatchByPlatform<IosServices, AndroidServices,
Params, Result>` signature (tests still passed the old 3-arg shape)
- `assertFlowRunResult` type guard for the `FlowRunResult |
FlowPrerequisiteNotice` union before `.steps` access
- swap broken `typeof import("supertest").default` for static `import
supertest` (supertest is `export = supertest`)
- fill in the 4 fields missing from a `NativeProfilerSessionApi` mock
(`xctraceProcess`, `recordingTimedOut`, `recordingExitedUnexpectedly`,
`lastExitInfo`)
- cast a vitest `Mock` to a plain function at one direct call site where
the `Procedure | Constructable` union loses callability
- fix `update-checker.test.ts` reading `../../package.json` (off-by-one
— went past tool-server; was relying on a Vite resolver quirk)
- replace one cross-package `../../../registry/src/index` import with
`@argent/registry` so registry's private `services` field isn't compared
between src and dist declarations
All fixes are type-level. Full `npm test --workspaces --if-present`
still reports **991 passed across 87 test files**.
</details>
|
||
|
Ignacy Łątka
|
bea43f555a |
feat: per-step delayMs on flow steps (#198)
- Tool steps in a flow YAML may carry an optional `delayMs` that sleeps that long *before* the step runs; `flow-add-step` exposes the same field. - New `longRunning?: boolean` field on `ToolDefinition`; when set, the MCP fetch wrapper bypasses its 30s per-request timeout (`fetchTimeoutMs: null`) so long invocations aren't aborted. `run-sequence` and `flow-execute` opt in. |
||
|
Ignacy Łątka
|
bef319134e |
fix: add project_root arg to flow-start-recording (#114)
* fix: detect project root via .argent dir instead of git Flow tools previously shelled out to 'git rev-parse --show-toplevel' to locate the project root, which failed outside git repos and pulled flows from the wrong place in nested worktrees. Walk up from cwd looking for an existing .argent/ directory; fall back to cwd so the first flow-start-recording still creates it in place. * fix: skip $HOME when walking up for .argent/ The MCP launcher stores its own state in ~/.argent/ (logs, tool-server pid). Without this guard, running any flow tool from anywhere under $HOME — but outside a real project with its own .argent/ — would stop at the home directory and treat it as the project root, mixing personal MCP state with flow files. * revert: restore getGitRoot in flow-utils The walk-up .argent approach is fundamentally broken because the tool-server is a singleton daemon — its process.cwd() is not the calling project's cwd. Reverting both |
||
|
Filip131311
|
8c2a313c76 |
chore: format codebase with Prettier
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Ignacy Łątka
|
629007ba54 | rename | ||
|
Ignacy Łątka
|
f8ff5492dc | add tests for edge cases | ||
|
Ignacy Łątka
|
d871b67a75 | add tests | ||
|
Ignacy Łątka
|
66dc7546f3 | rewrite tests to work with yaml | ||
|
Ignacy Łątka
|
f3a2a952ee | add tests |