The `exportAndResolveArtifacts` call in `runFlowDirectory` sat outside the
loop's try/catch, so a throw from the artifact fetch escaped the whole
function: no skip entries, no batch summary, and no `--json` aggregate.
Wrap it the way the single-flow runner already does, and treat the failure
as the infra wall it is - the flow is recorded failed and the batch stops.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follows #952 (merged); rebased onto main.
## Why
`argent-cli` had **seven** copies of the same argv loop — `server
start`, `link`/`unlink`, `flow run`, `lens`, `config`,
`enable`/`disable`, and (from #952) `telemetry` — each hand-rolling `--x
value` / `--x=value` / `-x` / unknown-flag / missing-value handling with
slightly different messages and gaps (e.g. `lens` silently ignored
unknown flags). Adding an option meant another `if (tok === "--x") …
else if (tok.startsWith("--x="))` pair.
## What
`packages/argent-cli/src/command-args.ts` (introduced in #952) now
covers the whole subcommand surface, and every command declares its
options as a spec:
```ts
const START_OPTIONS = {
help: { kind: "boolean", alias: "h" },
port: { kind: "value", alias: "p" },
…
} as const satisfies OptionSpecs;
```
Parser features: `--name value`, `--name=value`, single-letter aliases,
boolean switches, `--` end-of-options, `choices` validation, "don't
swallow a following flag as a value", and the guard against a stray
`true`/`false` word after a switch (previously duplicated in `flow` and
`lens`). Value *semantics* (port ranges, host wildcards, flag-name
charset, link targets) stay in each command — the parser only shapes
argv.
`flag-parser.ts` is untouched: it is the schema-driven `argent run
<tool>` payload builder (JSON Schema → tool args, `--args`/stdin
hatches), a different job.
## Behavior changes (bad-input paths only)
| Command | Before | After |
|---|---|---|
| `lens --bogus` / `lens extra` | silently ignored | `lens: Unknown
flag: --bogus`, exit 2 |
| `lens --terminal x` | `--terminal expects "iterm" or "terminal", got
"x"` | `--terminal must be "iterm" or "terminal", got "x"` |
| `flow run --platfrom=ios` | `unknown flag --platfrom=ios` | `Unknown
flag: --platfrom=ios` |
| `unlink foo` | `Unknown flag: foo` | `Unexpected argument "foo"` |
| `config … --scope x` | `--scope must be "global" or "project", got
"x"` | unchanged wording |
| `enable/disable`, `server start`, `link` | — | unchanged wording
(tests pinned) |
Happy paths are byte-identical; `server`/`link` still throw
`StartFlagError`, `flow` still throws `FlagParseException` (wrapped from
`UsageError`), so callers and tests keep their contracts.
## Tests
`argent-cli` 489/489 (26 files): rewritten `command-args.test.ts`
(aliases, `-` positional, empty-token value, true/false guard, choices
message), pinned-string updates only where the table above says the
wording changed (`flow.test.ts` ×4, `parse-link-flags.test.ts` ×1). `tsc
--build`, `typecheck:tests`, eslint, prettier, knip green. Smoke-tested
every command's error path through the built
`packages/argent/dist/cli.js` (see the "Behavior changes" table — all
exit 2 with usage where the command prints one).
## Docs
No docs update needed: the pages document flags and their meaning, not
error strings; no flag was added, removed or renamed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01BkkAihA8Kp26hHS4uhkNGu
## Summary
- add `argent flow run --json-stream`
- stream flow progress as JSON lines
- finish with a JSON result or error
## Why
`--json` prints only after a flow finishes. `--json-stream` lets scripts
and CI see live progress.
## Why this is a separate PR
This is useful without the custom-tools feature in #856.
## Validation
- `npm test -w @argent/cli -- flow.test.ts` — 106 tests passed
- `npm run typecheck:tests -w @argent/cli`
- `npm run build -w @argent/cli`
---------
Co-authored-by: Filip131311 <contact@filipkaminski.com>
A `launch:` on Chromium now starts its own Electron instance, so nested
e2e flows and mid-flow relaunches work as they do on iOS/Android.
Previously the runner booted one app per run and rejected any second
chromium launch.
A chromium device id _is_ its CDP port, so a launch that starts an app
from scratch has to boot one and move the run onto it: `state.device`
became reassignable (steps already read it per call), and `state.owned`
tracks what the runner booted, torn down in reverse at run end. An
instance the runner did not boot — pinned via `--device`, or
auto-detected — is attached to, never killed. Relaunching the same app
kills its instance and waits for the process to actually exit, or the
replacement loses a race against the app's single-instance lock.
The runner needs a device before step 1, so `resolveRunDevice` follows a
leading `run:` chain to find the launch the run _begins_ with: a
fragment composing a chromium e2e flow now boots instead of failing with
"No booted chromium device found". A launch declaring no app id for the
run's platform stays an error — the run's platform is fixed start to
finish.
Also fixes a pre-existing teardown leak: the spawned handle is the npm
`electron` node wrapper, not Electron, so signalling it left the app and
its helpers running (reparented to launchd, still in the dock). Teardown
now sweeps the child's process group.
**Behavior change:** a fragment leading with `run:` into a chromium e2e
flow now boots a fresh instance where it previously attached to a
running one; pass `--device` to keep attaching.
## Tests
Cover the boot/attach/settle three-way, teardown ordering and the group
sweep, device rebinding after a nested launch, same-app retire ordering,
cancellation mid-boot, both "no chromium app declared" guards, and the
leading-`run:` chain (multi-hop, cyclic, no-launch). The guard,
exit-wait and sweep tests are mutation-checked.
`argent flow run` now accepts a directory and runs every flow in it
sequentially:
```
argent flow run .argent/flows --recursive
```
Discovery uses the same acceptance rules as `flow list` (lowercase
`.yaml`, safe stem, readable file), sorted for a deterministic run
order. `-r`/`--recursive` walks subdirectories too, skipping
dot-directories and `node_modules` and never following directory
symlinks. A directory named `foo.yaml` still batches (stat wins over
extension).
Batch output is failures-only: each flow prints a `[i/n] <path>` header,
its failing/errored steps — numbered exactly as a single-mode rerun
would number them — and a per-flow summary; the run ends with a
`PASS/FAIL — N flows: X passed, Y failed, Z skipped` verdict. `--json`
prints one aggregate object with per-flow reports. Exit code reflects
the batch (0 pass, 1 any failure, 2 discovery/config errors).
`--output` exports keep the single-run `<output>/<flow>/` layout, keyed
by subdirectory in recursive runs so same-stem flows in different
directories can't clobber each other's snapshots.
**Failure semantics:** a flow failing its steps — or one the tool-server
rejects as invalid (bad YAML, unparseable step) — fails alone and the
batch continues. An infra error (transport death, unclassified failure)
stops the batch and counts the remaining flows skipped. To make that
distinction possible, tool-error responses now carry the server's
failure signal (`error_code`/`error_kind`) on the wire — on the
streaming terminal error line and on JSON error bodies — and the tools
client surfaces server-answered errors as a typed `ToolInvocationError`.
Against a pre-signal server this degrades to stopping on any throw.
Not breaking: single-file runs behave exactly as before.
`argent flow run` now takes an explicit YAML path instead of a
saved-flow name:
```
argent flow run .argent/flows/checkout.yaml --platform ios
```
The path may point anywhere on the local filesystem; the filename stem
names the run's report/artifacts, so it must be `[A-Za-z0-9_-]+` with a
lowercase `.yaml` extension. A bare name fails with the exact
replacement command, and `argent flow list` prints paths instead of
names.
Because sibling `run:` files and `__baselines__` resolve beside the
YAML, the CLI now requires the auto-started local tool server and
refuses `ARGENT_TOOLS_URL` / `argent link` routing. The `flow-execute`
tool stays remotely callable via `name` + `project_root`, and gains a
`flow_path` parameter (exactly one of `name`/`flow_path`) that is only
honored when the file-input boundary verified it as a real host path —
uploads and boundary bypasses are rejected, so it can't execute
arbitrary server YAML.
Also: the flow-name pattern is now a single shared export from
`@argent/registry`, and the recorder composes a `flow_path` call into a
`run:` step when it targets a sibling of the recording.
# Flow E2E
This PR turns flows from replayable tool-call recordings into a
declarative, portable e2e test system that runs headlessly via `argent
flow run <name>` — no LLM in the loop, non-zero exit on failure,
suitable for CI.
Agents can still replay flows via `flow-execute` and get the full output
from each step after it finishes.
## Flow format
Flows are YAML files in `.argent/flows/`, now built from **directives**
interpreted by the runner rather than raw tool calls:
- `launch` — terminate + relaunch the app under test (per-platform app
id map supported); its presence marks a flow as **e2e** (controls its
own start state) vs. a **fragment** (runs against current state,
declares an `executionPrerequisite`, composable via `run:` with
cycle/depth guards).
- `tap`, `type` (focus handshake + optional submit), `scroll-to`
(momentum-free, increments by half the container, `within:` for nested
scrollers), `await`/`assert` (`visible`/`exists`/`hidden`/`text`
conditions), `wait`, `snapshot`, `echo`, and raw `tool:` as escape hatch
— still fully supported, so previously recorded flows keep working
unchanged.
- **Selectors** are `{ text, identifier, role }`; a bare string is a
loose selector resolving identifier-first, then visible text. Any
directive failure hard-stops the flow; later steps report `skip`.
## Execution engine
- Flows store no device id — the runner binds a device at replay
(explicit `--device`/`--platform` or the single booted one).
- Selector resolution runs against **platform-unified UI trees**: iOS
native view hierarchy (full testID coverage, gated on native-devtools
connecting), Android full accessibility hierarchy (full resource-id
coverage), Chromium CDP DOM, and the Vega automation toolkit. A shared
flatten/text-hoisting pass lets `text` assertions read what an
identified container visibly shows. Vega is remote-driven, so touch
directives fail with guidance to use `tv-remote`/`keyboard`.
- Steps wait for the tree to **settle** (two consecutive identical
reads) before acting; `launch` additionally gates on platform
tree-source readiness instead of silently degrading.
## Visual snapshots
`snapshot` diffs a full-resolution screenshot against baselines in
`.argent/flows/__baselines__/<flow>/`, keyed by platform + resolution,
with configurable `maxMismatch` and `--update-baselines`. The status bar
is pinned for the whole run (iOS `simctl status_bar`, Android demo mode)
so clocks/battery don't drive diffs. Baseline/current/diff images are
returned as artifacts on failure (and the written baseline on first
capture/update); a clean pass carries none, keeping reports and agent
context lean.
## Recording
The record-live workflow (`flow-start-recording` → `flow-add-step` →
`flow-finish-recording`) now emits directives automatically: coordinate
taps are rewritten to portable selector `tap:` steps (selector captured
from the pre-tap tree), a leading `restart-app` becomes `launch:`,
`flow-execute` of a sibling fragment becomes `run:`, and device ids are
stripped. The skill instructs the agent to polish the saved YAML
afterwards (fold tap+keyboard into `type:`, swipes-to-reach into
`scroll-to:`, awaits into sugar form).
## Also included
- `argent flow run` / `argent flow list` CLI with per-step report
rendering, artifact materialization, and `--json` output.
- New structured failure codes for flow validation, device resolution,
composition, and devtools readiness.
- Extensive unit test coverage (tree adapters, selector matching,
scrolling, composition, type focus, snapshots, status bar).
Ignacy Łątka