Add one semantic finger flick for dismissing cards, paging carousels,
opening drawers, and other gesture-driven actions distinct from
goal-seeking scroll-to. Support bare directions plus from with exactly
one of direction, to, or by, optional settle, and positive finite
duration.
Resolve selector or normalized-point anchors through the shared target
path. Direction uses the Maestro geometry presets and now rejects
anchored gestures that would collapse or travel opposite to the
requested direction. Relative by deltas saturate supplied axes to [0, 1]
and reject any effective zero or sign reversal. Resolved starts must
remain on-screen, while explicit endpoints stay author-controlled.
Keep parse and serialization invariants aligned for programmatic steps,
improve unknown-key hints and selector diagnostics, surface swipe
targets in run reports and recording summaries, share
positive-millisecond validation, and reuse the common flow test harness.
Dispatch gesture-swipe on touch platforms and gesture-drag on Chromium,
reject touch directives on Vega, and document when to prefer scroll-to
or a raw system-edge gesture.
A `run:` value is now a YAML path resolved against the directory of the
flow file that contains the step, matching the CLI cutover:
```yaml
- run: login.yaml
- run: ../shared/login.yaml # cross-directory composition now works
```
Bare names fail at parse with a migration hint (`run: login` → `run:
login.yaml`). The filename stem stays the fragment's report attribution;
reports also carry the as-written path as `target`. Cycle detection
compares realpath-canonical paths (same stem in two dirs ≠ cycle; same
file via symlink/spelling = cycle). Snapshot baselines are unchanged —
keyed by and stored beside the root flow.
An uploaded root flow with `run:` steps is now rejected up front with a
contract error instead of a mid-run temp-dir ENOENT (its referenced
files stayed on the client, and relative resolution in the temp dir
could reach host files); same for a relative chromium app path. The
recorder emits the sibling form (`run: <name>.yaml`).
**Breaking:** existing `run: <name>` directives fail at parse until
renamed to `run: <name>.yaml`. Snapshot baselines for a saved flow whose
YAML is a symlink now anchor beside the real file (the symlink target's
directory) — move any committed `.argent/flows/__baselines__/<flow>/`
set beside the real file once, or the next run fails with a missing
baseline. A `run:` target must also resolve inside the project root or
beside/below the flow file that references it (arbitrary host YAML is no
longer reachable), and an uploaded flow with a `snapshot` step is
rejected up front (its baselines cannot anchor to the per-call temp
dir).
Press-and-hold on an element — context menus, reorder activation — with
the same resolution behavior as `tap` (settled tree, auto-wait, no
auto-scroll) and the same options shape: a bare/strict selector, or
`long-press: { on: <selector>, duration: <ms> }` with the selector
nested under `on` so option keys never mix with selector fields.
The default 800ms clears both platforms' recognizers (iOS 500ms,
Android ~400ms). Touch platforms dispatch one `gesture-custom` train
(`Down`, then `Up` delayed by the duration) so the hold length is
exact; Chromium maps to a mouse press-hold-release (`gesture-drag`,
from == to) — a desktop context menu is a right-click and is
deliberately not aliased. Vega rejects upfront with the other touch
directives. No coordinate form: a point long-press stays the
`gesture-custom` escape hatch.
# Flow E2E
This PR turns flows from replayable tool-call recordings into a
declarative, portable e2e test system that runs headlessly via `argent
flow run <name>` — no LLM in the loop, non-zero exit on failure,
suitable for CI.
Agents can still replay flows via `flow-execute` and get the full output
from each step after it finishes.
## Flow format
Flows are YAML files in `.argent/flows/`, now built from **directives**
interpreted by the runner rather than raw tool calls:
- `launch` — terminate + relaunch the app under test (per-platform app
id map supported); its presence marks a flow as **e2e** (controls its
own start state) vs. a **fragment** (runs against current state,
declares an `executionPrerequisite`, composable via `run:` with
cycle/depth guards).
- `tap`, `type` (focus handshake + optional submit), `scroll-to`
(momentum-free, increments by half the container, `within:` for nested
scrollers), `await`/`assert` (`visible`/`exists`/`hidden`/`text`
conditions), `wait`, `snapshot`, `echo`, and raw `tool:` as escape hatch
— still fully supported, so previously recorded flows keep working
unchanged.
- **Selectors** are `{ text, identifier, role }`; a bare string is a
loose selector resolving identifier-first, then visible text. Any
directive failure hard-stops the flow; later steps report `skip`.
## Execution engine
- Flows store no device id — the runner binds a device at replay
(explicit `--device`/`--platform` or the single booted one).
- Selector resolution runs against **platform-unified UI trees**: iOS
native view hierarchy (full testID coverage, gated on native-devtools
connecting), Android full accessibility hierarchy (full resource-id
coverage), Chromium CDP DOM, and the Vega automation toolkit. A shared
flatten/text-hoisting pass lets `text` assertions read what an
identified container visibly shows. Vega is remote-driven, so touch
directives fail with guidance to use `tv-remote`/`keyboard`.
- Steps wait for the tree to **settle** (two consecutive identical
reads) before acting; `launch` additionally gates on platform
tree-source readiness instead of silently degrading.
## Visual snapshots
`snapshot` diffs a full-resolution screenshot against baselines in
`.argent/flows/__baselines__/<flow>/`, keyed by platform + resolution,
with configurable `maxMismatch` and `--update-baselines`. The status bar
is pinned for the whole run (iOS `simctl status_bar`, Android demo mode)
so clocks/battery don't drive diffs. Baseline/current/diff images are
returned as artifacts on failure (and the written baseline on first
capture/update); a clean pass carries none, keeping reports and agent
context lean.
## Recording
The record-live workflow (`flow-start-recording` → `flow-add-step` →
`flow-finish-recording`) now emits directives automatically: coordinate
taps are rewritten to portable selector `tap:` steps (selector captured
from the pre-tap tree), a leading `restart-app` becomes `launch:`,
`flow-execute` of a sibling fragment becomes `run:`, and device ids are
stripped. The skill instructs the agent to polish the saved YAML
afterwards (fold tap+keyboard into `type:`, swipes-to-reach into
`scroll-to:`, awaits into sugar form).
## Also included
- `argent flow run` / `argent flow list` CLI with per-step report
rendering, artifact materialization, and `--json` output.
- New structured failure codes for flow validation, device resolution,
composition, and devtools readiness.
- Extensive unit test coverage (tree adapters, selector matching,
scrolling, composition, type focus, snapshots, status bar).
Jakub Piasecki