The motion oracle borrowed screenshot-diff's DEFAULT_THRESHOLD, a value
sized for a baseline held across sessions and machines and so obliged to
absorb real drift. This comparison holds two captures one poll apart
from a single live session, where a static screen reads back
byte-identical: both capture paths are lossless PNG and the downscale is
deterministic. The borrowed value put the gate at ~25.5 levels/channel,
which mattered because spatially uniform change is all-or-nothing —
every pixel moves by the same amount, so it clears the gate on all of
them or on none, and MOTION_FRACTION is never the deciding term. A
cross-fade slower than ~1.4s therefore counted zero pixels and reported
settled mid-animation, over a screen still painted and still
hit-testing; and being settled rather than timed-out, degradedReason
stayed silent while updateBaselines could adopt a mid-fade frame as
golden.
0.03 (~7.6 levels/channel) keeps a margin over the +5/channel drift the
tests treat as the noise floor and moves that cliff out ~3.3x, to ~4.5s
at full contrast. It stays a rate floor rather than a duration floor, so
the comment now names the gate as the only term bounding what is left.
This supersedes the parity pin below it: coordinated drift is covered
instead by pinning the value and the direction, so restoring parity
trips a test rather than silently re-widening the blind spot. The
+25/channel case gives way to a two-sided bracket at +7/+8, and two
samples of a 2s indigo-over-white dismissal — a distance deliberately
between the new gate and the old — pin the regression itself.
The parity pin stopped the two constants drifting apart but not
together: the behavioral tests bracket the tolerance only loosely from
above, so raising both flow-pixels' PIXEL_THRESHOLD and screenshot-diff's
DEFAULT_THRESHOLD to 0.2 widened the per-pixel gate — the whole motion
oracle — with the suite still green. Assert the shared value alongside
the mirror.
Dropping `&& settled.converged` from either arm of the crop settle
mapper left the suite green: without it a crop snapshot whose settle
never converged reports clean on the skipped arm, and swaps the
timed-out note for an unavailable one on the pixel-dark arm — blaming
a missing backend for a screen that simply never stopped moving. Both
shapes are now driven through the real settle and frame resolution — a
churning hierarchy that never matches two fingerprints, and a dark
capture whose sticky verdict survives a restart — with the full reason
strings, read counts, and capture counts asserted, so each mutation
fails exactly its own test.
The settle-timeout catch's timed-out half — the only report that a
snapshot was captured while the hierarchy source never answered — was
pinned by nothing: collapsing the catch to settled left the suite
green while an update-baselines run adopted a screen never proven
still under a bare, healthy-looking write. The new case hangs every
hierarchy read, drives the real settle to its typed timeout at exactly
the action deadline, and pins the full degraded reason, the elapsed
budget, the untouched capture backend, and the written file — leaving
both halves of the ternary independently load-bearing.
The latch's timed-out override and its outage catch were pinned, but
the in-loop guard covering the remaining pixel-dark shortfalls —
unavailable, and skipped without convergence — was held by nothing:
widening it to every verdict left the suite green while a snapshot
whose stillness was already proven picked up a spurious unavailable
note. The new case drives the real machinery through a stale-but-
settled first round and four completed pixel-dark freshness retries,
asserting the exact undegraded write plus the 15-read/6-capture shape;
the mutant fails on the suffix and the counts alike.
The revalidation read is the combined settle's newest await point and
its abort exit had no direct coverage — every abort case landed on a
different read. Fabricating a converged-and-fresh result there turned a
cancelled run into a dispatched gesture recorded as a pass, with the
raw-coordinate shape uniquely exposed: it accepts settled pixels
without consulting freshness. The new case reaches a genuine settled
verdict on a capture-capable device, aborts exactly on read 3, and
pins the skip status, the run-aborted reason, no dispatched gesture,
and the capture and read counts.
The raw-coordinate path's abort guard is the only thing turning a
cancelled settle into the uniform aborted skip, and it had no abort
coverage: replacing it with a literal-point return dispatched the
gesture to the device after the client gave up, recorded as a pass —
runTap re-checks nothing before invokeOnDevice. The new case aborts
inside the settle's pair-completing read and pins the skip status, the
run-aborted reason, and that no gesture-tap reaches the device.
Deleting the visual === 'settled' disjunct left the suite green: the
tests named for it asserted only ok and coordinates, both satisfied by
the terminal best-effort dispatch at deadline exhaustion. The existing
pair now pins the dispatch instant (the first settled round at the 5s
combined phase window, not the 7.5s deadline), and a new pair covers
the transition the clause exists for — a source that stops hanging and
starts failing after pixels already settled taps the literal point it
never needed the tree for, pinned by ok, dispatch time, and a read
count proving the failing source is never consulted.
settleSnapshot keys its stale-settled override on the exact string the
pixel loop emits at budget exhaustion, but every snapshot test scripted
settleTree wholesale — relabelling the sleep-clamp arm to unavailable
left the suite green while a stale-settled update-baselines run wrote a
bare baseline with the degradation silently dropped. The new test
drives the real settleTree and real settleSnapshot end to end: a
stale-but-settled first round, a freshness retry whose animator burns
the pixel budget through the sleep-clamp arm, and the degraded write
asserted as the full reason string, pinned by read and capture counts.
Every comparison test fed the motion oracle grey — each channel term
could be deleted independently, and misindexing blue onto alpha (the
byte the docstring says is ignored) survived too, admitting motion
confined to one channel: a navy scrim fading in over black. Full-frame
red-only, green-only, and blue-only changes (plus the mid-amplitude
navy case) now register as motion, and an alpha-only change registers
as still — killing each channel-drop mutant and catching the alpha
misindex from both directions.
MOTION_FRACTION's comment claimed the fraction sits far below any
screen-filling transition — false for spatially uniform change: only
pixels individually clearing the per-pixel gate (~25 levels/channel
between captures) count, so a slow fade, dim, or tint shifts every
pixel by the same sub-gate amount and measures zero changed pixels by
construction, whatever it covers or how long it runs. The comment now
states the rate-floor contract and names the class the degradation
note can never fire for.
Two guards justified themselves with 'a zero-budget settle would
misreport a healthy tree source as an outage' — impossible: with no
budget no read ever launches, so settleTree can only throw
FlowTreeSettleTimeoutError, which both surrounding catches already
convert to the exact outcome the guards produce. The guards are real
micro fast-paths (skip one doomed settle round); the comments now say
so instead of stating an invariant the next person tuning these
deadlines would falsely reason from.
An already-spent caller deadline collapses into timed-out where the
flow-actions twin splits out not-attempted — deliberate here, but
undocumented and unpinned: relabelling the arm to aborted left the
suite green while turning an outage-fallback snapshot from a degraded
pass into a skip blaming a cancellation that never happened.
The collapse is now justified in prose (the outcome feeds only snapshot
degradation reporting, with no freshness contract for the twin's split
to protect, and the note text is honest for both expiry shapes) and
pinned twice: a zero-budget settlePixels call resolves timed-out with
no backend consulted, and the real fallback driven through runSnapshot
after a window-consuming outage settle degrades instead of skipping.
The simulator-server writes its temp PNG before replying, and the reply
is the only place the path is learned — so an abort landing mid-capture
severed the fetch, capturePixels soft-failed without ever learning the
path, and the file was orphaned, breaking the stated ownership contract
(the capture promise decodes and deletes its own temp file when it
eventually completes).
captureFile now passes no signal, matching the Chromium arm that never
took one: cancellation stays responsive because settleWithin abandons
the promise, while the capture self-completes on its own bounds, learns
the path, and cleans up in the finally on every path.
The pixel phase's not-attempted arm hardcoded treeFresh: true on the
grounds that zero time passed since the converged pair — false whenever
the awaited capability probe itself spent the budget: a cold iOS
runtime-kind lookup is a real simctl round trip, so the pair could be
~7s old while the settle reported it safe, and the verdict flipped from
honest to dishonest on whether that probe resolved just before or just
after the deadline.
The arm now compares the pair's convergence instant against the pixel
window: a window already spent at convergence means the probe answered
from cache and staleness is bounded by the final-read reserve, while a
window the probe consumed marks the tree unsafe — the same reasoning as
the probe-deadline sibling.
A tree-phase read outliving the source read budget returned the carried
treeFresh: after one successful read, a hung second read handed back a
tree up to the full 7.5s budget old while reporting it safe for selector
coordinates. scroll-to — the one deadline-less settleTree consumer — was
the visible casualty: its stale-round skip never fired for the case it
was written for, so one 8s hierarchy read dispatched a swipe off the
stale tree and misread the scroll as ended.
The hung-read exit now returns treeFresh: false. That is unconditional
because no path reaches it with a recent successful read in hand: every
phase window is capped by the read budget, so a non-first poll only
launches with budget open, and a first read starting past the budget
holds no successful read and throws instead. The treeFresh docstring
gains the fourth staleness source.
One combined settle on a screen with perpetual tree-invisible motion
polled captures to the caller's hard deadline minus the 250ms reserve,
consuming the entire action budget: waitForFrameResult got a single
resolution attempt instead of retrying across the window, a late retry
whose no-read window closed threw FlowTreeSettleTimeoutError out of the
step, and every directive on such a screen paid ~6s of added wall clock.
Polling for a pixel pair now stops at the ordinary phase window, the
same boundary tree polling already honors; the hard ceiling bounds only
work in flight when it closes — a first capture's first-frame-aware
budget plus the one warm capture completing its pair. After a
phase-bounded pixel timeout the mandatory revalidation read runs on the
ordinary source read budget instead of being cut to the reserve, and
waitForFrameResult treats a late retry's settle timeout as deadline
exhaustion (best-effort resolve from the last settled tree) rather than
a step error, matching the throw taxonomy.
Collapsing COMBINED_HARD_TIMEOUT_MS to the 3s tree window left the
suite green while cutting the slow-first-frame capture the constant's
pixel term exists for — scroll-to's round-0 settle, the one combined
settle with no caller deadline, silently degraded instead of proving
stillness. Drive a 6.25s first capture plus a matching warm capture
through the deadline-less settle and assert full convergence, and pin
the constant's composition directly.
The crop mapper's skipped arm was held by nothing: deleting it sent a
converged tree-only settle to the timed-out fallback, stamping every
Vega and Apple TV cropOn snapshot with a degradation note for a pixel
phase that is architecturally absent. Drive runSnapshot through the
mapper with the no-backend settle shape and assert the exact undegraded
baseline-write reason.
Collapsing the finalTreeDeadline ternary to the 250ms reserve left the
suite green while cutting every ordinary-latency post-pixel read short:
a 400ms uiautomator revalidation after a fast pixel phase came back
non-fresh, the verdict selector resolution refuses, wasting a round per
settle. Drive the everyday scenario and assert full convergence on
exactly three reads and two captures.
The two pixelsDiffer colour tests probed only the extremes, leaving the
band where a real transition lives — a dim, a fade, a crossfade —
unexercised: a 5x threshold mutant swallowed a full-frame 120-per-channel
change with the suite green. Bracket the tolerance from both sides
(grey120 registers, +25 stays still) and assert the documented mirror of
screenshot-diff's DEFAULT_THRESHOLD directly, exporting both constants.
Every existing settlePixels test fed a constant frame, hung the capture,
or aborted, so nothing past the first comparison was pinned — deleting
the predecessor advance left the suite green while a transition coming
to rest looking different from its first frame would burn the whole
window and report timed-out. Drive A/B/C/C through the real settle and
assert settled on exactly four captures.
The docstring predated the settle rework and claimed a throw means every
read in the window failed. Name the two typed shapes and their meanings
— a completed failure with zero successes is a proven outage, a window
with no completed read at all is a settle timeout that proves nothing —
and that any successful read in hand turns deadline exhaustion into a
best-effort result, the contract resolveTargetPoint and settleSnapshot
are written against.
SnapshotSettle.treeFresh was written at every construction site and read
nowhere — runSnapshot keys degradation on the outcome alone — and its
docstring claimed the cropOn path consumes it, which it never did. The
snapshot settlers now return the bare outcome; the settle-level
SettleResult.treeFresh consumers are untouched.
The throw on the preceding line already narrows lastTree to defined, so
the ternary's undefined arm could never be taken — and undefined is the
value settleTree reserves for aborts alone. The exit now matches its
sibling deadline exits.
scroll-to is the only settleTree caller that supplies no action
deadline, so its tree reads fell back to the 5s combined-phase cap —
below the budgets the sources themselves get (getHierarchy 15s,
uiautomator up to 20s) — and a healthy 5.2s hierarchy read failed only
scroll-to steps as a fabricated settle timeout. The unowned fallback now
matches DEFAULT_ACTION_TIMEOUT_MS, putting the timeout cliff at the same
7.5s every deadline-owning caller already has.
settleSnapshot's stale-settled latch treated every later reading as
unable to take back established stillness, so a retry that re-ran the
whole pixel phase without re-proving a matching pair was discarded and
the deadline exit wrote an undegraded baseline — one transient slow
post-pixel read made a mid-animation capture look clean under
--update-baselines. A completed retry ending timed-out now returns that
degradation over the latch (the capture happens after the settle, so the
retry describes the captured screen better than the stale claim); only
pixel-dark retries — outages and hard tree timeouts — still honor it.
When the tree pair converges inside the final-read reserve of the
caller's deadline, the first capture was never launched — no backend was
consulted and zero time passed — yet the settle reported visual
"timed-out", the verdict snapshots degrade on. Align the code with the
SettleResult contract: a phase that never ran returns the sticky visual
verdict (skipped by default) with the still-fresh converged tree, and no
revalidation read is spent. Budget exhaustion after a capture ran stays
an honest pixel timeout.
A combined settle whose healthy tree read consumed the caller's whole
budget threw an untyped Error from the capability-probe deadline check —
outside the settle error taxonomy, hard-stopping steps whose tree had
just converged. The budget check now distinguishes a probe that was
never consulted (no time passed — the converged tree is returned fresh)
from a probe that was awaited and outlived the wait (returned non-fresh),
so raw-coordinate gestures dispatch best-effort and snapshots degrade
honestly instead of erroring.
The existing rejection tests fail every read (an outage, thrown before
the post-pixel stage) and the restart tests move the tree via a
successful differing read, so a describe blip landing exactly on the
revalidation read exercised an untested arm: re-seed with the stable
fingerprint and restart. Two tests now pin it by read/capture counts —
the seeded restart converges on one extra read (5 total), the pixel
phase re-runs after the restart (4 captures), the blip surfaces nowhere,
and the settle returns fully converged and fresh.
The post-pixel tree revalidation gets as little as a 250ms reserve, which
one slow uiautomator/full-hierarchy read can exceed on a healthy source,
and review showed settle outcomes leaking into step failures. Settling
now stabilizes best-effort and never fails a step by itself; the only
settle-caused failure left is a genuine tree-source outage (every read
in the window failed):
- Snapshots retry a settled-but-stale settle for freshness on the shared
action deadline (the planned cropOn selector will consume the tree),
accept the stale-but-settled screen at exhaustion, and no longer
refuse --update-baselines on a timed-out settle — the write proceeds
with a degradation note instead of failing.
- Raw-coordinate tap/long-press accept a fresh tree OR settled pixels,
retry to the action deadline, then dispatch anyway: literal
coordinates depend on nothing the settle failed to prove.
- Selector resolution prefers fresh trees while budget remains and falls
back to the last valid settled tree at exhaustion, so a slow freshness
read cannot masquerade as "element not found".
- scroll-to skips a stale round instead of failing the step.
So callers can trust visual:"settled", settleTree downgrades it to
"skipped" on a tree-phase restart: the tree moving after the pixels
matched proves those pixels stale.
Fixes#644.
## Before
```
$ argent flow run echo-only # .argent/flows/echo-only.yaml: two `echo` steps
[Tool:flow-execute] No booted device found. Pass a device id or platform explicitly.
Available devices: 18BE573F-… (ios, Shutdown), <70 more>
```
and when a device *was* booted:
```
PASS on 18BE573F-… — 0 passed, 0 failed, 0 errored, 0 skipped
```
## After
```
$ argent flow run echo-only # nothing booted
Flow "echo-only"
› first step
› second step
PASS — 0 passed, 0 failed, 0 errored, 0 skipped (no test steps)
```
## Symptom 1 — the device
Whether a device is needed is decided from the flow's own steps, before
resolution.
Classification is **per step kind and defaults to needing one**, so a
kind added later inherits
today's behaviour instead of silently running against no device — and
the `never` binding makes
leaving a new kind unclassified a compile error, backed by a test keyed
on the step union so it
fails to build rather than to run.
Two classifications worth calling out:
- **`when` always needs a device**, whatever its body contains, because
the guard reads one itself
(its platform, or its view tree). A `when: {platform: ios}` block
containing only narration still
resolves.
- **`run` needs one without the fragment being read here.** The named
flow is resolved at run time;
resolving it a second time would duplicate that lookup and could
disagree with it if the file
changed in between. A deliberate over-approximation: composing a
narration-only fragment still
resolves a device. Stated in the code and in the tool description.
For `tool:` steps the tool's own schema decides, via a superset of the
keys the runner injects —
`device` is included because a nested `flow-execute` step drives a
device without being handed the
run's own. A tool that declares no input at all
(`stop-all-simulator-servers`) is device-free, and
reading its absent schema must not throw — it previously would have.
`ExecState.device` is nullable, which is the safety mechanism rather
than just typing: the compiler
enumerates every site that acts on a device, each of which now says so
via one `deviceEnv()`
narrowing. `flow-actions.ts`, `flow-visual.ts` and `ActionEnv` are
untouched. The executor also
re-checks each step against the **same predicate**, so if the scan and
the executor ever disagree
the step reports it instead of failing obscurely deeper in.
A run that resolved no device reports `device: ""`, and neither renderer
claims it ran somewhere.
## Symptom 2 — the counters
Reproducing this showed the issue's framing is slightly off: `--json`
doesn't only disagree with the
text, it disagrees with **itself** — counters all zero while its own
`steps[]` marks both steps
`"pass"`. So the CLI text was a faithful rendering of the server's
counters.
**The counters are right and stay as they are.** Narration is
deliberately excluded, and it should
be: after a hard stop every remaining step *including narration* is
reported skipped, so counting it
would inflate `skipped` on every real failing run — a worse and far more
common lie than the vacuous
one being reported here. It would also contradict the JUnit model in
#578, where echo is
`<system-out>` and not a `<testcase>`.
What was missing was saying *why* zero. Both the CLI and the MCP
renderer now append
`(no test steps)`, and only on a pass — a cancelled run can be a FAIL
with all four counters zero,
where "no test steps" would read as though the failure had no cause.
No wire format changes: `steps[].kind === "echo"` already distinguishes
counted from uncounted steps,
so the JSON is self-describing without widening the status enum.
## Conflicts, for whoever lands second
This is a hot area — seven open PRs touch `flow-run.ts`.
- **#578** rewrites the CLI summary line I also change (one hunk, one
7-line function, semantically
independent — resolution is keep-both), and its
`attachFailureDiagnostics` dereferences
`env.device.platform`. **It will need a null-device guard**, or every
failure in a device-less flow
silently degrades to `read-failed` diagnostics. Happy to rebase onto it
rather than the reverse.
- **#585** rewrites `resolveRunDevice` at the same seam and edits the
same description sentence.
- **#538** rewrites `flow-device.ts`. Worth noting it makes
`resolveFlowDevice` reject a physical
iPhone for flows — skipping resolution means a narration-only flow now
passes on a
physical-iPhone-only host, which I think is desirable but is a second
behaviour change on this seam.
All new tests are in new files, since every existing flow test file in
the blast radius is rewritten
by an open PR.
## Behaviour changes a reviewer should weigh
- A device-free flow no longer prints `PASS on <id>` **even when a
device is booted**. That is the
point: otherwise an identical flow reports differently in CI (no
simulator) than on a laptop with
one open. An explicit `--device` is still honoured and still named.
- `--platform` is ignored for a device-free flow, where it previously
errored with nothing booted.
- A 0.18.1 CLI against a newer server would print `Flow "x" on ` —
cosmetic version skew.
## Verification
Live against the built CLI: device-less flow with nothing booted (the
reported bug) and with a device
booted (identical output, no ambient attribution); `--json` showing
`device: ""`; explicit `--device`
still naming it; a mixed flow and a composed flow both still demanding a
device; a mixed flow with a
device booted still reporting `PASS on <id> — 1 passed`.
24 new tests. tool-server 3103 passed, CLI 284 passed, MCP 78 passed —
no existing test modified.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---
> **#697 is stacked on this branch** — it conflicts with this one in
`flow-run.ts`'s import block, so it was rebased on top. Merge this
first.
Fixes#617.
## Reproduced three ways
**The reported false green**, using an image whose 2× downscale is
lossless so no resampling noise can mask it:
```
baseline 800x1600 current 400x800 (exact half)
→ status: unchanged pixel_mismatch: 0% (size difference never mentioned)
```
**Worse on real screenshots** — a 1152×648 capture vs its
aspect-preserving half:
```
→ status: changed pixel_mismatch: 0.35% changed_areas: 8
Text changes: Disappeared "General" / Appeared "Remotes and Devices"
```
The content is identical; every region and both OCR "text changes" are
resampling artifacts. So the tool doesn't merely stay silent about the
size difference — it manufactures a plausible-looking regression report
out of noise.
**The control shows the asymmetry is the bug** — aspect-*breaking*
correctly reports `dimension_mismatch: expected=1,152x648
actual=1,152x400`. Sizes are disclosed exactly when the tool refuses to
compare, and never when it silently rescales.
## The fix
`PngDiffResult` gains `sizeNormalization` (both original sizes plus the
size compared at), populated only when a rescale actually happened — so
its presence *is* the signal that the comparison carries resampling
uncertainty. The summary prints it directly under the status:
```
- status: resized_no_change
- size_normalized: baseline=800x1,600 current=400x800 compared_at=400x800
- the inputs share an aspect ratio but not a resolution, so the larger was downscaled before
comparing; any pixel or text differences below may include resampling artifacts, and the diff
images are at compared_at rather than full size
- re-capture the baseline at the same scale as the current image (screenshot with scale: 1.0)
to compare without resampling
```
**`resized_no_change` is checked last**, after the difference checks, so
a rescaled comparison that *did* find something is still a plain
`changed` — a caveat status must never soften a real regression. There's
a test pinning that ordering specifically.
**The name matters.** `unchanged_after_resize` contains `"unchanged"`,
so the one naive gate this protects — `summary.includes("status:
unchanged")` — would still read it green. `resized_no_change` contains
neither `"unchanged"` nor `"changed"`.
## What I deliberately did not do
**Report `dimension_mismatch` whenever sizes differ** — the issue's
primary suggestion. The normalization is deliberate:
`screenshot-diff/index.ts:292-294` states that same-aspect normalization
*"keeps a scaled capture diff-compatible with a baseline saved at any
scale"*. The `screenshot` tool defaults to **0.3** for iOS/Android while
the diff's live capture is full-res, and on Android the live side falls
back to the default scale when full-res capture fails. Hard-failing
would break the case this code exists for.
**Expose the internal `normalizeSizes` flag.**
`screenshot-diff-tool.test.ts:10-23` is titled *"rejects public tuning
options so defaults stay internal"* and asserts exactly this kind of
param is refused. And the better remedy is the one the summary now gives
— re-capture the baseline at the same scale, which the skill already
instructs and which removes the resampling rather than reporting it.
Worth reconsidering separately if a user actually wants "green means
byte-identical".
*(An earlier draft proposed exposing it via `z.coerce.boolean()`. Worth
recording why that was dropped beyond the posture conflict: in zod,
`z.coerce.boolean("false")` is **`true`** — so a user asking for strict
comparison would have silently got normalization anyway, re-creating
this exact bug through the fix's own escape hatch.)*
**Suppress the OCR noise.** Text analysis runs on each *original file at
its own resolution* and only rescales the resulting bounds, so a
diff-time filter cannot separate "OCR read a 576px image badly" from a
real text change. Suppressing it whenever normalization occurred would
silence text analysis on every scaled comparison — a new, worse false
green of exactly the class this issue is about. Disclosure is the honest
treatment; running OCR on the normalized pair is a separate change to
`text-diff`'s contract.
## Tests
Three new cases plus three strengthened ones: the false green now
discloses and no longer reads `unchanged`; a normalized comparison with
a real change still reports `changed` *and* still discloses; equal sizes
say nothing; and the retina case still compares rather than
hard-failing.
Mutation-verified both halves — removing the status fails the #617
regression test; moving its check before the difference checks fails the
ordering test.
Full tool-server suite green — 3089 passed. No existing assertion needed
changing, which is also why this survived: nothing pinned it.
## Notes for the reviewer
- The flow snapshot path is untouched and **cannot** reach
normalization: full-screen baselines key on the capture's own pixel
dimensions (`flow-visual.ts:212`), so a differently-sized capture misses
the baseline entirely and fails; `cropOn` diffs already pass
`normalizeSizes: false`.
- `SummaryStatus` is unexported and nothing switches on it — the status
reaches the outside only as text inside `summary`, which the MCP layer
passes through verbatim. No skill doc, flow assertion or e2e script
matches on a status string.
- The diff artifacts were already produced at the normalized size; that
was simply never stated. The caveat now says so.
Fixes#634.
```
before: [Tool:chromium-tabs] Not supported
after: [Tool:chromium-tabs] Could not open a tab: Not supported. An Electron app has no
browser-level target creation, so a tab cannot be made from outside it — have the app open
one (e.g. `window.open()` through `debugger-evaluate`) and it will show up in
`chromium-tabs list`. The list, select and close actions are unaffected.
```
The old text is Electron's own CDP answer to `Target.createTarget`,
passed through untouched. The
limitation is real, but the message names neither the cause nor a way
forward, and reads as though the
tool were unavailable — when only this one action is.
## Two things I checked before writing the message
- **The workaround actually works.** On the Electron testbed I ran
`window.open('about:blank')` via
`debugger-evaluate` and the new window appeared in `chromium-tabs list`
as `t2`. I didn't want to
ship advice I hadn't run.
- **The rest of the tool really is fine** — `list` returns correctly
both before and after.
## Why the cause is kept rather than matched on
`Target.createTarget` can fail for reasons other than Electron's missing
browser shell — a dropped
connection, or a real Chrome refusing the url. The CDP client keeps only
the error's `message` and
discards its numeric code, so branching would mean matching on the
string `"Not supported"`, which
breaks the moment Electron rephrases it.
Instead any failure is wrapped, with the original message embedded. The
explanation is worded as a
property of Electron rather than a claim about *this* failure, so it
stays accurate when the cause is
something else — and the underlying error is still right there to read.
## Verification
Live against the Electron testbed (Electron 42 / Chrome 148): the
message above is the real output,
and `list` still works alongside it. tool-server 3087 passed, plus a new
test asserting the message
keeps the original cause *and* carries the reason, the workaround, and
the note that the other actions
are unaffected.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Patch bump. First release whose npm tarball carries `mcpName` (added in
#598), which unblocks the MCP registry publish (`mcp-publisher publish`
rejects `0.18.0` because its manifest predates the field).
`server.json` is bumped alongside `packages/*` for the first time —
`scripts/check-workspace-versions.mjs` now enforces it (`All workspace
packages and server.json are at 0.18.1` locally).
| Commit | Change |
| --- | --- |
| `eea6c27c` | `style`: fix prettier formatting on main (#604) |
| `7e4ece69` | `chore`: server.json for the official MCP registry (#598)
|
| `fc09612f` | `feat(config)`: `recordings.directory` — configurable
save location for screen recordings (#601) |
| `cccda92d` | `feat(config)`: `ios.additionalDeviceSets` — additive
array of extra CoreSimulator device sets (#542) |
Version-only diff, rebased on main as of #604.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Problem
Field report: *"electron needs focus to be tested."* Analysis confirmed
three focus/visibility dependencies in the Chromium/Electron surface:
1. **Hidden window → throttled renderer.** A minimized/occluded window's
compositor throttles: mouse-input acks stall **~5 s per event** (they
wait on hit-testing), wheel scrolls hang (`gesture-scroll` already
refused), `visibilityState` flips to `hidden`.
2. **Unfocused window → focus-gated app logic breaks.**
`document.hasFocus()` is `false` before any input arrives;
focus/blur-dependent UI misbehaves.
3. Focus-gated web APIs (clipboard already worked around it locally with
`Page.bringToFront`).
## Fixes
**1. CDP focus emulation on every page session** (`primePageSession`, né
`enableCoreDomains`): `Emulation.setFocusEmulationEnabled(true)`,
best-effort, re-applied automatically after tab switches (Playwright
precedent). Live A/B on the Electron testbed: identical
boot→unfocus→probe sequence gives `hasFocus:false` on main, `true` on
this branch. Bonus verified by A/B (kill server → same minimized window
flips to `hidden`): in current Chromium/Electron, emulation also pins
reported visibility to `visible` and keeps input unthrottled while
genuinely minimized — minimized tap latency **~5 s → ~65 ms**, wheel
scroll works.
*Trade-off (same as Playwright ships):* while attached, the app can
observe neither real blur nor real hidden, so pause-on-hidden behavior
is unobservable during a test.
**2. Anti-throttling switches for every argent-booted Electron app**
(`--disable-background-timer-throttling
--disable-backgrounding-occluded-windows
--disable-renderer-backgrounding`). Sessions are created lazily and die
with the idle-exiting tool-server while the app outlives it — the flags
cover that detached lifecycle plus emulation-less runtimes. Also covers
flow-booted apps via the shared `bootElectronApp`.
**3. Hidden-window fail-fast extended from gesture-scroll to gesture-tap
+ gesture-drag** via shared `assertChromiumWindowVisible`, under a new
dedicated `CHROMIUM_WINDOW_HIDDEN` failure code (scroll migrated;
`failure_stage` strings unchanged so telemetry stays joinable —
analytics keyed on `CHROMIUM_INPUT_INVALID` +
`chromium_scroll_window_hidden` should switch to the new code). With
emulation active the guard is a backstop that fires only where emulation
couldn't be applied — exactly where the stall is real. Keyboard/button
stay unguarded on purpose: measured live, key events skip hit-testing
and stay fast on hidden windows.
## Verification
- Unit: 3075 tool-server + 87 registry tests green, `typecheck:tests`
green, new tests for emulation priming, boot args, and the guard (all
three tools).
- Live matrix on the Electron testbed (dev tool-server): focused
baseline; unfocused (`hasFocus` true pre-input, typing lands); minimized
booted app (tap 69 ms, scroll works, typing + slider drag land,
screenshot fine); minimized **external** un-flagged app on :9222 (tap 63
ms, scroll 460 ms — emulation alone suffices while attached).
- Plan + implementation went through three adversarial review rounds
(scope, CDP semantics, failure-code taxonomy/telemetry compat, doc
accuracy).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
> Recreates #543 — GitHub auto-closed it when the base branch of the
stacked PR (#542) was deleted on merge, and a closed PR cannot be
reopened after its head is force-pushed. Same commit, rebased onto main.
## Summary
**Was stacked on #542** (now merged). Consumes the
`ios.additionalDeviceSets` configuration across the iOS tool surface, so
simulators living in extra CoreSimulator device sets — e.g. **Radon
IDE's** `~/Library/Caches/com.swmansion.radon-ide/Devices/iOS` — are
discovered by `list-devices` and drivable by every tool, by UDID,
exactly like default-set devices.
### The core piece: a UDID → device-set map (`utils/ios-device-sets.ts`)
simctl scopes every operation to one device set, so each per-device call
must know the target's owning set. The new module owns that mapping and
the single argv choke point:
- **Learned from discovery**: `list-devices` and the simulator watcher
enumerate the default set + every configured set and `rememberDeviceSet`
each sighting — any tool used after a list hits a warm cache.
- **Lazy fallback**: an unknown UDID triggers one `simctl list` probe
per configured set, cached on hit. An unknown-everywhere UDID resolves
to the default set **without** caching (a device that appears later is
found on the next call) — preserving pre-feature behavior and error
messages byte-for-byte.
- **`simctlArgsForUdid(udid, args)`** builds the full argv (`--set`
injected only for additional-set devices); `simctlPrefix` covers
multi-call sites that resolve once; sync variants
(`simctlArgsForUdidSync`, `cachedDeviceSetForUdid`) cover the
`ChildProcess`-returning and `execFileSync` call sites, each of which
runs after an async step that warmed the cache.
- Config is **read off disk per call** (the #534 design): editing
`ios.additionalDeviceSets` applies without a tool-server restart.
- **Guard**: a configured set whose directory doesn't exist is never
queried — simctl would materialize the dir, turning a config typo into a
stray directory.
### Wired consumers
- **`list-devices`** — enumerates every set, tags additional-set devices
with `deviceSet`, documents the field.
- **`simulator-watcher`** — polls every set (a set failing to list
throws → the tick is skipped, so a transient error still can't
mass-dispose tracked services); warms the map for the injection path.
- **simulator-server blueprint** — passes `--device-set` (the same flag
Radon IDE passes to this shared binary); default-set spawns unchanged.
- **`ax-prefs`** — the pre-boot AX plist now lands under the owning
set's device dir (previously hardcoded to the default
`CoreSimulator/Devices` root, so the write would land nowhere for these
devices and describe would degrade).
- **`boot-device`** — set-aware shutdown/boot/bootstatus; skips the
Simulator.app attach + `CurrentDeviceUDID` write for additional-set
devices (one Simulator.app instance displays one device set — these run
headless and are streamed, exactly how Radon IDE presents them).
- **Every remaining per-device simctl call site**: launch-app /
restart-app / reinstall-app (both the direct impls and the
`SimctlBackend` shared handlers), open-url, settings-permissions
(privacy + install probe), the tvOS screenshot fallback, status-bar
pin/restore, device-shutdown, the ios-host `simctl spawn` family behind
describe/native-devtools, tv-control's AX daemon, and the native
profiler's sync helpers (cache warmed at `startNativeProfilerIos`).
**Out of scope**: `sim-remote` — the remote orchestrator host has its
own device-set story.
## Testing
- **Unit**: new `ios-device-sets.test.ts` (mapping, lazy probe + caching
semantics, no-probe fast path with no sets configured, discovery
enumeration/tagging/dedup, missing-dir guard); blueprint test asserts
the `--device-set` argv; the profiler shell-injection mock extended for
the new import. Full tool-server suite green: **276 files / 2752
tests**; `tsc --build`, eslint, prettier clean.
- **Live, end-to-end on a real Radon IDE device** (iPhone Air, iOS 26.4,
in `com.swmansion.radon-ide/Devices/iOS`), driving this branch's
tool-server over HTTP with the config set at project scope:
- `list-devices` → the Radon device appears, tagged with its `deviceSet`
(invisible to argent 0.16.1 on the same machine);
- `boot-device` → boots headless in its set, no Simulator.app window;
- `screenshot` → real home-screen PNG via simulator-server spawned with
`--device-set`;
- `launch-app` (Settings) → launched via `simctl --set launch`;
- `describe` → full accessibility tree from the Settings screen, i.e.
the AX bootstrap (set-aware prefs + in-sim ax-service daemon via `simctl
--set spawn`) works;
- clean shutdown; default-set devices meanwhile listed and behaved
exactly as before.
## Summary
Adds a `recordings.directory` configuration entry (built on the #534
config system) that lets users choose where finished screen recordings
(mp4) are saved, instead of the fixed `<project>/.argent/recordings/`.
- **Schema entry** (`configuration-core`): scopes `project` + `global`,
`prioritize-local` merge (project wins when both are set). Picked up
automatically by `argent config set/get/unset/list`.
- **Value semantics**: absolute path, `~`-prefixed (expands to home), or
relative — relative anchors at the project root (home dir when not in a
project).
- **Client-side wiring** (`argent-tools-client` `durableSaveTarget`):
the wire `saveDir` hint stays `.argent/recordings` (wire contract
unchanged → old/new client-server interop). Past the existing client
allowlist, the recordings hint is remapped to the configured directory.
## Security model unchanged
The allowlist exists because `saveDir` arrives from a possibly-hostile
remote tool-server. That gate still runs first and is unchanged: a
server can only *select* the sanctioned recordings category, never name
a destination. The configured directory comes from the client's own
trusted config files. Exclusive never-overwrite writes (`name (2).mp4`),
the durable download size caps, and the symlink real-path guard all
remain; for a configured dir the realpath check degenerates to "the dir
resolves to itself" (a user-chosen symlink is legitimate, matching how
the default base may be symlinked). A test asserts hostile wire values
(`.git`, `..`, absolute) stay rejected even with config set.
## `argent link` interaction
Materialization always runs on the **client** (co-located → copy from
`hostPath`; remote → download over `/artifacts/:id`), and the config is
read there at materialize time — so with a remote tool-server the mp4
lands on the client host per the *client's* config, on both paths.
## Testing
- New unit tests: 10 in `argent-tools-client` (absolute/`~`/relative
resolution, local-vs-global precedence, both materialization paths,
hostile wire values with config set) + 4 in `configuration-core` (scope
writes, trim, precedence, validation).
- Suites green: tools-client 188, configuration-core 87, tool-server
screen-recording 62, CLI config 16; full workspace vitest 4727 pass (3
pre-existing `scripts/*.test.mjs` node:test files don't parse under root
vitest — unrelated).
- **Live E2E** on an iPhone 15 Pro simulator via the branch tool-server:
real recording captured, then materialized through the branch client
with configs written by the real `argent config` CLI in a sandboxed
HOME/project. 7 scenarios green: default location, global-only (`~`
path), project-only (relative), both-set → project wins (co-located
**and** remote download with Bearer auth), unset fallbacks, `argent
config list` rendering. Remote-downloaded mp4 ffprobe-verified
(byte-exact size, 4.17 s duration); name-collision suffixing observed
live.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Adds tool-owned, human-readable interaction messages for every tool
definition.
Each tool defines its own started, completed, and failed wording (as
implemented in https://github.com/software-mansion/argent/pull/367.
Messages can use validated parameters, successful results, and safe
failure codes. They are to avoid sensitive input such as typed text,
cookie values, JavaScript expressions, and URL credentials.
Bridges the gap between current Argent
<img width="954" height="493" alt="Screenshot%202026-07-28%20at%2015 48
37"
src="https://github.com/user-attachments/assets/2c689a93-8ba8-4208-b35e-66eb53c96671"
/>
and current agent-device
<img width="1188" height="476" alt="Zrzut ekranu 2026-07-28 o 20 14 42"
src="https://github.com/user-attachments/assets/6aed0f38-625b-4748-83e5-d384e79266c2"
/>
## Test
Selected fields from the generated JSONL:
```jsonl
{"level":30,"type":"tool.invoked","toolId":"list-devices","toolInvocationId":"decb864f-51c1-41ee-91fc-15bc2494e268","msg":"Listing devices","time":"2026-07-29T10:37:19.896Z"}
{"level":30,"type":"tool.completed","toolId":"list-devices","toolInvocationId":"decb864f-51c1-41ee-91fc-15bc2494e268","durationMs":96.77333299999918,"msg":"Listed 22 devices and 0 AVDs","time":"2026-07-29T10:37:19.993Z"}
{"level":30,"type":"tool.invoked","toolId":"screenshot","toolInvocationId":"8504abb8-83ed-4490-8210-3355d6c638c9","msg":"Capturing screenshot","time":"2026-07-29T10:37:24.503Z"}
{"level":50,"type":"tool.failed","toolId":"screenshot","toolInvocationId":"8504abb8-83ed-4490-8210-3355d6c638c9","failureSignal":{"error_code":"SIMULATOR_SERVER_READY_EXITED","failure_stage":"simulator_server_spawn_ready","failure_area":"tool_server","error_kind":"subprocess","failure_command":"simulator_server","failure_exit_code":1},"durationMs":750.4639999999999,"msg":"Failed to capture screenshot: SIMULATOR_SERVER_READY_EXITED","time":"2026-07-29T10:37:25.254Z"}
{"level":30,"type":"tool.invoked","toolId":"screenshot","toolInvocationId":"63b938fc-3d86-4eb5-9972-a008474ffb01","msg":"Capturing screenshot","time":"2026-07-29T10:38:01.107Z"}
{"level":30,"type":"tool.completed","toolId":"screenshot","toolInvocationId":"63b938fc-3d86-4eb5-9972-a008474ffb01","durationMs":25.509208000003127,"msg":"Captured screenshot argent-screenshot-chromium-cdp-9222-1785321481132-81477.png","time":"2026-07-29T10:38:01.133Z"}
```
## What
Adds the CSS combinators to flow selectors, read off frames. Any action
or condition can now scope its matches by where an element sits relative
to another:
```yaml
- tap: { text: Delete, within: { id: profile-card } } # CSS "A B" — inside that container
- tap: { role: Switch, next: { text: Wi-Fi } } # CSS "A + B" — the nearest follower
- assert: { visible: { role: Button, after: { text: Danger zone } } } # CSS "A ~ B" — any follower
- assert: { hidden: { any: true, within: { id: empty-state } } } # CSS "*" — the universal selector
```
| Scope | CSS | Reads as |
| --- | --- | --- |
| `within: <sel>` | `A B` | the match's frame sits inside the scope
element's frame |
| `after: <sel>` | `A ~ B` | the match follows the scope element in
reading order |
| `next: <sel>` | `A + B` | as `after`, narrowed to the nearest follower
of each anchor |
Plus `any: true` — the CSS `*` universal selector, accepted only
alongside a scope and never beside `text`/`id`/`role`.
The child combinator `>` has deliberately **no analog**: flow trees
flatten to leaves under one synthetic root, so parent/child structure
does not survive to replay.
## How
- **Geometric, not ancestry-based.** Frames are what survives
flattening, so every scope is resolved from them by `findAll` in
`ui-tree-match`. The synthetic root never counts as a scope element, and
every relation requires a *distinct* node (nothing scopes itself).
- **Reading order is row-band aware.** An element follows the anchor
when it starts below the anchor's bottom edge, *or* shares its row band
and sits entirely to its right. That band rule is what makes `{ role:
Switch, next: { text: Wi-Fi } }` resolve the Wi-Fi row's own switch even
though the taller switch's frame starts a couple of pixels *higher* than
the label's — a raw top-y comparison gets that backwards. `followKind`
is the single classification both the follower test and the nearest-pick
grouping read, so they cannot disagree.
- **`next` is the union over anchors**, exactly as CSS `A + B` is: three
rows on screen give three switches, one per label.
- **Scopes compose and nest**, capped at `MAX_SELECTOR_SCOPES` (6)
counted across the whole relation *tree* — a size bound, not a depth
bound, because three relations branch and the runner's loose-alternative
expansion is exponential in the bare-string scopes it finds.
- **Flow-YAML only.** `await-ui-element`'s strict schema rejects
`within`/`after`/`next`/`any`, so a scope can never be silently dropped
on that path.
- Adding a fourth relation is a **compile error**: `RELATION_RESOLVERS`
is a `Record` keyed by the relation union, so a new scope cannot be
wired through parse/serialize/reporting and then ignored by the match
engine.
## Testing
- **Unit:** full tool-server suite green — **3006 passed / 1 skipped**.
New coverage in `ui-tree-match.test.ts`, `flow-utils.test.ts`,
`flow-selector-scopes.test.ts` (renamed from
`flow-within-selector.test.ts`, now covering all scopes) and a new
`flow-skill-docs.test.ts` that parses every YAML snippet in the skill so
the docs cannot rot. Each fix is mutation-checked: reverting it fails at
least one test.
- **Real-device E2E** (Pixel 3a, Bluesky on Metro 8081, via a branch
tool-server): three flows, 20 steps, all green — geometric ordering both
ways, two `text` asserts that pin `next`'s pick *by identity* out of
five candidates, empty-scope `hidden` semantics, and two observable taps
(`next`-scoped and `any: true`-scoped) that navigated to the *adjacent*
topic rather than the anchor's own.
- **iOS not covered.** Native-devtools does not attach on either booted
simulator in this environment, so *every* flow selector fails there — an
unscoped `{ text: "Video" }` fails identically. Verified environmental,
not caused by this branch: the globally-installed argent reports the
same, direct `SIMCTL_CHILD_DYLD_INSERT_LIBRARIES` injection also fails
to connect, and it survives a forced simulator reboot on both iOS 18.3
and 26.5. Worth re-running the iOS leg once that is healthy.
## Notes for reviewers
Two deliberate deviations from CSS, both documented in the skill:
- **`next` is looser than `+`.** Where `A + B` matches nothing unless
the immediate sibling is a `B`, `next` keeps looking and returns the
nearest match further on. That is what makes it survive the wrapper and
spacer nodes a flattened tree is full of — but a row *missing* the
control you asked for resolves to the next row's rather than failing.
Happy to switch to strict `+` semantics if the loud failure is
preferred.
- **"follows" is not transitive.** Against a tall anchor an element can
follow something that follows the anchor without following it directly,
so nesting `after` scopes is not the same as chaining `~`.
Other things worth a look:
- `nearestAfter` and `afterTester` are direct scans, not indexed. An
earlier sorted index was a *second* implementation of the nearest rule
and the two disagreed on frames near the tolerance; it also measured
2-13x slower than the scan on realistic shapes, with a prune that
provably never fired. The quadratic tail is documented with measured
numbers: 1.2 ms for a single-anchor `next` at the 12k-node tree cap,
~555 ms only when the anchor selector matches essentially every node.
- A bare-string *anchor* cannot self-correct the way the loose fallback
usually does — a decoy `testID` matching your anchor text wins the
identifier pass whenever anything sits after it. The skill tells authors
to spell sibling anchors as maps.
- `MAX_SELECTOR_SCOPES = 6` replaces the previous
`MAX_SELECTOR_WITHIN_DEPTH = 10`, so a `within` chain deeper than 6 that
parsed on the earlier commits of this branch now rejects. Nothing
released depends on it.
---------
Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com>
Jakub Piasecki