mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
jpiasecki/flow-repeat
267 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jakub Piasecki
|
ea59cdf894 | docs(skills): put the repeat section in the flow-yaml reference contents | ||
|
Jakub Piasecki
|
8bc616f2e2 | test(tool-server): pin the repeat scope on invokeSubTool's unattributed path | ||
|
Jakub Piasecki
|
4694abcb4f | fix(flow): echo a non-finite bound as the number the author wrote | ||
|
Jakub Piasecki
|
8247374fda | fix(flow): name the { times } bound in the repeat type and its shape error | ||
|
Jakub Piasecki
|
aed8b1d894 | fix(flow): point a drain's idle refusal at the end of the block's steps | ||
|
Jakub Piasecki
|
11f29b4962 |
style: cut the repeat stack's comment prose back to the load-bearing whys
Trims the multi-paragraph comment blocks across the repeat directive work. Cuts hand-maintained call-site inventories (blockSteps' "read at nine sites", isStructuralBlockMarker's "five sites"), rejected-alternative debates (MAX_REPEAT_ITERATIONS' three paragraphs on why the product is not capped), and worked examples of step numbering that the tests already pin. Keeps the non-obvious whys: the drain's probe-not-wait grace and why settling belongs to the body, why the cancellation line is a skip and not structural, why the device-free branch does not stamp structural, the CLI marker blank's forward lean. Comments only - no behaviour change. |
||
|
Jakub Piasecki
|
977c872b61 | docs(flow): price the block directives' device answer at all three host shapes | ||
|
Jakub Piasecki
|
a6f4b3bdcc | docs(flow): say a drain's guard is a probe and the body owns the settling | ||
|
Jakub Piasecki
|
e483c782f5 | fix(flow): spell the block remedy in the run-time prerequisite refusal | ||
|
Jakub Piasecki
|
286d8a20d7 | test(flow): pin that the hoist boots the launch a times block leads with | ||
|
Jakub Piasecki
|
abd3215084 | test(flow): pin the fragment-snapshot fence a second run: hop below a block | ||
|
Jakub Piasecki
|
5e982d5b2d | docs(flow): stop calling when the only block kind in the device walks | ||
|
Jakub Piasecki
|
71b9b245f8 | fix(flow): scope a converged drain's stand-in skips to the guard, not the step | ||
|
Jakub Piasecki
|
0159df7c49 | fix(flow): name the failed final read on a drain's cap verdict | ||
|
Jakub Piasecki
|
347aca8710 | docs(flow): justify the snapshot refusal at every bound, not just above one | ||
|
Jakub Piasecki
|
502de45d1a | fix(flow): echo the bound, not the body, when a repeat body holds a snapshot | ||
|
Jakub Piasecki
|
ae22c79bb0 | fix(flow): point the prerequisite refusal at a step the flow actually has | ||
|
Jakub Piasecki
|
8ce451e74a | docs(flow): scope the description's paste claim and name until's vocabulary | ||
|
Jakub Piasecki
|
3f411416bc | fix(flow): close a times block only over the iterations it still owed | ||
|
Jakub Piasecki
|
85753ab313 | fix(flow): give the leading-launch scans the run's own repeat scope | ||
|
Jakub Piasecki
|
33cbd124d1 | test(flow): pin one closing line per enclosing repeat on a nested cancellation | ||
|
Jakub Piasecki
|
ad34cfc15e | fix(flow): keep a repeat bound error's echo inside the render cap by echoing the bound | ||
|
Jakub Piasecki
|
024fdb368f | fix(flow): give up the launch scan on a hop the fragment-snapshot fence refuses | ||
|
Jakub Piasecki
|
43c6fea1a2 | fix(flow): fence the tool: flow-execute route into a repeat body like run: | ||
|
Jakub Piasecki
|
694da6f173 | test(flow): pin fragment attribution on a repeat block's terminal lines | ||
|
Jakub Piasecki
|
9e4b87a8c4 | test(flow): pin the guard parser's error naming across every until-reachable site | ||
|
Jakub Piasecki
|
4b518bc4db | test(flow): pin the count bound at every repeat spelling, not just the bare sugar | ||
|
Jakub Piasecki
|
c7ea4916b0 | test(flow): pin that a drain converging on its max-th iteration passes | ||
|
Jakub Piasecki
|
09649e4694 | test(flow): pin the drain's exit paths — hard stops, stand-in reasons, singular wording | ||
|
Jakub Piasecki
|
a271683758 | docs(skills): complete the drain's closing-line list and name the { times } map form | ||
|
Jakub Piasecki
|
41787d7d55 | fix(flow): fail a run: step whose fragment snapshots inside a repeat block at load time | ||
|
Jakub Piasecki
|
ddcf1655f5 | fix(flow): see a leading launch through times-mode repeat blocks in both launch scanners | ||
|
Jakub Piasecki
|
452c334bfc | fix(flow): walk the upload preflight through blockSteps so a repeat: cannot hide a run: | ||
|
Jakub Piasecki
|
a350bdcb02 |
feat(flow): add repeat: blocks — bounded, reconvergent repetition
A `repeat:` block runs the same steps several times, with the same sibling
shape as `when:` so the two nest in each other and around `run:` for free.
Two bounds, exactly one per block:
- repeat: 3 # literal count, 1-100
steps: [...]
- repeat: { until: { hidden: X }, max: 15 } # drain, max defaults to 10
steps: [...]
It earns its place by adding no semantic power, only compression: `times` is
literally sugar for pasting the block N times, and `until` is the N-stacked-
`when:`-blocks drain pattern named — with the one improvement the desugar
cannot express, that hitting the cap FAILS instead of silently leaving the
drain incomplete.
Decisions worth flagging in review:
- `until` is checked BEFORE each iteration, including the first, so an
already-drained list runs zero iterations. That is a PASS, not a skip: the
drain converged, there was simply nothing to drain. The authored steps
still report one skip line each, so no authored step drops out of the
report — not a line-for-line match with a run that had work to do, which no
`repeat:` block can offer (an entered one repeats its body and marks each
pass), just the same accounting of what the block was going to do.
- A block's children sit between its opening marker and its closing line, on
every path. That is why the three zero-iteration exits report the body
BEFORE their verdict: pushing the verdict first would hang depth-1 lines
under a depth-0 line that closes the block rather than opening it, and the
same block would bracket two different ways depending on whether the drain
found work.
- Repeat is NOT retry. A failure inside any iteration hard-stops the flow;
re-running a side-effecting iteration would double-fire it.
- `until` takes the when-guard vocabulary minus `platform` — the platform
never changes between iterations, so such a loop is infinite or empty by
construction. Parse error, not a runtime surprise at `max`.
- `snapshot:` inside a repeat body is a parse error: one baseline, N
comparisons, and iteration 2's screen legitimately differs.
- No loop variable. An index invites computing with it, and that is an
expression language; differing iterations are different steps.
- Nothing is padded. A `times` block cut short by a failure does not emit its
remaining iterations as skips (they are re-runs of steps already reported,
not authored steps left undone), and a drain never pads to `max`.
- The block's opening and per-iteration lines are `structural` (a new
StepReport field): reported, but excluded from the counts and from the
CLI's step numbering the way `echo` already is. Counted, they would scale
the totals with N and break the equivalence above — `repeat: 3` over one
tap has to report what those three taps pasted out report. A drain's
terminal line is NOT structural: it is the block's verdict, and its `fail`
at the cap is the only thing making a drain that never converged report
FAIL.
- Every report line states its thing once. The opening marker carries the
bound as its target and NO reason — the block has not run, so a reason
could only restate the bound, and for a drain in a second selector spelling
(`selectorLabel` builds targets, `describeSelector` reasons). The
cancellation line is the mirror image and the one terminal line that takes
a target: its reason names only the cancellation, so without one it read as
a bare `repeat — run aborted` belonging to no block in particular. The
drain's converged/cap/errored lines take no target — their reasons already
spell the condition out. The count pluralizes on both surfaces that render
it (stepTarget and the recorder's re-summarized listing): the bound starts
at 1, so `repeat: 1` is a flow an author can write.
- A cancelled block says so once, the same way, at either exit — the drain's
guard probe or after an iteration body, in both bounds. The abort test has
to precede the `stopped` test: a mid-body cancellation arrives with both
flags set, since execSteps skipped the body's remaining steps on its own
abort branch. An ordinary failure stays silent — the failing step's line
already explains the stop.
- stepRequiresDevice answers true for `repeat` unconditionally, like `when`.
A `times` block over only wait:/echo: steps genuinely needs no device, so
this is conservative, and the author pays for it before the run starts:
flowRequiresDevice says yes, so the run throws `No booted device found` at
device resolution, naming neither the repeat nor the fact that the same
steps unwrapped would have run device-free. Kept for one answer per step
kind; the cost is pinned by a test rather than left to be rediscovered.
Guard evaluation is shared with `when:` (probeGuard), so an indeterminate
probe errors the step in both: an unreadable tree must not end a drain early
nor keep it spinning. `until`'s parse errors are spelled `repeat.until` and
echo the whole `repeat:` step, since an `until` guard is not a step of its own
and the bare fragment appears nowhere in the author's file.
Serialization holds the parse-side bound only for the iteration count: a
`times`/`max` outside [1, 100] throws rather than emitting YAML the parser
would refuse, matching targetToYaml. The `until` guard is part of the bound and
is NOT checked — a hand-built text condition with no expectedText serializes to
`contains: ""`, which the parser then refuses — and neither are the block
body's own invariants. Both holes are shared with `when:` (the guard one
through waitToYaml, with `await:`/`assert:` too) and neither is reachable from
anything the parser produced, so they want one block-level guard if anything,
not a per-arm patch. The comments there say so rather than implying coverage
the code does not have.
41 flow-repeat tests; 605 flow tests pass; 3153 tool-server tests pass.
|
||
|
Hubert Gancarczyk
|
2740a3ec38 |
chore(knip): clear the parked dead-code backlog and drop the ceiling to 0 (#770)
> Built on top of #769, so this PR carries that commit too — the workflows only run for PRs targeting `main`, which is why it is based there rather than on the branch. #769 on its own is the minimal, low-risk fix for the currently-red job; if it merges first this shrinks to its own commit, and if this one merges first it closes #769 as well. ## What this does #572 landed the gate with 215 findings parked under `--max-issues`. That bounded the backlog but left it in place, and the first thing to touch it (#663) pushed the total to 218 and turned the job red. This clears the backlog and sets the ceiling to **0**, so the next unused export fails the job instead of consuming headroom nobody knew was there. | | before | after | | --- | ---: | ---: | | Unused exports | 47 | 0 | | Unused exported types | 156 | 0 | | Unused exported class members | 12 | 0 | | `--max-issues` | 215 | **0** | ## Scope check first Every workspace holding a finding is `private: true`. `@swmansion/argent` — the one published package — had **zero**, so nothing here moves a public API. ## Two shapes of fix **Still referenced inside its own file → drop `export`** (146 types, 36 exports). The declaration stays put; it just leaves the module's public surface. `tsc` keeps emitting these into the `.d.ts` as local declarations that the exported signatures reference. For the 146 **types** that is the whole story: they are erased, so the emitted `.js` is byte-identical. For the 36 **values** it is not. Dropping `export` from a `const`/`function`/`class` removes its `exports.X` binding from the emitted `dist/*.js` — 36 declarations across 26 files here. `packages/tool-server/dist/utils/device-info.js` no longer carries `exports.REMOTE_PREFIX` or `exports.VEGA_SERIAL_PREFIX`; `telemetry/dist/posthog.js` loses `exports.POSTHOG_PROJECT_TOKEN`; in the ESM-emitting `argent-installer`, `isTempRunnerPath` / `resolveLocalArgentDir` / `PACKAGE_ROOT` leave the module namespace object entirely. Nothing in the repo or in the `packages/argent-private` submodule at the pinned revision reads any of them by path, so nothing breaks. Flagging it because a `require()` of a built `dist/*.js` loses a dropped `export` exactly the way it lost the deleted `__resetAndroidDevtoolsInstallCache` this branch had to restore — so this half of the diff does deserve runtime scrutiny, not a skim. **Referenced nowhere → delete.** Orphaned test seams (`__test`, `_testValidators`), stale compatibility re-exports, and a few real orphans. The ones worth a reviewer's eye: - **`localSimctl`** — the `SimctlBackend` strategy is only ever constructed with `remoteSimctl`. What keeps the local iOS impl off it is the handler signature: `buildIosLaunchHandler` / `buildIosRestartHandler` read `services.nativeDevtools`, and `ios-remote` is the only platform that declares that service, so the local impl — which resolves native-devtools per device inside its own handler — has nothing to hand them and shells out to `xcrun` itself. - **`sendCharInsert`** — removing the `__sendCharInsert` test alias exposed the underlying helper as unreferenced. It is superseded, not unwired: the Chromium keyboard impl dispatches its own `type: "char"` event inline (`keyboard/platforms/chromium.ts:58`), which is exactly what this helper did. Chromium typing is unaffected. - **Error metadata fields** — `ServiceNotFoundError.serviceId`, `ToolNotFoundError.toolId` and their two siblings were never read. The `e.serviceId` in `registry-error-events.test.ts` is a field on the event object the handler pushes, not on the error class; the `err.toolId` in `http.ts` is `NotImplementedOnPlatformError`, a different class that keeps its fields. Flagging these in case they are wanted as deliberate diagnostics. Deleting `SourceMapsRegistry.toGeneratedPosition` / `findMatchingSource` cascaded into their only helper (`buildSourceCandidates`) and then into `projectRoot`, which had no remaining reader — so the constructor parameter goes too. `StubSourceMapsRegistry` existed only to pass `super("")`, so it goes with it. ## The three cross-workspace members `ArtifactStore.register`, `ArtifactStore.list` and `TypedEventEmitter.off` are all called from other workspaces — the artifact route and screenshot tools in `tool-server`, and `registry-listener` in `telemetry`. They are reported unused anyway, and the reason is the **unbuilt tree** the gate runs against: every workspace resolves `main`/`types` to a `dist/` that does not exist, so `@argent/registry` resolves to nothing and the cross-workspace edge never forms. Build the tree and all three findings disappear on their own; a control member that really is dead is still reported in the built run, so the pass is live either way. Nothing about this is specific to `classMembers` — every cross-workspace reference is invisible the same way. Each is exempted at the declaration with a `@public` JSDoc tag rather than by name in `knip.jsonc`, for one reason: **scope**. `ignoreMembers` matches a name across the whole workspace, so it would also hide a future dead `register`/`list`/`off` on any other class in `packages/registry`; `@public` binds to the member it is written on. What `@public` does **not** buy is a staleness check, and neither does `ignoreMembers`. Measured on a clean unbuilt worktree: a bogus `ignoreMembers` name and a redundant `@public` each leave the run at exit 0 with no hint, while a bogus `ignoreDependencies` name exits 1 with `Remove from ignoreDependencies`. `treatConfigHintsAsErrors` is live and simply has no producer for either exemption style — so when one of the three stops being earned, nothing says so, and the JSDoc at each member is the only record a re-audit has. Each one names its callers, and `off` records how to re-derive them (rename the member and read the `Property 'off' does not exist` errors; grepping `.off(` over-reports, since 14 of the 38 call-site hits are Node `EventEmitter`s). One correction to the commit message on that change: it says 35 hits where the exact count is 38 — 14 Node `EventEmitter`, 24 `TypedEventEmitter` (21 production, 3 in a test). The 14 and the 21 are right. `knip.jsonc`, `CONTRIBUTING.md` step 5 and the workflow's failure-explanation step all described a parked backlog and a ceiling to stay under. All three are rewritten for a gate that must simply come back empty, and all three now name the `@public` escape hatch — the remedy for a symbol whose only caller is out of knip's reach. ## Verification Run against a clean worktree with **no build output**, the way CI counts: - `npm run knip` — **exit 0, prints nothing**. Both passes empty, no config hints. - `npx tsc --build` — clean. - `npm run lint` (`eslint . --max-warnings 0`) — clean. - `npx prettier --check .` — clean. - `npm test --workspaces` — tool-server 329 files / 3857 passed; registry, telemetry, update-core, tools-client, mcp, cli green. - `npm run typecheck:tests --workspaces` — 13 workspaces define the script and all 13 are clean, but **the command itself exits 1**: npm errors `Missing script: "typecheck:tests"` for the three that do not define it rather than skipping them. Worth knowing before re-running this bullet and reading the exit code as a failure. End-to-end on a tool-server built from this branch, driven over its HTTP API: - **Chromium** (Electron smoke app) — `describe`, `screenshot`, `gesture-tap` (counter advanced 0 → 1), `POST /api/clipboard/text` read back out of the renderer, the WS `clipboardSync` command, and `debugger-connect` / `debugger-status` / `debugger-evaluate` (`sourceMapReady: true` with `StubSourceMapsRegistry` gone). - **iOS** — `boot-device`, `launch-app`, `describe` via ax-service, `screenshot`. - **tvOS** — `boot-device`, `launch-app` on an Apple TV udid, focus-driven `describe`. The registry snapshot then carries `NativeDevtools:<tvOS udid>`, confirming injection **is** resolved on tvOS through the local impl. - **Android** — `boot-device` on a dedicated AVD, `describe` via android-devtools, `gesture-tap`. - **Metro source maps** — the registry driven against a real 12.5 MB Metro `.map` over loopback, the way `Debugger.scriptParsed` drives it: allowlist passes and `waitForPending()` resolves on both, while retained heap after gc drops from 17.3 MB to 3.6 MB. ## Review follow-ups Five commits on top, one per finding. The base is back on `main`, so the workflows run again. - **`docs(launch-app)`** — the services comment explained the empty shape through `LaunchAppAndroidServices`, deleted here, and skipped `LaunchAppVegaServices`, which is still there. Same edit in `restart-app/types.ts`. - **`docs(registry)`** — the `@public` tag on `ArtifactStore.register` named 2 of 7 callers. Derived the full set the way the sibling tag on `off` prescribes: renamed the member, read the 16 `TS2339` errors `tsc --build` reports across flow-visual (5), screenshot (4), native-profiler-stop (2), screenshot-diff (2), native-profiler-analyze, react-profiler-analyze and screen-recording-stop. The re-derivation recipe is recorded too — `.register(` greps as badly as `.off(`. - **`docs(knip)`** — "the build, the tests and this gate all stay green on a wrong delete" held for argent-private and not for the cross-workspace case that justifies three of the four `@public` tags. Deleting `ArtifactStore.register` gives 16 `TS2339` errors and `9 failed | 2 passed` in `test/artifacts.test.ts`; only the gate stays green. Split in the workflow comment, the failure annotation and `CONTRIBUTING.md` step 5. - **`perf(debugger)`** — `doRegister`'s `data:` branch still base64-decoded and `JSON.parse`d a payload it dropped, inside a `catch` that swallows. Both arms returned the same `void`, so the branch collapses to an early return; `scriptUrl` / `scriptId` go with it, since their last reader was the deleted `this.maps.push(...)` and `doRegister` is private. - **`test(debugger)`** — the allowlist check moved to a top-level early return with nothing pinning it: delete the line and both SSRF files still pass 17/17. Two tests now assert `doRegister` consults it — four rejected URLs reach no `fetch`, and the loopback `*.map` Metro emits still does. Re-verified after: `npm run knip` exit 0 and empty on a `tsc --build --clean` tree, `tsc --build`, `eslint . --max-warnings 0`, `prettier --check .` all clean, and `npm test --workspaces` with tool-server at 329 files / 3854 passed. The one red test, `argent-installer`'s `globalPath returns ~/.config/opencode/opencode.json`, fails identically with these commits stashed — it reads the real `~/.config/opencode`, which holds a `.jsonc` on this machine. End-to-end on a tool-server run from this branch over its HTTP API, on every platform: - **Chromium** — `launch-app`, `describe`, `screenshot` (returned an artifact handle, `GET /artifacts/<id>` served the 137542-byte PNG back), `gesture-tap` (counter 0 → 1), `debugger-connect` / `debugger-status` (`sourceMapReady: true`) / `debugger-evaluate` (read `taps: 1` out of the page). - **iOS** — `launch-app`, `describe` via native-devtools, `restart-app`, `screenshot`. - **tvOS** — `launch-app`, focus-driven `describe` (12 focusables in TVSettings), `tv-remote`, `restart-app`. - **Android** — `launch-app`, `describe` via android-devtools, `restart-app`, `screenshot`. - **Vega** — `boot-device` on the `tv` VVD, `describe` via the automation toolkit. - **Metro** — the real `debugger-connect` / `debugger-status` / `debugger-evaluate` tools against a stand-in Metro (HTTP `/json/list` + CDP WebSocket) emitting three `Debugger.scriptParsed` events: a loopback `*.map` on a counting server, a 1.4 MB inline `data:` map, and `http://169.254.169.254/latest.map`. `sourceMapReady: true`, `loadedScripts: 3`, evaluate 42, the loopback map fetched exactly once and the metadata URL never. A malformed inline payload gives byte-identical output, which is the finding. ### Merge with `main` Putting the base back on `main` made CI resolve the merge, and the Unit Tests job went red on a conflict no side could see on its own: `#771` fixed the same red gate this branch's first commit did, the other way round — it made `component-names.test.ts` import `StrippedName`, `ComponentAnnotation` and `ComponentNameResolution` instead of dropping their `export`. Merged `main` in, reproduced it (`tsc --noEmit -p tsconfig.test.json` → three `TS2459`s) and took main's resolution: the three exports are back. The gate stays green because knip treats test files as entry points, so a type a test imports is not an unused export. All nine workflows are green on the head commit. |
||
|
Jakub Piasecki
|
2e182f7971 |
fix(flows): pin iOS tree reads to the launched app instead of auto-resolving every connection (#772)
## The problem Flow runs on CI iOS fail with `ViewInspector RPC timed out: Application.getState` at seemingly random steps - mid-flow after typing, on the first read after a fresh launch, in a probe flow 25 s after the original failure - while the app under test is provably healthy: spindumps taken at failure time show its main thread idle in the runloop, display links firing, and the injected devtools queue empty. The same flows pass locally. The chain: argent's iOS injection is simulator-wide (`launchctl setenv DYLD_INSERT_LIBRARIES` inside the simulator's launchd), so every process the simulator spawns afterwards loads the devtools dylib, not just the app under test. The dylib's constructor connects unconditionally (no UIKit or `com.apple.*` guard) and the tool-server registers whatever bundle id shakes hands, so Spotlight workers (`mdworker_shared`), background-launched system apps doing first-boot work (`MobileCal`), and bare daemons (nil bundle id, registered as `"unknown"`) all become connections indistinguishable from the app under test. Flow tree reads auto-resolved the target on every read: `queryFullHierarchyTree` passed `undefined` to `resolveNativeTargetApp`, which `Promise.all`s `Application.getState` over every connection. The dylib serves `getState` with a `dispatch_sync` onto the main thread, and a background-launched or suspended process never services its main queue, so its `getState` parks until the 5 s RPC timeout and the `Promise.all` rejects. One poisoned sibling connection fails every tree read in the run; the app under test is never even asked. That one mechanism explains everything the failures did. CI spindumps caught the poisoner red-handed (`MobileCal` parked in `HandleViewInspectorRequest -> CurrentApplicationState`) next to a healthy app. "Permanent wedge" vs "recovery" was just poisoner lifetime: a Spotlight worker exits and its connection drops with it, so reads recover; Calendar lingers and the whole job stays poisoned, fresh app launches and probe flows included. And it never reproduced locally because a warm simulator did its first-boot indexing and migrations long ago, while CI boots a pristine device and does exactly that work during the test window - which also made the failures correlate with typing and tapping purely by timing. ## The fix Pin the target instead of resolving it per read. `runLaunch` records the launched app on the run state once the tree-source gate passes, as `ActionEnv.treeTarget` - a `FlowTreeTarget` carrying the `bundleId`, whether the runner still vouches for it (`pinned`), and whether its probe has ever answered in this run (`probeAnswered`). `fetchFlowTree` threads that target down to `queryFullHierarchyTree`, the only platform source that consumes it. A pinned read talks to the pinned app and nobody else: it never calls `resolveNativeTargetApp` at all, and runs four gates of its own instead (`flow-ios-tree.ts`). A `com.apple.*` id is refused first, terminally and before anything is spent - a system app is never the app under test. Then `isConnected`, a map lookup, so a dead pin reports the dropped connection instead of driving env repair from 300 ms read polls, which could otherwise latch the device's process-wide give-up. Then a probe of the pinned app's own `getAppState`, and only that app's: a timeout before this pin has ever answered is ridden out as the cold start it usually is, one after an earlier answer is refused as an app that stopped servicing its main queue. Finally a foreground check on the answered state, as lenient over one app as auto-resolve is, so a flow that left the app fails with the crisp diagnosis instead of describing a screen that is not on screen. A poisoned sibling connection is never touched. The pin is only trusted while the runner can vouch for what is on screen, and it comes off in two tiers. The five `FOREGROUND_CHANGING_TOOLS` (`launch-app`, `restart-app`, `reinstall-app`, `open-url`, `button`) drop the target outright, because the launched app may no longer be on screen at all; `launch-app` and `restart-app` then re-set it from their own `bundleId` arg, unpinned, since they name the app they switched to - which is how a recorded flow switches apps, the recorder keeping them as raw `tool:` steps. Every other raw `tool:` step demotes the pin to an unpinned hint, its effect on the foreground being opaque to the runner; a launch attempt drops the target outright until it succeeds. Either tier also retires a proven tree-outage memo: a verdict proven against the pinned branch's four gates says nothing about the path the next read runs. An unpinned read is the pre-pin behavior with one arbiter added. `resolveNativeTargetApp` fans out over every connection, and only when that fan-out times out - the CI condition this change exists for - does the hint take the read, and only after passing the same terminal `com.apple.*` policy gate, the connections check, and a foreground-checked probe of its own. A resolution that answers always wins, including its deliberate "single app but backgrounded" refusal, so the arbiter never overrides a guard that fired: it only rides out a probe a stalled sibling made unanswerable. Nested `run:` fragments share the run state, so they inherit the pin, its demotion and its clearing. E2e flows begin with a `launch` by definition, so every selector read in a CI run happens pinned. Only iOS consumes the target; the other platforms' tree sources are per-device and never auto-resolve. The review rounds also surfaced that the `isInjectableBundleId` "library validation, the dylib can never load" rationale was disproven by this PR's own spindump evidence, so that text - and every other home of the same claim, `unreadableHierarchyReason` and `treeSourceGate` included - is rewritten as the policy it actually is (system apps are never the app under test; their observed connections are background processes whose reads hang or lie) with the gates' behavior unchanged. ## What this doesn't cover Auto-resolve's own fan-out is still all-or-nothing, so every read that reaches it without a hint the gates accept stays poisonable: top-level fragment runs, `describe` without an explicit bundle id, and the stretch after a raw `tool:` step whose hint is refused or cannot answer. Recording (`captureTapSelector`) now passes the session's last `launch` id as an unpinned hint, so it arbitrates the same timed-out fan-out, but it never pins - the recorder cannot vouch for the foreground app the way a `launch:` step can. Closing the rest means hardening auto-resolve itself (exclude connections that don't answer instead of failing the fan-out) and refusing the connections at the source (dylib constructor bail for non-UIKit and `com.apple.*` processes, plus a server-side handshake filter to cover already-shipped dylibs). Both are follow-ups. ## Tests Three files, and every behavior below is mutation-proven: disabling the seam, dropping the target from any single call site, removing an invalidation, or removing any one of the four gates each fails at least one dedicated test. The reviewers' repros were turned into tests rather than argued away. `flow-ios-tree-pinned-target.test.ts` works at the tree source. It reproduces the failure shape - a healthy app plus a poisoned `com.apple.mobilecal` sibling whose `getState` only times out - and proves a pinned read probes only the pinned app while an unpinned read still fails. It then covers the states auto-resolve could never reach: the terminal `com.apple.*` refusal, including its position ahead of the connections gate so a disconnected system app is not sent into a restart loop; a dead pin, where the dropped connection is reported and no env repair runs; both sides of the `probeAnswered` split; and the foreground guard over the states a real device produces (`inactive` under a permission dialog, `inactive` with no scenes, `background` still holding a foreground-inactive scene, and `background` with none). A second block covers the unpinned arbiter: the hint read on a timeout, refused when its own probe answers from the background, and rethrowing the fan-out timeout when the hint is disconnected, cannot answer, or when the resolution failed for any other reason. `flow-launch-pins-tree-target.test.ts` proves the runner wiring through the real `fetchFlowTree` seam: every read after a `launch` carries the pin across all five read paths (assert, tap, type, rotate, idle), retry polls included; fragments inherit it; a foreground-neutral `tool:` step demotes it and a foreground-changing one that names no app drops it outright; a later launch re-pins on a fresh target with the ride-out re-armed; and a launch-free run keeps the fallback. `flow-tool-step-tree-target-e2e.test.ts` drives `createRunFlowTool` with only `resolveService` stubbed, so the real `fetchFlowTree` and `queryFullHierarchyTree` run. It pins what the two tiers cost against a wedged sibling: the launched app still arbitrating across a foreground-neutral `tool:` step, the outage memo being retired when that step demotes the pin, the `com.apple.*` and foreground refusals staying in force through the demote, and a `hidden:` assert failing after the demote instead of going green against a backgrounded app. |
||
|
Hubert Gancarczyk
|
50c9580e30 |
fix(tools): make a schema failure name the parameter that was sent (#718)
> Stacked on #717. ## The problem An invalid tool call returned Zod's raw issue array. Someone who wrote `flow_name` instead of `name` got back, depending on the tool: ``` Invalid params for tool "flow-start-recording": [{"expected":"string","code":"invalid_type","path":["name"]}] Invalid params for tool "flow-execute": [{"code":"custom","path":["flow_path"],"message":"Pass exactly one flow source: name or flow_path."}] ``` The first names the parameter the tool **wanted** and never the one the caller **sent**. The second — flow-execute's, whose `name` is optional so `flow_path` can stand in — names neither: it is anchored on the one source field the caller had no reason to send. Either way the mistake is invisible, and working it out costs a whole turn. ## What changes **1. `describeParamIssues` — one sentence per bad parameter.** Missing fields read as `` `name` is required (string) and was not provided`` rather than as a type error about `undefined`. Unknown keys are named and path-qualified, so a key nested in `selector` is not reported as a bare name that contradicts the top-level list printed beside it. The message closes with the caller's own keys — `You sent: \`flow_name\`, \`project_root\`.` — which is what makes the mistake self-evident. Key names only, never values: this string reaches logs, telemetry and the agent transcript, and a params object can carry a secret. Wired into both entry points: the registry's `invokeTool` and the HTTP boundary. **2. `flow-execute` accepts `flow_name` as an alias for `name`.** The tool is called `flow-execute`, so "the flow's name" spells itself that way; this is a mistake worth absorbing rather than rejecting. The name is resolved in code rather than by the schema — a Zod `required` failure would once again name only the field it wanted — and the error text names the spellings that are silently discarded (`flowName`, `flow`, `flow_file`), because by then Zod has already stripped them and the caller has no other way to learn their key was dropped. **3. A flow DIRECTIVE passed as `command` points at the tool that records it.** `command: "echo"` used to come back as a bare "Tool not found". It now names `flow-add-echo` — and says to call it **directly**, since routing it through the recorder would write the echo *and* a `tool: flow-add-echo` step that fails on every replay. The hint distinguishes directives the recorder **rewrites** from those it stores **raw** for the polish pass, because promising a rewrite that does not happen sends the author looking for a directive that is not in the file. `wait` and `long-press` get their own answers: neither has a recording tool at all. Only a genuine not-found is rewritten — a tool that ran and failed still reports its own error. **4. A recorder tool as `command` is refused.** `flow-add-echo`, `flow-add-step`, `flow-start-recording`, `flow-finish-recording` each mutate the recording itself, and nesting one reported success either way. Each is refused for its own reason: `flow-add-echo` would write the echo **and** a replay-breaking `tool:` step, `flow-start-recording` would **erase** this flow at replay, `flow-finish-recording` would end the take it is a step in, and `flow-add-step` cannot record itself. Now nothing is executed and nothing is written. --------- Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com> |
||
|
filip131311
|
efae6a8fb6 |
feat(tools): opt-in emulator sound on boot-device + boot-sound default flag (#813)
Field report ([Slack thread](https://swmansion.slack.com)): when a session needs to test audio, it is very non-obvious that argent boots the Android emulator muted — the agent (and the user) burned significant time discovering it. The mute default is intentional (several agent-driven emulators all playing sound on the host would be worse), so this PR keeps it and instead makes it **discoverable and overridable**: ## 1. `sound` argument on `boot-device` (default `false`) - `sound: true` boots the emulator with audio output (drops the mute launch flag). - Applied consistently to the snapshot-loadability probe, hot boot, and cold boot — the qemu-device-topology parity invariant is preserved. A `default_boot` snapshot saved in the other audio mode fails the probe and falls through to a cold boot that re-saves it, so toggling costs one slower boot and nothing else. - iOS/Vega/Electron are unaffected (argent never mutes them); the argument documents itself as Android-only. ## 2. Agent-facing information: argument description only Per the thread, the info lives in the tool argument's description (no skill changes): it states the muted default, *why* (multiple agent-driven devices sharing the host's speakers), the `force: true` hint for an already-running muted emulator, the cold-boot cost of toggling, and the `boot-sound` flag. The underlying emulator flag is described behaviorally ("muted"), not by name, per our agent-docs convention. **Validated that the description alone is sufficient** by driving three models through the Cursor `agent` CLI with the schema excerpt and the exact scenario from the field report ("testing notification sound, hear nothing, emulator already booted with defaults"). All three — composer-2.5, gemini-3.7-flash, gpt-5.2 — correctly diagnosed the muted default and produced the exact fix `boot-device({avdName, sound: true, force: true})`, including the weakest model. Transcript excerpts in the PR discussion below if wanted. ## 3. `boot-sound` feature flag `argent enable boot-sound` defaults the `sound` **argument** to `true`. It only moves the argument's default, read live per call: an explicit `sound: false` on a call still boots muted, and without the flag the default remains muted. Registered in `FLAG_REGISTRY` so `argent enable`/`argent flags` pick it up with no further wiring. ## Tests - New `boot-device-sound.test.ts`: mute-by-default, `sound: true` dropping the mute flag from **both** probe and hot-boot spawn (parity), cold-boot spawn, flag-driven default, and explicit-`sound:false`-beats-flag. - The three existing Android boot test files now pin `isFlagEnabled` to `false` so a developer's real `flags.json` can't flip the args they assert on. - `tool-server` boot suites 62/62 green, `argent-cli` flags 33/33, `configuration-core` 122/122; lint, `typecheck:tests`, and knip all pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
filip131311
|
b232114422 |
docs: audit every comment in src against the code it describes (#931)
Audits every comment in `src/` and `scripts/` against the code it
describes.
**462 files, 57 commits, net −8,194 lines.** Comment text only — the
whole branch is code-identical to `main`.
## Method
One subagent per file, strictly sequential. Scope was `src/` +
`scripts/` (459 files); three more files were added at the end because
they carried dead references of the same kind — two test headers citing
design docs that do not exist, and `publish-npm.yml` citing a retired
workflow. Each agent verified every comment — line, block, JSDoc, file
header, trailing — against the surrounding code and the rest of the
repo, following identifiers, paths, tool ids, config keys, env vars and
issue links to see whether they still exist and still behave as
described. Rules:
- **A false or misleading comment is deleted, not reworded.** If a claim
could not be confirmed by reading the source, it went. That is why the
deletion count is so much larger than the rewrite count.
- Survivors are cut to the shortest form carrying something the code
does not already say. Restatement, preamble, hedging, changelog prose
and ASCII banners are gone; the non-obvious *why* stays.
- Preserved byte-identical: license headers, pragmas and directives
(`@ts-*`, `eslint-disable`, shebangs, `/// <reference>`), JSDoc tag
tokens, everything inside a string or template literal, and the sole
comment inside an otherwise empty block (ESLint `no-empty` counts a
comment-bearing block as non-empty).
## Verification
Every file passed two independent gates before being recorded as done:
1. `comments-only` — the required check.
2. A second comment-stripping comparator with a proper mode stack,
written for this pass because `comments-only`'s flat scanner desyncs on
nested template literals and quote-bearing regex literals and then
reports comment lines as code changes. Two files hit that false FAIL
(`utils/android-profiler/pipeline/index.ts`,
`scripts/extract-tools.mjs`); in both the "changed code" it printed was
literally `//` lines, and the second checker confirmed the code was
byte-identical.
After the last file, all 462 changed files were re-checked against
`main` with the same comparator, rather than trusting any agent's
self-report. **459 code-identical; 2 are non-code (`.svg`, `.md`); 1
intentional.**
The intentional one is `packages/argent/scripts/bundle-tools.cjs`: the
changed template literal *is* the comment header of the file it
generates, `packages/native-devtools-android/src/bundled-meta.ts`.
Fixing only the generated file would have been reverted by the next
build, so the generator changed too — and it has been verified to
reproduce the committed generated file byte-for-byte.
## Representative false claims removed
Not wording nits — statements a reader would have acted on:
- **Reversed directions.** `proxyStart`'s JSDoc had the tunnel backwards
(it is a reverse tunnel: the host binds first and the simulator dials
in). A `paste()` doc had the pasteboard copy direction reversed.
- **Contradicted by the code below it.** A timeout budget multiplied by
three where the probes run concurrently — the same comment said so six
lines later. A "warn once" that warns on every call. A "binary search"
that is a linear scan.
- **Named things that do not exist.** A `vega-fast-cli` binary, a
`finish-recording.ts`, a `publish-next.yml` workflow, two
`profiler-react19-*.md` design docs, a `DebuggerTarget.ts`, a commit
hash git does not know, two tool ids, an `ensureEnv` cycle.
- **Wrong by construction.** "Welford accumulators" across four files
where the code keeps naive `n`/`sum`/`sumSq`; `sum`/`sumSq` documented
over `actualDuration` when reduce sums `selfDuration`; a strict-mode
halving written `n/2` where the code ceils; field docs listing enum
values the producers never emit.
- **Guarantees the code does not make.** A validation matrix claiming to
cover "EVERY tool" that skips flagless ones; a Pareto cutoff that
`slice(0, 20)` makes inert; an idempotence claim where the real rule is
at-or-ahead; a capability note describing a clean 400 the shape-based
device resolver can never produce.
- **Unverifiable assertions** about prebuilt binaries, external CLIs and
the cloud SDK — deleted rather than kept as folklore, since nothing in
the repo can confirm them.
- **Stale numbers**: invented Android tool versions, hard-coded tool
counts and description lengths that had drifted.
## Review
A Fable agent reviewed both halves adversarially for over-deletion,
misread code, `no-empty` hazards and byte-identity violations.
Second-half verdict: **SHIP**, with two one-line restores, both applied
in the final commit — the `npm view ""` rationale behind a blank-token
guard, and the note that `argent-mcp` keeps a copy of
`SECRET_PLACEHOLDER_MARKER` it cannot import.
## Code issues surfaced but deliberately not fixed
This pass changes comments only. Eight genuine findings are logged for a
follow-up:
1. `telemetry/src/consent.ts` — a non-ENOENT read error returns null and
falls through to the default-on path, so file errors *can* silently flip
telemetry on.
2. `chromium-server/navigation.ts` — `navigate()` is reachable from
`POST /api/navigate` with only a `typeof === "string"` check; open-url's
schema is a bare `z.string()`, so the "already validated by zod" premise
never held.
3. `http.ts` — `constantTimeEqual` returns early on a length mismatch,
so the auth token's length is observable.
4. `describe/index.ts:~114` — the ios-remote branch passes `{ isTvOs:
false }` unconditionally, so a remote tvOS simulator takes the iOS
ax-service path, though `isRemoteTvOsSimulator` exists and shake/paste
do use it.
5. `devices/boot-device.ts` — `-crash-report-mode never` is passed
unconditionally *and* appended again by the feature-detecting path, so
every emulator spawn passes it twice.
6. `react-profiler/pipeline/04-rank.ts` — `PARETO_THRESHOLD_PCT` is
dead: `slice(0, 20)` always wins.
7. `reaped-sessions.ts` — a user-facing hint string tells the agent that
`react-profiler-start { force: true }` disposes the debugger and
profiler session; it does not. Left byte-identical because it is a
string literal, not a comment.
8. `utils/simctl-backend.ts` — `localSimctl` is exported with no
importers anywhere.
## Docs
No documentation change is needed: this pass touches only source
comments, and no user-facing capability, tool, CLI flag, config key or
flow-file behaviour changed.
---------
Co-authored-by: filip131311 <f.kaminski2000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
filip131311
|
5d76ed4d66 |
feat: tag artifacts with a semantic kind at registration (#564)
Fresh, from-current-main take on the artifact-kind idea from #398 / #466. Knowing what an artifact *represents* (not just its MIME type) is what a UI needs to group and name artifacts (e.g. in EAS); this PR delivers exactly that — the tagged handle — without the declared-output-map machinery. ## Summary - Adds a closed `ArtifactKind` union to the artifact wire contract; `kind` is carried on `ArtifactHandle`, `ArtifactEntry`, and the `/artifacts` inventory items. - `ArtifactStore.register` now takes a single `{ hostPath, kind, ... }` options object, so the compiler forces every producer — current and future — to declare what it is emitting. - Tags **all** producers on main today, including the ones #398/#466 predate: screenshot (all four capture paths), screenshot-diff, **flow snapshot baselines/diffs** (`screenshot` / `screenshot-diff-context`), **screen recordings** (new `screen-recording` kind), and the native/React profiler exports and reports. - Types the native-profiler export keys at their source (`IosExportKey` from the `EXPORTS` table, `AndroidExportKey`) and maps them through a kind table that is **total over both unions** — a newly added export key fails to compile until classified, replacing the previous stringly-keyed loop (and the `?? "native-profile-trace"` silent-default this design would otherwise need). - Client-side (`@argent/tools-client`), the mirrored handle type keeps `kind` **optional and widened** (`ArtifactKind | (string & {})`): an older, already-running tool-server sends no kind, and a newer server may send kinds this client build predates. Consumers treat it as advisory metadata; unknown kinds are opaque, never an error. ## Differences from #398 / #466 (deliberate) - No `@argent/artifacts` package, no third `ToolDefinition` generic, no per-tool output-name registrar, no runtime throw for “undeclared output” — the kind is plain data at the registration call site, checked at compile time where it can be, and a wrong label can never fail a tool at runtime. If a pre-invocation `artifacts` declaration in tool-listing metadata is ever needed (an actual EAS consumer), it can be layered on later without changing this wire contract. - The two `ArtifactKind` definitions (registry / tools-client) stay deliberately separate — same as the existing `ARTIFACT_MARKER` must-match convention — because the client-side type is intentionally *looser* (optional + open) than the server-side one for version skew. - Extending the union stays backward-compatible on the wire. ## 2026-08-24 update Merged current main (0.22.0) into the branch. The flow-visual rewrite that landed since (cropOn crops, per-app snapshot-key dedupe, diff scratch-dir hygiene) is preserved verbatim; its five registration sites now carry kinds (`screenshot` for baseline/current, `screenshot-diff-context` for the annotated diff — the cropped `current` included). No producer added since the PR was opened needed a new kind: `paste`, `shake`, and the other new tools emit no file artifacts, and every register call site in the tree now goes through the typed options object. No docs update is needed: the docs site (added since this PR) does not document the artifact handle wire contract; `kind` is advisory wire metadata with no user-facing surface. ## Validation At the merge with main (0.22.0), 2026-08-24: - `npm run build` (full `tsc --build`) — clean - `npm run typecheck:tests` in registry, tools-client, tool-server, mcp, cli — clean - Full suites green: registry 90, tools-client 191, tool-server 4495 (+1 skipped), mcp 80, cli 472 - `prettier --check .`, `eslint . --max-warnings 0`, and the knip gate — all pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Hubert Gancarczyk
|
f21811d069 |
fix(flow): re-probe a recorded wait against the tree the runner reads (#717)
> Stacked on #715. ## The problem `await-ui-element` evaluates against the tree it reads — the **accessibility hierarchy** on iOS and Android, the **CDP DOM** on Chromium, the **toolkit page source** on Vega. The `await:` / `assert:` directive that the polish pass converts a recorded wait into is evaluated against the **runner's** tree instead. They overlap, but outside Vega neither contains the other — an id present in one can be absent from the other, and on iOS even the role vocabularies are disjoint. Both directions have been hit in practice: an id visible to the runner but not the recorder, and text-field values visible to the recorder but not the runner. The consequence is that a check can pass live and fail once converted. That makes the core promise — *each step is executed live so you verify it works before it's recorded* — untrue at exactly the point it is relied on. ## What changes After a recorded `await-ui-element` succeeds, the recorder re-runs the same condition against the tree the runner resolves directives against, and reports the result on the step it just wrote. **It warns; it does not refuse.** What gets written is a raw `tool: await-ui-element` step, and at replay that tool reads the same tree it just passed against — so "this would fail every run" was never true of the form actually recorded. What the probe genuinely reports is whether the **polish-time conversion** is safe, which is why the warning says to act on it *then* rather than re-record now (re-recording would duplicate the step). Because polish begins after `flow-finish-recording`, each verdict is also carried there: it is repeated on its own step's `summary` line, and `message` counts how many steps carry one. A warning raised at step 7 of a 40-step recording is otherwise long out of reach by the time it is actionable. ## Three outcomes, kept distinct | Runner tree says | Reported as | | --- | --- | | condition holds | nothing — step recorded clean | | condition does not hold | recorded + warning naming the causes to rule out before converting | | tree unreadable, or too slow to answer | recorded + warning that the conversion is **UNKNOWN, not known-bad** | The middle row is deliberately not "the conversion will fail". The identical verdict comes back when the screen simply moved on between the live wait and the re-probe, and on `text` checks when the selector matches several elements and the two sides elect different ones — so the consequence stays conditional on the cause, and the message names the causes in the order worth ruling out. The third row matters: nothing was compared, so calling it a divergence would send the author off to rewrite a selector that may be perfectly good. It is also the injection-free case the flow skills explicitly sanction, so refusing there would block a supported way of working. On iOS the quoted reason ends "provide bundleId explicitly" — advice aimed at the native tools. The warning corrects it, because no directive carries a bundleId and forwarding one would make the probe answer a question the runner never asks. ## Platform-specific explanation The *why* differs per platform, and stating the iOS story everywhere makes the message wrong where someone is trying to act on it. iOS gets AX-vs-native-hierarchy; Chromium gets "the flow tree keeps only addressable nodes"; Android gets the importance trim; Vega gets "the trees agree, so the screen changed". The named tool for reading the runner's side follows the same rule — pointing an Android author at `native-find-views` points at a tool they cannot call. The remedy is per **condition** as well as per platform: on `hidden` a determinate verdict means the runner's tree still *has* the element, so "retarget at an id the full hierarchy carries" is the opposite of what helps. ## A failed live wait is reported by cause `await-ui-element` returns `{ success: false }` for three different things: a condition read and found false, a tree source that never answered, and a cancelled wait. Only the first judges the condition, so only the first is narrated as "the wait never held" and prescribes re-recording or deleting the step. The other two observed nothing and are reported as UNKNOWN — the same unknown-vs-known-bad distinction the probe draws for the runner's tree. A wait nested inside a recorded `run-sequence` gets neither warning: that tool reports its own shape, which carries no `success` key. The skill says so, and says to read `toolResult` there. --------- Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com> |
||
|
filip131311
|
172e4d3a64 |
fix(android): capture the screen the way it is actually oriented (#693)
Part of #609. ## What was wrong On a rotated Android device `screenshot` returned a portrait-framed image with the content lying sideways. `describe` frames and gesture coordinates were **already upright** — uiautomator measures against the rotated display — so the capture was the one surface out of step. An agent reading that image saw geometry that disagreed with every coordinate it was given. Measured on a landscape Pixel_9 (API 36, `mCurrentOrientation=1`): | | space | |---|---| | `describe` | upright — `"Search Settings"` centre `(0.530, 0.234)` | | taps | upright — tapping describe's `"Display & touch"` centre opens it | | `screenshot` (default) | **portrait-native, sideways** | ## The fix simulator-server already applies `rotation` correctly; nothing was telling it which one. The rotation is now queried from the device per capture, so `screenshot`, `screenshot-diff`'s live captures and the auto-attached "Screen after action" screenshot all land in `describe`/tap space. **Nothing is tracked.** A cached orientation could only be staler than the ~40 ms probe, and serving a stale one is exactly the failure being fixed. When the rotation cannot be read the request is left byte-identical to before rather than guessing — a wrong orientation is a confidently wrong image, which is worse than a sideways one. **iOS is deliberately untouched.** There the whole surface is consistently portrait-native, so rotating only the capture would *break* an agreement rather than restore one. The iOS half of #609 needs a different change and is not in this PR. ### Also fixed: the legacy uiautomator describe path Same root cause, worse symptom. `wm size` reports the **unrotated** size — on the landscape Pixel_9 it still answered `Physical size: 1080x2424`. Dividing rotated bounds by that does not merely squash frames: `isVisibleRect` prunes nodes whose left edge is past the screen width, so **the right-hand half of a landscape screen vanished from the tree**. The dump carries `<hierarchy rotation="N">`, so the divisor is oriented from bytes already in hand — no extra round-trip on the path that runs precisely when the helper could not be reached. ## On the rotation-name table Pinned by measurement, never by what the names mean — the reporter of #609 found the same names are 180° inverted on iOS. `adb exec-out screencap` is rotation-aware, so it is ground truth; each candidate was scored as mean absolute difference against it both as-is and rotated 180°, so an inverted mapping could not pass: ``` rotation 1 LandscapeLeft 2.19 (vs 6.54 for its 180° twin) rotation 3 LandscapeRight 2.19 (vs 6.34) rotation 2 PortraitUpsideDown 2.67 (vs 13.13) rotation 0 Portrait identity ``` This table is the **inverse** of simulator-server's own `90° → LandscapeRight` convention, because it compensates for a rotation that layer applies while decoding the scrcpy stream. That makes it a property of a particular simulator-server build rather than of Android — which is why a table test alone is not enough, and the delivered PNG's aspect is checked against the rotation requested, falling back to an unrotated capture if they disagree. ## Verified live Branch tool-server against a real emulator: - landscape, default capture → **2424×1080 upright**, MAD 2.19 vs ground truth (6.54 vs its 180° twin) - cropping `describe`'s frame for the toolbar out of that default capture shows exactly the "Display & touch" toolbar — the two spaces agree - rotation 0 → 1080×2424, MAD 2.82, unchanged - rotated **iPad** → still 2064×2752 sideways, i.e. iOS provably untouched Plus 27 new unit tests, including one that asserts the legacy path *drops* the right-hand node before the fix and keeps it after. Full suite 3125 passing; lint, prettier, `typecheck:tests` and the tool-description gate all clean. ## Not in scope - The iOS half of #609. - `rotate` on Android silently no-ops when auto-rotate is enabled (it returned `{orientation:"LandscapeLeft"}` while `dumpsys` still read `cur=1080x2424`). Filing separately. - `screen-recording` still records sideways — same root cause, different pipeline. Worth noting a session's screenshots are now upright while its video is not. - Existing visual baselines captured on a rotated Android device will stop matching, since the baseline key embeds capture dimensions and those now swap. This fails loudly as a missing baseline rather than silently comparing the wrong thing — and those baselines were sideways captures of an upright UI. --- > **#673 and #658 are both stacked on this branch** — each conflicts with this one (`screenshot/index.ts` and `screenshot-diff/index.ts` respectively) but not with each other, so both were rebased onto this branch as siblings rather than chained. Merge this first; GitHub then retargets both to `main`. --------- Co-authored-by: Filip131311 <f.kaminski2000@gmail.com> |
||
|
filip131311
|
d0b3356ebf |
feat(tool-server): paste tool (device clipboard + paste shortcut) (#879)
## Summary Registers a real `paste` tool. The `tools/paste/index.ts` from #506 was never registered and sent a `{cmd:"paste"}` WebSocket command simulator-server doesn't implement — dead on every platform. - **iOS simulator**: simulator-server `POST /api/clipboard/text` fills a private pasteboard and pushes it into the sim synchronously (host clipboard untouched); the tool then sends the ⌘V chord over the existing `key` command (the same `simulatorSendKeyEvent` path the keyboard tool uses). - **Android emulator**: same route over the emulator gRPC bridge, then `KEYCODE_PASTE` via `adb shell input`. - **ios-remote**: the MoQ transport's existing paste primitive, awaited. - Clean 400 for tvOS / Android TV (runtime probe), physical devices, Chromium (a renderer paste would clobber the *host* clipboard), Vega. - `{{secret:<NAME>}}` placeholders as in `keyboard`; allowed inside `run-sequence`; device-interact SKILL updated. - Description/skill tell the agent `paste` is **not** a `keyboard` substitute — only where a real user would paste (OTP/2FA code, long link/token, testing paste handling). ## Blocked on a simulator-server built with the `clipboard` feature: software-mansion/radon#185 adds it to the `argent` alias (no sim-server API change). Until a release + bin bump, the tool reports `PASTE_CLIPBOARD_UNSUPPORTED` (with an update hint) against the shipped binary. ## Test plan - 13 new unit tests (`test/paste.test.ts`) + secret-placeholder tests; full tool-server suite, lint, knip green (the 4 telemetry/bind test files that fail locally fail identically on `main`). - Live against a locally built simulator-server: iOS 26.5 Settings search (12 back-to-back pastes in order), Messages composer (multi-line/tab/emoji/diacritics preserved), `{{secret:EMAIL}}` never echoed or logged; Pixel_9 Settings search (ASCII + `Zażółć🔑`), Expensify RN login field; `run-sequence` paste steps; live 400s for tvOS sim / phone serial / chromium id. |
||
|
Ignacy Łątka
|
08f91fffd1 |
perf(screenshot): lower the default capture scale to 0.25 (#878)
Screenshots are billed to an agent's context by **pixel area**, not by file size — so PNG level, JPEG quality and codec choice buy nothing here, and the default scale is the only real lever. `0.3` had never been measured against what a model can actually read, so I measured it. ## What was run Bluesky (`bsky.app`) built from source and driven on an iOS 18.6 simulator, signed in against the repo's own seeded dev PDS (`yarn e2e:mock-server` + `e2eSignInAlice`) — a live feed, profile, notifications, Explore and Settings, not marketing screens. Every image came out of argent's own `screenshot` tool over the tool-server HTTP API, so the downscaler under test is the shipped one. **160 API calls**: 5 screens × 5 scales × 2 models × 3 reps, plus a blind control per screen per model. Questions were the things an agent actually needs off a screen — an `@handle` two rows down, a relative timestamp, a follower count, a badge number, which tab carries the underline, the label of the row above another row. Two controls decide whether the numbers mean anything: - **Blind pass.** Each model answered every question with *no image at all*. The 6 of 30 questions it still got right from priors alone (the screen title, the tab selected by default) were struck from scoring. 24 survived. - **Counting split out.** "How many rows between X and Y" was unreliable at *every* scale including full resolution — it measures arithmetic, not legibility — so it is excluded from the headline and reported separately. ## Result Text-reading accuracy, 66 graded answers per cell. Token counts are the API's own reported `input_tokens`, image column net of the ~345-token prompt. | scale | image tokens | Haiku 4.5 | Opus 5 | |---|---|---|---| | 1.0 | 2692 | 100.0% (66/66) | 100.0% (66/66) | | 0.3 *(old default)* | 451 | 98.5% (65/66) | 100.0% (66/66) | | **0.25** *(new default)* | **298** | **100.0% (66/66)** | **100.0% (66/66)** | | 0.2 | 196 | 98.5% (65/66) | 100.0% (66/66) | | 0.15 | 114 | 81.8% (54/66) | 97.0% (64/66) | 0.25 is the only scale that scored perfectly for both models on every repetition, and it costs **34% less context** than 0.3. ## The cliff, and why the margin Every wrong answer at 0.15: | question | truth | answered | model | | |---|---|---|---|---| | 2nd author's handle | `@bob.test` | `@carla` | Haiku ×3 | read the row above | | selected feed tab | `Following` | `Feeds` | Haiku ×3 | couldn't resolve the underline | | 4th profile tab | `Videos` | `Likes` | Haiku ×3 | drifted two positions | | 3rd settings row | `Moderation and content filters` | `Privacy and security` | Haiku ×3 | off by one row | | bell badge number | `1` | `UNREADABLE` | Opus ×2 | declined rather than guess | **Opus fails safe; Haiku fails silent.** Opus refused the unreadable badge. Haiku returned a confidently wrong *neighbouring* row — the same wrong answer three times out of three. For an agent that turns a read into a tap, a confident off-by-one row is worse than a refusal, because nothing downstream catches it. ## Why 0.25 and not 0.2 0.2 measured no worse than the old default and would have saved 57% instead of 34%. I didn't take it: the evidence is five screens of one app at one device size, and Bluesky sets fairly generous type. A denser interface — a table, a settings screen with subtitles, a smaller phone — reaches the floor sooner. 0.25 keeps two steps of margin above the cliff instead of one. `ARGENT_SCREENSHOT_SCALE=0.2` stays available for anyone who wants to spend that margin deliberately. ## Scope - **Chromium is untouched.** Its default is 1.0 and its `scale` is silently dropped without the optional `sharp` dep — that's #673's territory and I'm not duplicating it. Worth noting the two interact: once Chromium can downscale (via the in-repo `resizeDecodedPng`, as #673 suggests as a follow-up), it should pick up this default too, and it is currently by far the most expensive platform per capture. - **`ARGENT_SCREENSHOT_SCALE` is documented as `0.3` in `argent-private/docs/environment-variables.md`** — a submodule, so it needs its own PR to stay in sync. - Stale prose fixed in passing: `getScreenshotScale` carried a `// default: halve the resolution` comment next to a `0.3`, and `vega-screen.ts` documented the fallback as `0.3`. ## Checks `npm run build` clean, `tsc --noEmit -p packages/tool-server/tsconfig.test.json` clean, eslint `--max-warnings 0` clean, prettier clean. Shard 1/4 of the tool-server suite: 88 files / 1261 tests pass; the four files touching screenshot scale pass individually (22 tests). The default had **no test coverage at all** — the constant could be edited to any value and the suite stayed green. This adds one. Mutation-checked: reverting the constant to `0.3` fails 6 of the 8 new cases. ### On the red Unit tests check Inherited, not introduced. This branch's Unit tests job fails on exactly one test — `describe-tool.test.ts > states the limitation without ordering a reboot when the degraded read returned a tree` — which is the same single test failing on `main` itself (main has been red since #583, this branch's base). Already tracked as #871 and fixed by #877. `flow-composition.test.ts` passes in CI on both. Everything else is green: ESLint, Prettier, Knip, Static checks, SpiderShield, lockfile sync, Linux + Chromium E2E, macOS + Chromium E2E, macOS + iOS E2E. |
||
|
Ignacy Łątka
|
d43938e4fe |
fix(describe): drop the redundant restart assertion that reddened main (#877)
Fixes #871. Supersedes #872, which fixed this the wrong way round. ## The assertion was demanding a restatement `describe-tool.test.ts:793` asserted: ```ts // It still names the remedy and what the remedy costs. expect(result.hint).toMatch(/boot-device/); expect(result.hint).toMatch(/restart/i); ``` against `DEGRADED_STANDING_HINT`, which reads: > …If something you expect is missing, **boot-device with force=true reboots the simulator** with the full accessibility settings "reboots the simulator" is already there. So `/restart/i` could only ever be satisfied by a *second* clause restating the reboot — and that is precisely what #583 carried: ``` … with the full accessibility settings — at the cost of restarting it and everything running on it. ``` A review suggestion applied through the GitHub web editor trimmed that clause (`dd99a2625`, `07f39cea8`) at 07:41:49Z and 07:41:57Z. The squash-merge fired at **07:42:23Z** — 34 seconds later, so no CI ran on that head. The trim was right: it removed eight words that said the same thing twice. The test was what was wrong. #872 restored the redundant clause to turn CI green. This deletes the assertion instead, and leaves the shipped hint byte-for-byte as it is on main today. ## No coverage leaves with it The four remaining assertions are the ones that carry the test's purpose — #583 existed to stop the caveat *ordering* a reboot that kills the developer's Metro session: ```ts expect(result.hint).not.toMatch(/MUST/); expect(result.hint).not.toMatch(/\bnow\b/i); expect(result.hint).not.toMatch(/before continuing/i); expect(result.hint).toMatch(/boot-device/); ``` Mutation-checked. Restoring the pre-#583 wording — `"You MUST call boot-device with force=true now to restart the simulator … before continuing."` — fails the test on the first assertion: ``` × states the limitation without ordering a reboot when the degraded read returned a tree AssertionError: expected 'This simulator was not booted through…' not to match /MUST/ Test Files 1 failed (1) | Tests 1 failed | 36 passed (37) ``` So the regression the test guards is still guarded. The deleted line was pinning a redundancy, not a behaviour. ## Verification Branched off `dd5d82383` (current main). Reproduced the failure there first — `1 failed | 36 passed` on the untouched tree. | gate | result | |---|---| | `vitest run` (tool-server) | **351 files, 4313 passed, 1 skipped**, exit 0 | | `tsc --build` | clean | | `typecheck:tests --workspaces` | clean | | `eslint . --max-warnings 0` | clean | | `prettier --check` | clean | Diff is one test file, one assertion and half a comment: ```diff - // It still names the remedy and what the remedy costs. + // It still names the remedy. expect(result.hint).toMatch(/boot-device/); - expect(result.hint).toMatch(/restart/i); ``` The comment loses "and what the remedy costs" because that half described the assertion being removed. |
||
|
Ignacy Łątka
|
da61d4f2eb |
test(tool-server): pin the ARGENT_* suite guard and correct what it documents (#859)
Post-merge review of #707. Three findings, all from that change's own surface; no behaviour outside the test suite moves. ## 1. Nothing pins the guard `test/setup/clear-argent-env.ts` strips the developer's `ARGENT_*` overrides so unit tests assert argent's shipped defaults. On a machine with nothing exported its loop body never runs, and CI is exactly such a machine, so no suite failure can report the guard being weakened. Measured at `9dc7c7cfc`, each mutation applied alone to the shipped file: | mutation | full tool-server suite | |---|---| | body replaced with `export {}` | 350 files / 4300 passed, **exit 0** | | `startsWith("ARGENT_")` -> `startsWith("ARGENT")` | 350 files / 4300 passed, **exit 0** | | `delete process.env[name]` -> `process.env[name] = ""` | `bind-failure-telemetry.test.ts` 3 passed, **exit 0** | | entry removed from `setupFiles` | suite green | All four survive every gate: `vitest`, `eslint`, `prettier`, `tsc`, `typecheck:tests`, `knip`. `test/clear-argent-env.test.ts` re-imports the module against sentinels planted at call time, so the loop runs on demand rather than only when the developer happens to have an override exported. Each mutation now kills exactly one test, and the mapping is distinct: ``` body removed -> removes an ARGENT_ key outright instead of blanking it prefix underscore -> keeps a name that shares the prefix without the underscore delete -> blank -> removes an ARGENT_ key outright instead of blanking it setupFiles entry -> is registered as a setup file, so it runs before any test module ``` `ARGENTINA_REGION` is the look-alike: it pins that the underscore is load-bearing. The `in` check rather than a truthiness check is deliberate -- `= ""` reads as cleared to `if (process.env.X)` while still reaching a spawned child as a set key. ## 2. The header named the wrong file, and argued from the smallest cases > `ARGENT_PORT` / `ARGENT_HOST` retarget the bind that the startup-telemetry tests inspect. `startup-telemetry.test.ts` exists in that directory, mocks `createHttpApp`, and asserts nothing about host or port. `bind-failure-telemetry.test.ts` is the file that inspects the bind, and it is the one #707's own table named. A reader following the comment opens the sibling that literally bears that name and finds no bind assertion. The example list also covered only the four variables worth 9 failures, while the widest one was absent. Guard removed, one variable, full suite: ``` ARGENT_AUTH_TOKEN=devtoken -> Test Files 10 failed | 340 passed (350) Tests 80 failed | 4220 passed | 1 skipped (4301) ``` It puts every HTTP route behind a bearer check (`src/http.ts:59`), so ten files fail on status code alone. That is the strongest argument for scrubbing the whole prefix instead of a hand-maintained list, and the paragraph defending exactly that choice did not have it. ## 3. A comment #707 silently falsified `malloc-stack-logging.test.ts:89-95` promised to "restore the ambient value afterwards", capturing `process.env.ARGENT_IOS_CAPTURE` at module scope. The guard clears that variable before the module is imported, so the capture is always `undefined` and the `else` arm of the `afterEach` restore is unreachable. Clearing in `afterEach` is what the file actually needs and all it now claims. `export {}` is appended to the setup file so `tsconfig.test.json` -- which includes `test/**/*` -- treats it as a module rather than a global script; without it `typecheck:tests` fails with TS2306. ## Verification Run at `d0cb84de5`, rebased on `9dc7c7cfc`. - Full `packages/tool-server`: **351 files, 4306 passed, 1 skipped, 0 failed** on a clean environment, and identical with `ARGENT_EMULATOR_GPU_MODE=host ARGENT_EMULATOR_NO_WINDOW=1 ARGENT_SIMULATOR_NO_WINDOW=1 ARGENT_PORT=9999 ARGENT_HOST=0.0.0.0 ARGENT_AUTH_TOKEN=devtoken ARGENT_IOS_CAPTURE=all-processes` exported. - Mutation matrix above re-run against the new test: 4/4 killed, intended test each time, control green. - `prettier --check .` 0, `eslint . --max-warnings 0` 0, `tsc --build` 0, `typecheck:tests` 0, `knip` 0 (206 issues against the 215 ceiling, unchanged -- the new file is not flagged). ## Checked and left alone - The four `it.each(PLATFORMS)` retitles from #707 render correctly (`on linux ... -gpu swiftshader`); all four sites were fixed and a sweep of all 77 `.each(` sites in the repo found no other transposition. - The guard is genuinely unnecessary in the other ten packages: the union of all 53 `ARGENT_*` names read anywhere in `packages/*/src`, exported in two opposing polarities, leaves every other suite green. `argent-tools-client` already has its own copy from #714. - No CI workflow exports an `ARGENT_*` variable into a vitest process, so nothing in CI is destroyed by the scrub. `wayland-e2e.yml:46` sets `ARGENT_EMULATOR_NO_WINDOW` at job level but runs no vitest. - The other vestigial ambient save/restore pairs (`window-shake`, `http-auth`, `http-upload`, `http-flow-path-boundary`, `update-argent-tool`, `boot-device`, `boot-device-hotboot`) carry no false prose and are harmless defence-in-depth, so they stay. |
||
|
Ignacy Łątka
|
13f43bd0cd |
test(chromium): pin the ports-file guard and the readiness failure code (#862)
Follow-up to #709, which merged while this was still under review. #709 landed the port fix; these are the two gaps a review pass over it turned up, both in the same two files. ## The ports-file guard was pinned by nothing #714 added `vi.mock("../src/utils/chromium-discovery", …)` to `boot-electron-spawn-error.test.ts` so a completed boot stops persisting its port into the developer's real `~/.argent/chromium-cdp-ports.json`. Nothing asserted it still worked. Measured, on the pre-change file: deleting the mock leaves the suite **green at 9/9** while `$HOME/.argent/chromium-cdp-ports.json` is created containing a live ephemeral port. Replacing it with a call-through variant — the shape `list-devices.test.ts:76-81` already uses on this same module — does the same: ```ts vi.mock("../src/utils/chromium-discovery", async () => { const actual = await vi.importActual<...>("../src/utils/chromium-discovery"); return { ...actual, trackChromiumPort: vi.fn(actual.trackChromiumPort), untrackChromiumPort: vi.fn(actual.untrackChromiumPort) }; }); ``` → `Tests 9 passed (9)`, and the ports file contains `[55770]`. That is how the file accumulated entries before #714, so the guard rotting back is not hypothetical. Each file now scopes `HOME` with #714's own `scopeTempHome()` helper and asserts, after a completed boot, that no ports file exists. Both mutations now fail. **All three files that carry this mock are fixed, not just the one.** `boot-electron-kill.test.ts` and `boot-electron-ready-race.test.ts` hold the byte-identical mock — under a comment naming each other — and each completes a real boot. Measured before this change: deleting the mock left `kill` green at **15/15** while eleven live ports, one per boot, were written; the call-through variant did the same to `ready-race` **while still satisfying its own `expect(trackChromiumPort).toHaveBeenCalledWith(port)` at line 175**, which is precisely why the absence assertion is the right shape. Asserting the absence rather than `expect(trackChromiumPort).toHaveBeenCalledWith(port)` is deliberate: the call is already pinned by `boot-electron-ready-race.test.ts:175`, and a spy assertion only proves the export *is* a spy — the call-through mutation satisfies it while still writing. ## The readiness timeout was pinned by its wording, not its code The catch-path test matched `/CDP never became reachable/`. `FAILURE_CODES.CHROMIUM_ELECTRON_CDP_TIMEOUT` had **no assertion anywhere in the repo** — `grep -rn CHROMIUM_ELECTRON_CDP_TIMEOUT packages/ --include=*.ts` outside `dist/` hits only the throw site and the enum. Swapping the throw site to `CHROMIUM_ELECTRON_SPAWN_FAILED` left `boot-electron-spawn-error` + `-kill` + `-ready-race` + `failure-classification` + `chromium-discovery` at **60/60 passed**. So the half agents and telemetry bucket on was free to change, while a harmless reword would have broken the test. Now pinned by code via `getFailureSignal`, matching how `boot-electron-ready-race.test.ts:137` pins its own failure path. It still discriminates the vacuity the message match was added for: a nonexistent `appPath` yields `CHROMIUM_ELECTRON_APP_PATH_INVALID` and fails. `failure_stage` is pinned alongside it, as `ready-race` already does for its own path — `electron_cdp_ready` likewise had no assertion anywhere. ## Two rationales corrected - The catch-path comment blamed "no HTTP server on the port" for the probe failing. `fetch` declines port 1 as a WHATWG bad port *before it dials* — verified with `net.connect` hooked: **0 sockets opened** for port 1, 2 sockets for ports 2, 1023 and 19222. No listener is involved either way, and the file's own constant JSDoc says so. - The `chromium-discovery.test.ts` constant pointed at "a `listen(0)` fake CDP server **in a sibling test file**", when the server that makes the point is `startFakeCdpServer` in this same file, above the constant. - It also credited "anything in the unprivileged range" with being bindable by `listen(0)`. `listen(0)` draws from the kernel's *ephemeral* range — `net.inet.ip.portrange` 49152-65535 on macOS, `ip_local_port_range` 32768-60999 on the Ubuntu runner. That narrower range is what put the old `43211` at risk; calling it "unprivileged" also revives the privilege reasoning the sibling constant explicitly disowns. ## `chromium-discovery.test.ts` gets a second layer too It is the one file here that drives the **real** `persistPorts`, and its only guard was a single `beforeAll` line setting `ARGENT_CHROMIUM_PORTS_FILE`. That line is pinned — deleting it fails 3 tests — but the write lands before the detection, and `persistPorts` merges rather than replaces, so on a real machine the test ports would be appended for `list-devices` to probe afterwards. It now scopes `HOME` as well, so a lapsed redirect falls back somewhere disposable. Measured with a pre-seeded `$HOME/.argent/chromium-cdp-ports.json` = `[9222]`: deleting the redirect still fails 3 tests, and the seeded file comes back **unchanged**. ## Verification | mutation | before | after | |---|---|---| | delete the module mock | 9/9 pass, ports file written | fails | | call-through `vi.fn(actual.trackChromiumPort)` | 9/9 pass, ports file written | fails | | swap `CHROMIUM_ELECTRON_CDP_TIMEOUT` at the throw site | 60/60 pass | fails | | nonexistent `appPath` in the catch-path test | — | fails (`APP_PATH_INVALID`) | | swap `failure_stage` at the throw site | — | fails | | delete / call-through mock in `boot-electron-kill` | 15/15 pass, 11 ports written | 11 failed \| 4 passed | | call-through mock in `boot-electron-ready-race` | 2/2 pass | 1 failed \| 1 passed | Full `packages/tool-server` in 6 shards: 4303 passed, 1 skipped, every shard exit 0, and no `.argent` directory created under a redirected `HOME`. `eslint` 0, `prettier --check` 0, `tsc --build` 0, `tsc -p tsconfig.test.json` 0, `knip` 0 — exit codes read directly. |
||
|
Ignacy Łątka
|
6d31aed752 |
fix(describe): stop ordering a simulator reboot on every externally-booted read (#583)
Fixes #521. ## The report holds, and is worse than described Reproduced against an unmodified `origin/main` tool-server driving a simulator booted the way a developer boots one (plain `xcrun simctl boot`, no argent involvement), iOS 18.6: ``` ===== describe call #1 ===== ===== #2 ===== ===== #3 ===== hint: "This simulator was not booted through argent — system dialogs and native modals may not appear. You MUST call boot-device with force=true now to restart the simulator and apply full accessibility settings before continuing." ``` Identical on every call, as reported. The part that makes it more than a wording nit is what came back *alongside* it — a complete, working tree: ``` ROOT AXGroup (0.000, 0.000, 1.000, 1.000) AXStaticText "19:51" (0.139, 0.023, 0.102, 0.023) AXGroup "Maps" value="Widget" (0.082, 0.096, 0.395, 0.206) ... ``` And the caveat's own claim does not hold either. Triggering a real system permission alert on that same externally-booted sim (`simctl privacy reset location com.apple.Maps` + launch): ``` AXGroup "Allow “Maps” to use your location?" (0.193, 0.367, 0.614, 0.296) AXButton "Allow Once" (0.193, 0.524, 0.614, 0.046) AXButton "Allow While Using App" (0.193, 0.570, 0.614, 0.046) AXButton "Don’t Allow" (0.193, 0.617, 0.614, 0.046) ``` The system dialog reads fine, buttons and all. So an agent obeying "You MUST ... now" restarts the simulator — killing the Metro session, dev client and staged app state — to fix nothing. ## Root cause `AXServiceApi.degraded` is `!entitlementBypassActive`, read from `IgnoreAXServerEntitlements`. That pref is only writable pre-boot, so `boot-device` is the only thing that ever sets it and *any* externally-booted sim reads as degraded for its whole life. The pref bypasses the AX-server entitlement check introduced in iOS 26.5 — on earlier runtimes it is irrelevant to whether the read works. The flag describes how the sim was booted, never whether describe succeeded, and the hint conflated the two. ## Change Resolve the caveat against what the accessibility read produced: - **Blind read** — the reboot has something to gain, so the actionable guidance stays, on every call, and now names what the reboot costs. - **Populated read** — the read is its own proof no reboot is needed. States the limitation instead of ordering anything, once per device per server lifetime. The per-device gate is applied at the `describe` tool boundary only. `await-ui-element`, the flow runner and `await-screen-idle` poll `describeIos` directly, treat a present `hint` on an empty tree as "this emptiness is untrustworthy" (`isBlindRead`), and fold it into one terminal timeout note — they keep every copy, so a `hidden` wait still cannot resolve off a blind read. The native-devtools fallback can fill a tree the accessibility read left empty, and that split has to key on the accessibility read rather than on the tree that comes back. Its elements are the injected app's own view hierarchy: no SpringBoard chrome, no system dialog, and no evidence the accessibility subsystem works. Found doing exactly that on a fresh iOS 18.6 sim. Describing within seconds of `simctl boot` leaves the accessibility read blind for the rest of that boot (3/3; it survives a tool-server restart, and only `boot-device force=true` recovers it). With the Maps location prompt on screen in that state, the fallback returned the app's own 20 elements with the dialog absent — and resolving against *that* tree picked the standing note, which the per-device gate then deleted, so from the second call the agent had a tree silently missing a modal blocking the screen and no caveat at all. The note it briefly carried says "everything else reads normally", which nothing there did. Keying on the accessibility read makes both wordings true of the state that produces them. I skipped the suppression env var / flag the issue floated as a "maybe" — once the caveat is a single non-directive line per device, there is little left to suppress, and it seemed better not to add config surface for it. Happy to add it if you'd rather have the switch. ## Verification E2E against the branch build, same externally-booted sim: | | elements | hint | |---|---|---| | describe #1 | 14 | standing caveat, no MUST/now | | describe #2-4 | 14 | *(none)* | | after `simctl shutdown` (blind) ×2 | 0 | blind caveat, both times | | after `boot-device force=true` ×2 | 28 | *(none)* — `IgnoreAXServerEntitlements` = 1 | Unit: 6 new tests in `describe-tool.test.ts`. Each was mutation-checked — restoring the MUST wording, removing the dedup, collapsing the blind/standing split, keying the gate globally instead of per device, and re-resolving the caveat against the native-devtools tree each fail exactly one test and nothing else. Full gate: `vitest run` 4715 passed / 1 skipped, `tsc --build`, `typecheck:tests`, `eslint --max-warnings 0`, `prettier --check`, `test:scripts` 59 passed. |
||
|
Ignacy Łątka
|
9dc7c7cfcb |
test(chromium): stop electron boot tests depending on host CDP ports (#709)
## Problem
`boot-electron-spawn-error.test.ts` drove `bootElectronApp` against
hardcoded CDP ports 19222-19226 and relied on nothing answering there.
## It is not load, and it is not the timeout
I chased the obvious hypothesis first — vitest's 5000 ms default
colliding with a ~3.5 s built-in wait — and it is wrong. Measured at
`origin/main` before any edit:
| condition | result |
|---|---|
| 15 isolated runs | 15/15 pass |
| 10 concurrent copies | 10/10 pass |
| 3 full-suite runs (load avg 16) | 3/3 pass |
| 10 runs under sustained full-suite load (load avg 19-20) | 10/10 pass
|
**38/38, a 0% spontaneous failure rate.** And the one test whose
`readyTimeoutMs` is 5000 against vitest's 5000 ms default completes in
15-54 ms at load ~20 — nowhere near the boundary.
## The real trigger, and it is deterministic
Anything on the host answering the hardcoded CDP ports the test used.
Squatting one with an HTTP server that serves `/json/version` fails the
file 100% of the time. Reproduced against `origin/main`:
```
× rejects with a clear, actionable message when spawn emits ENOENT
AssertionError: promise resolved "{ platform: 'chromium', …(5) }" instead of rejecting
+ "id": "chromium-cdp-19223",
Tests 1 failed | 8 passed (9)
```
**Mechanism.** When a real endpoint answers, `waitForCdpReady` wins
`bootElectronApp`'s `Promise.race` in ~1 ms, long before the test's 10
ms `setTimeout` fires its synthetic error. The boot then completes and
*resolves*, so the rejection the test asserts never arrives.
The source is correct throughout. The test bound a global shared
resource it did not own. `chromium-cdp-19222` is argent's canonical
device id (`src/utils/chromium-discovery.ts:9`) and
`e2e-device-smoke.yml:139,173` boots a real Electron on exactly that
port — a developer with an Electron app open, or a second agent on the
same machine, is enough.
## Fix
One `UNREACHABLE_CDP_PORT` constant pinned to port 1, which
`ensureCdpReachable` can never reach: it dials through Node's `fetch`,
which refuses port 1 as a WHATWG **"bad port"** before it opens a
socket. Measured:
| port | result |
|---|---|
| **1** | `fetch failed` → cause `bad port`, no socket opened |
| 2 | `connect ECONNREFUSED 127.0.0.1:2` |
| 22 | `bad port` |
| 1023 | `connect ECONNREFUSED 127.0.0.1:1023` |
Privilege is *not* the mechanism, and would not be sufficient: 2 and
1023 are equally privileged yet dial normally, and Windows has no
privileged range at all. The JSDoc says so, so the constant cannot be
"tidied" below 1024 and quietly reinstate the flake. Not a timeout
widening.
## Second defect, same file
This file was the only one of the three Electron boot tests that did not
isolate port persistence, so its success-path boot wrote a real
ephemeral port into the developer's actual
`~/.argent/chromium-cdp-ports.json` on **every run**. That file
currently holds 107 entries, and I watched a concurrent session on this
machine append one mid-review.
Both siblings — `boot-electron-kill.test.ts:27` and
`boot-electron-ready-race.test.ts:30` — already guard this with
`vi.mock("../src/utils/chromium-discovery", …)`. This file now uses the
same mock, and asserts `trackChromiumPort` was called with the booted
port, so deleting the guard fails the suite instead of silently resuming
the writes.
## Sibling fixed
`chromium-discovery.test.ts` asserted port `43211` was unreachable, but
that sits inside Linux's ephemeral range (`ip_local_port_range` → `32768
60999`), so a `listen(0)` fake CDP server in a parallel test file can
legitimately be handed it. It now shares one named `UNREACHABLE_PORT`,
replacing that literal and a second bare `1` that was already there.
**Left alone, verified safe:** `boot-electron-kill.test.ts`,
`flows/flow-chromium-boot.test.ts`, `list-devices.test.ts` — each owns
its port via `listen(0)` or fully mocks the launcher. The fixed ports
`43210`, `54321`, `54322` (all in `chromium-discovery.test.ts`) are only
ever tracked or untracked, never network-probed.
## Verification
- Deterministic repro above, reproduced at `origin/main` and re-run
after the change with all five of 19222-19226 squatted: **20/20 pass**.
- **Mutations, each of which passed 9/9 before the guard was added:**
deleting the module mock → `TypeError: [Function trackChromiumPort] is
not a spy`; pointing the catch-path test at a nonexistent `appPath` →
`expected … /CDP never became reachable/ but got 'path does not exist'`.
- Ports file: full suite under a redirected `HOME` writes no
`chromium-cdp-ports.json` at all; the real file's md5 is unchanged
across runs.
- `eslint` 0, `prettier --check` 0, `tsc --build` 0, `tsc -p
tsconfig.test.json` 0 — exit codes read directly.
- Full `packages/tool-server` in 6 shards: 4065 passed, 1 skipped, every
shard exit 0.
|
||
|
Ignacy Łątka
|
f79e53a6e3 |
test(tool-server): scope vega-cli-timeout sentinels to the running process (#711)
## Problem
`vega-cli-timeout.test.ts` fails intermittently, always the same two
tests, always the same way:
```
rejects on its own deadline when the CLI never returns
→ expected { stdout: '', stderr: '' } to be an instance of Error
reaps the ENTIRE worker tree on timeout
→ promise resolved "{ stdout: '', stderr: '' }" instead of rejecting
```
Both at **~60-130 ms against a 400 ms deadline** — `runVega` *resolved*
rather than timing out. It never fails in isolation.
## Not a tight budget, and not a reaping race
| condition | failures |
|---|---|
| isolated, 20 runs | 0/20 |
| under a parallel full-suite run, 15 runs | **2/15** |
| pinned to one CPU (`taskset -c 0`), 8 runs | 0/8 |
The single-CPU pin is the discriminator: if the 400 ms budget were too
tight, or the reap raced under contention, starving it to one core would
make it worse. It passes 8/8.
`packages/tool-server/src/utils/vega-cli.ts` is **not** implicated and
is unchanged here.
## Cause: the sentinel namespace is machine-global
The test's worker sentinels were fixed constants (`sleep 69101` … `sleep
69107`), and cleanup matches them with `pkill -f` / `pgrep -f` — which
are **global to the machine, not scoped to the process**.
Two concurrent runs of this file therefore share one namespace, and each
run's `afterEach` `pkill`s the *other* run's live workers. The fake
`vega` launcher's blocking `spawnSync("sleep", …)` returns early, the
launcher exits 0, and `runVega` resolves cleanly well inside its
deadline — exactly the observed signature.
The existing comment shows the pattern was already narrowed once
(*"Match only this suite's sentinels (6910[0-9]) rather than a loose
`sleep 691`"*). That tightened it against other suites but not against a
second copy of this suite, which is the case that actually occurs: CI
matrices, `vitest --no-file-parallelism` reruns, and any machine running
more than one checkout.
## Deterministic reproduction
A neighbour issuing this file's own sweep (`pkill -f '^sleep
6910[0-9]$'`) on a 20 Hz loop reproduces it **every time**:
```
before → Tests 2 failed | 7 passed (9) (3/3 rounds, both named tests)
after → Tests 12 passed (12) (3/3 rounds, same neighbour live)
```
## Fix
Derive the sentinels from a per-process tag (pid plus three random
digits) and scope every `pgrep`/`pkill` to it. Two live processes cannot
share a pid, so two concurrent runs never collide; the random digits cut
the leftover case — a worker leaked by an earlier run whose pid has
since been recycled — to 1/1000. A concurrent run computes a different
tag and its sweeps no longer reach these workers.
The tag rides in the *fractional* part of the sleep duration, which also
shortens a leaked worker's life from ~19 h to 10 min. That matters more
than it looks: previously a later run of this file would clean up an
abandoned worker as a side effect of the shared namespace, and scoping
the namespace deliberately removes that accident.
The same investigation turned up three more problems in the file, and
the rest of the diff is those:
- **Regressions reported a bare timeout.** Several tests here outrun
vitest's 5s default - the
delayed-close test's own deadline is 10s - so a real failure printed
`Test timed out in 5000ms` and
named nothing. The file now sets its own 20s budget; `beforeAll`
carries an explicit one because a hook resolves its budget when it is
registered.
- **Four assertions could pass with nothing to assert against.**
`waitForClear` reads 0 just as readily
for a worker that never spawned, so both reap tests observe their
workers alive first, and `strayCount`
returns -1 rather than 0 when it cannot look at all. The two output-cap
reaps have no such window (#841).
- **The reap test's premise was unasserted.** It is named for a worker
that escaped the launcher's process
group, but nothing checked that it had: with the observation below
removed, dropping `detached: true`
from the `hang` fake leaves every test green, and with that fixture the
descendant sweep could be
deleted from `reapVegaGroup` with everything still green. One polled
sample - a single `ps`, so the count and the groups cannot disagree
about the
instant they describe - now returns both, so dropping the flag fails
with `{ workers: 2, groups: 1 }` — naming which half broke — while
deleting the sweep still fails with `expected 1 to be +0`.
Smaller, each covered below: the near-deadline `timeoutMs` 600 -> 900ms,
now sharing one const with the
bound that pins it; the reap deadline 400ms -> 4s, to give the launcher
room to spawn both workers
before the reap ends the observation window; a `beforeAll` warm-up that
absorbs macOS's first-exec penalty; and three added tests (sweep
positive control, random-block width,
longer-pid decoy). Three comments about production behaviour were
corrected too - the header's account of
which mechanism reaps which worker, the reap test's on what the
descendant sweep covers by itself, and
the fake's on what reaps the launcher's own sleep.
## Verification
- Deterministic repro above, before and after, same neighbour process.
- **Teeth check** — a sweeper matching the post-fix *shape* but ignoring
the tag (`^sleep 600\.[0-9]+$`) still kills the fixed code 4/4. The
workers remain real and killable; only the tag protects them, so the fix
is not passing by making them unmatchable.
- **Vacuity check** — mutating the source (`collectDescendantPids` →
`[]`) makes `strayCount` report a live stray (`expected 1 to be +0`),
proving the new escaped `pgrep` pattern genuinely matches.
- **Mutation** — pinning `RUN_TAG` to a constant, with a neighbour using
the pattern a concurrent run of the *mutated* file computes: 4/4 fail.
It now takes down four tests, not the two originally named — the two
observations this PR adds, the near-deadline elapsed bound and the drain
count, are killed by the same neighbour.
- **Positive control** — every reap assertion here reads
`waitForClear(...) === 0`, which is also what
`strayCount` returns when the pattern matches nothing, so the suite
passed with a pattern that matched
nothing. Two of the three `sentinel bookkeeping` tests require a count
of 1 first, and the drain reap counts its
worker before the reap; the timeout reap counts through `ps` instead, so
it is the bookkeeping tests
that stand behind it here. Neutering `cmdlinePattern` fails those three
(`expected +0 to be 1`),
neutering `sweep()` fails one (`expected 1 to be +0`); the rest stay
green in both cases.
- **The workers under test are the ones observed** — giving only the
fake launcher's `hang` workers a
sentinel `strayCount` cannot see leaves the whole suite green without
the counts, and fails
`{ workers: +0, groups: +0 }` against `{ workers: 2, groups: 2 }` with
them. Same for the drain path. The two overflow tests keep no count, and
are
therefore **not** pinned: breaking only the `flood` branch's sentinel
leaves all 12 green. Their reap
fires within ~20-100ms of the spawn, so an observation taken after the
call never catches it (0/8), and
one taken concurrently loses the race about a third of the time under
load. Filed as #841, which
records a non-racy alternative.
- **Each observation is budgeted off the call it watches** — a poll
shorter than that call turns a slow
launcher start into a failed observation (`expected +0 to be 1` on the
drain path, `{ workers: +0,
groups: +0 }` on the reap) rather than into the reap it guards. The
drain
observation gets its call's whole 10s; the timeout reap's window is
ended by the reap itself, so there
the deadline *is* the budget, and it and the poll are both 4s. The reap
observation's worst was 1485ms at
50-way concurrency against that 4s (2.7x); the drain observation's own
worst was 1836ms against its 10s
(5.5x). Stalling the `linger-grouped` branch 3.5s fails the drain
observation on the 3s default poll and
passes on the call's 10s; stalling `hang` 2.5s fails the reap
observation on a 2s deadline, which kills
the launcher mid-stall, and passes on 4s.
The reap and drain observations are both new here, so the reap deadline
is what the added runtime buys: that test goes from
~450ms on main to 4.13s, about 3.7s of the file's 4.4s increase.
Measured against the branch's own
intermediate 2s deadline the step is +1.95s idle (7.18 -> 9.13s) and
+1.99s under load (10.52 -> 12.51s), and it takes
that observation from 1.35x to 2.7x. The file's thinnest margin is the
near-deadline row above, which no
deadline change can widen. The drain-side change is free, since the poll
returns as soon as the count
matches. An interleaved A/B soak, 40 runs per arm at load 90-240,
produced **no** failure of either observation in **either** arm, so it
is neutral rather than
confirming: the case rests on the injected stall and on the measured
margins, not on a reproduced
natural rate.
- **Slot width is pinned** — `SentinelSlot` makes a 2-digit slot a
compile error
(`TS2345: Argument of type '10' is not assignable to parameter of type
'SentinelSlot'`). A 2-digit slot
would both escape the sweep's `[0-9]` and break the equal-width argument
the isolation rests on.
- **The launcher is warmed in `beforeAll`** — the drain-grace test gives
the launcher 900ms to start,
and no larger value is available: it also has to stay under
`VEGA_EXIT_DRAIN_GRACE_MS` (1000) to force
the race at all. macOS charges the FIRST exec of a newly written file
and nothing
after it, and that cold exec does not fit — measured p90 1099ms / max
1684ms at 20-way concurrency
under 92 hog processes, which made that test fail **38 of 60 (63%)**
when run on its own. It is not
`node` starting: a fresh `#!/bin/sh` script shows the same jump, and
warming `node` elsewhere leaves the
solo rate at **55/60** - worse than not warming at all. Without the
warm-up a whole-file run never saw
it, because whichever tests run first pay the exec incidentally. Paying
that cost once in `beforeAll` makes it invocation-independent: startup
at that call site measured
136-148ms without the warm-up and 32-37ms with it.
- **Margins, worst observed against budget** — across ~950 runs at up to
80-way concurrency under 92-132 hog
processes (load avg 218-275):
| wait | budget | worst observed | margin |
|---|---|---|---|
| near-deadline launcher start | 900ms | 700ms (80-way) | **1.29x** |
| `elapsed < 5_000` (delayed close) | 5000ms | 3017ms | 1.66x |
| worst `waitForClear` | 3000ms | 1648ms | 1.82x |
| reap observation | 4000ms | 1485ms | 2.69x |
| `elapsed < 2_500` (deadline) | 2500ms | 742ms | 3.37x |
| slowest test vs `testTimeout` | 20_000ms | 4130ms (idle) | 4.84x |
| drain observation | 10_000ms | 1836ms | 5.45x |
The slowest-test row is an idle measurement, and a floor rather than a
worst: the reap test's launcher
never exits, so that test cannot settle before its own 4s deadline. The
thinnest margin is structural,
not tunable: that test's `timeoutMs` must clear the launcher's startup
*and*
stay under `VEGA_EXIT_DRAIN_GRACE_MS` (1000) for the race it forces to
exist at all. Both bounds sit in
that one sub-second band, so buying margin here spends it on the
`toBeGreaterThan` bound below; the
`beforeAll` warm-up is the only lever that does not. It still
discriminates —
dropping the exit path's `clearTimeout` fails it with `promise rejected
"FailureError: vega linger ..."
instead of resolving`. The `toBeGreaterThan(900)` bound on that same
test sits at 1.11x but is safe by
construction: the drain cannot fire before `grace` ms and scheduler
delay only pushes it later, so load
can never move it toward the bound.
- **Considered and not taken: sweeping the eight exact sentinels** —
replacing the `[0-9]` slot glob with an
ERE alternation of the literal sentinels looks like it would drop the
equal-width invariant and the two
tests that pin it. It would not. `pkill -f` still matches every process
on the machine, so isolation still
rests on equal width: pid `1234` / rand `567` / a two-digit slot `11`
spells `600.123456711`, byte-identical
to pid `12345` / rand `671` / slot `1`, so the second run's alternation
lists the first run's worker
verbatim and kills it. The unpadded random block collides the same way
(`1234`+`56`+`7` = `123`+`456`+`7`).
`SentinelSlot`, the padding test and the `$` anchor are load-bearing
under either form; only part of
`cmdlinePattern`'s docblock would go.
- **Deliberate idiom deviation, flag if you disagree** — `vi.setConfig`
and `testTimeout` appear nowhere
else in the repo; the convention here is a positional `it(..., 10_000)`
(`boot-device-hardening.test.ts:214`)
or `{ timeout: N }` (14 uses). Several tests in this file can exceed the
5s default, and two rounds of
per-test budgeting each missed some, so the budget is set once for the
file instead of enumerated per
test. The wall-clock assertions (`elapsed < 2_500`, `< 5_000`) are
unaffected — they assert speed
directly rather than relying on the test budget.
- **The wrapper self-match is real, but GNU-only** — `pgrep`'s own `sh
-c "pgrep -f '<pattern>' || true"`
wrapper carries the pattern in its argv. Verified in debian against
procps-ng 4.0.2: a bare sentinel with
no anchors returns **2** matches (the second being the wrapper), and
`^`, `\.` or `$` alone each cut it
to 1. On macOS it cannot be reproduced at all — BSD `pgrep` excludes its
own ancestors — so the docblock
says which platform the hazard belongs to.
- **The `$` anchor is pinned** — a decoy shaped like a longer-pid run's
sentinel must survive `sweep()`.
Dropping `$` kills it (`expected +0 to be 1`). `^` is deliberately left
unpinned: without it the pattern
reaches only command lines ending in this run's unique tag, which
nothing can produce.
- **The slow-failing regressions stay legible** — tests here outrun
vitest's 5s default on a runVega
deadline of up to 10s, a 3s clearance poll, or both. Past 5s vitest
reports `Test timed out in 5000ms` and
names nothing, so the file raises its own budget with `vi.setConfig({
testTimeout: 20_000 })`. Measured,
each with the matching production path disabled: the delayed-close test
`Test timed out in 5000ms` @5029ms
→ `promise rejected "FailureError: vega linger …" instead of resolving`
@10031ms; the flood test
`Test timed out in 5000ms` @5059ms → `expected 'timeout' to be
'subprocess'` @5030ms; the reap test
→ `expected 1 to be +0` @7254ms. No regression in this file reports a
bare vitest timeout any more.
- Two simultaneous runs, 4 rounds: 8/8 green; 15 sequential and 15
concurrent-triple runs green, no strays.
- `eslint` 0, `prettier --check` 0, `tsc -p tsconfig.test.json --noEmit`
0 — exit codes read directly.
## Sibling audit
No other test identifies its workers by matching a pattern against the
machine-wide process table — the only other `pkill`/`pgrep` invocation
under `packages/` or `scripts/` is a research CLI
(`argent-private/research/physical-ios/cli/argent-ios.ts:441`), which is
not a test and owns no sentinel workers. Every candidate checked:
`adb-terminal-error-format` (mocks `child_process`), `await-ui-element`
/ `await-screen-idle` / `mjpeg-stream` (in-process timers, no
subprocess), `artifacts` (real `tar`, no deadline assertion),
`list-devices` (`ps` mocked), `vega-process` (real `ps`, read-only, owns
no workers), `launcher-helpers` (TCP, no subprocess).
That is the narrow claim. The broader one — "the only test that spawns
real workers into a global namespace under a deadline" — does **not**
hold: `launcher-spawn`, `launcher-duplicate-spawn`,
`launcher-version-gate` and `launcher-sweep` all spawn real long-lived
servers (the first three through `spawnToolsServer`, so detached;
`launcher-sweep` a plain `spawn`), and `launcher-spawn` asserts on
elapsed time. They are safe for a reason worth stating, because it is
the same principle as this fix: each redirects `HOME` to a `mkdtemp`
dir, so their state directory is per-run.
Several unrelated tests do share the defect and are filed separately,
since none is in this diff: #835, #836, #837.
The mutation sweep run over this suite also surfaced `runVega`
behaviours its assertions do not pin: the signal-death `subprocess`
classification (#845), `settle()` clearing the exit timer (#847), the
output cap's UTF-8-byte semantics (#849), and both `destroy()` pairs
(#850) each survive as a mutant through a fully green run, as does
`VEGA_KILL_SIGNAL` softened to `SIGTERM` (#848). The process-group
SIGKILL (#846) is caught, but incidentally and flakily - by the
stderr-cap test, in 1 of 3 consecutive runs. All of these predate this
PR: the nine tests that exercise `runVega` are the same nine on `main`,
the three added here covering the sentinel helpers themselves.
|