skills/software-mansion__argent
1
0
Fork 0
mirror of https://github.com/software-mansion/argent.git synced 2026-09-14 19:27:14 +08:00
Code Issues Packages Projects Releases Wiki Activity
849 Commits 225 Branches 38 Tags
feat/android-atomic-clear
Commit Graph

5 Commits

Author SHA1 Message Date
Jakub Piasecki 6d907cc86b feat(flow): boot a Chromium instance per launch step (#585) 
A `launch:` on Chromium now starts its own Electron instance, so nested
e2e flows and mid-flow relaunches work as they do on iOS/Android.
Previously the runner booted one app per run and rejected any second
chromium launch.

A chromium device id _is_ its CDP port, so a launch that starts an app
from scratch has to boot one and move the run onto it: `state.device`
became reassignable (steps already read it per call), and `state.owned`
tracks what the runner booted, torn down in reverse at run end. An
instance the runner did not boot — pinned via `--device`, or
auto-detected — is attached to, never killed. Relaunching the same app
kills its instance and waits for the process to actually exit, or the
replacement loses a race against the app's single-instance lock.

The runner needs a device before step 1, so `resolveRunDevice` follows a
leading `run:` chain to find the launch the run _begins_ with: a
fragment composing a chromium e2e flow now boots instead of failing with
"No booted chromium device found". A launch declaring no app id for the
run's platform stays an error — the run's platform is fixed start to
finish.

Also fixes a pre-existing teardown leak: the spawned handle is the npm
`electron` node wrapper, not Electron, so signalling it left the app and
its helpers running (reparented to launchd, still in the dock). Teardown
now sweeps the child's process group.

**Behavior change:** a fragment leading with `run:` into a chromium e2e
flow now boots a fresh instance where it previously attached to a
running one; pass `--device` to keep attaching.

## Tests

Cover the boot/attach/settle three-way, teardown ordering and the group
sweep, device rebinding after a nested launch, same-app retire ordering,
cancellation mid-boot, both "no chromium app declared" guards, and the
leading-`run:` chain (multi-hop, cyclic, no-launch). The guard,
exit-wait and sweep tests are mutation-checked.
 2026-08-06 07:28:27 +02:00
filip131311 11f384d37f fix(chromium): make Electron apps testable without window focus (#605) 
## Problem

Field report: *"electron needs focus to be tested."* Analysis confirmed
three focus/visibility dependencies in the Chromium/Electron surface:

1. **Hidden window → throttled renderer.** A minimized/occluded window's
compositor throttles: mouse-input acks stall **~5 s per event** (they
wait on hit-testing), wheel scrolls hang (`gesture-scroll` already
refused), `visibilityState` flips to `hidden`.
2. **Unfocused window → focus-gated app logic breaks.**
`document.hasFocus()` is `false` before any input arrives;
focus/blur-dependent UI misbehaves.
3. Focus-gated web APIs (clipboard already worked around it locally with
`Page.bringToFront`).

## Fixes

**1. CDP focus emulation on every page session** (`primePageSession`, né
`enableCoreDomains`): `Emulation.setFocusEmulationEnabled(true)`,
best-effort, re-applied automatically after tab switches (Playwright
precedent). Live A/B on the Electron testbed: identical
boot→unfocus→probe sequence gives `hasFocus:false` on main, `true` on
this branch. Bonus verified by A/B (kill server → same minimized window
flips to `hidden`): in current Chromium/Electron, emulation also pins
reported visibility to `visible` and keeps input unthrottled while
genuinely minimized — minimized tap latency **~5 s → ~65 ms**, wheel
scroll works.
*Trade-off (same as Playwright ships):* while attached, the app can
observe neither real blur nor real hidden, so pause-on-hidden behavior
is unobservable during a test.

**2. Anti-throttling switches for every argent-booted Electron app**
(`--disable-background-timer-throttling
--disable-backgrounding-occluded-windows
--disable-renderer-backgrounding`). Sessions are created lazily and die
with the idle-exiting tool-server while the app outlives it — the flags
cover that detached lifecycle plus emulation-less runtimes. Also covers
flow-booted apps via the shared `bootElectronApp`.

**3. Hidden-window fail-fast extended from gesture-scroll to gesture-tap
+ gesture-drag** via shared `assertChromiumWindowVisible`, under a new
dedicated `CHROMIUM_WINDOW_HIDDEN` failure code (scroll migrated;
`failure_stage` strings unchanged so telemetry stays joinable —
analytics keyed on `CHROMIUM_INPUT_INVALID` +
`chromium_scroll_window_hidden` should switch to the new code). With
emulation active the guard is a backstop that fires only where emulation
couldn't be applied — exactly where the stall is real. Keyboard/button
stay unguarded on purpose: measured live, key events skip hit-testing
and stay fast on hidden windows.

## Verification

- Unit: 3075 tool-server + 87 registry tests green, `typecheck:tests`
green, new tests for emulation priming, boot args, and the guard (all
three tools).
- Live matrix on the Electron testbed (dev tool-server): focused
baseline; unfocused (`hasFocus` true pre-input, typing lands); minimized
booted app (tap 69 ms, scroll works, typing + slider drag land,
screenshot fine); minimized **external** un-flagged app on :9222 (tap 63
ms, scroll 460 ms — emulation alone suffices while attached).
- Plan + implementation went through three adversarial review rounds
(scope, CDP semantics, failure-code taxonomy/telemetry compat, doc
accuracy).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-07-31 15:36:36 +02:00
Jakub Piasecki 551c781fb0 feat(flow): support running end-to-end flows without agent (#429) 
# Flow E2E

This PR turns flows from replayable tool-call recordings into a
declarative, portable e2e test system that runs headlessly via `argent
flow run <name>` — no LLM in the loop, non-zero exit on failure,
suitable for CI.

Agents can still replay flows via `flow-execute` and get the full output
from each step after it finishes.

## Flow format

Flows are YAML files in `.argent/flows/`, now built from **directives**
interpreted by the runner rather than raw tool calls:

- `launch` — terminate + relaunch the app under test (per-platform app
id map supported); its presence marks a flow as **e2e** (controls its
own start state) vs. a **fragment** (runs against current state,
declares an `executionPrerequisite`, composable via `run:` with
cycle/depth guards).
- `tap`, `type` (focus handshake + optional submit), `scroll-to`
(momentum-free, increments by half the container, `within:` for nested
scrollers), `await`/`assert` (`visible`/`exists`/`hidden`/`text`
conditions), `wait`, `snapshot`, `echo`, and raw `tool:` as escape hatch
— still fully supported, so previously recorded flows keep working
unchanged.
- **Selectors** are `{ text, identifier, role }`; a bare string is a
loose selector resolving identifier-first, then visible text. Any
directive failure hard-stops the flow; later steps report `skip`.

## Execution engine

- Flows store no device id — the runner binds a device at replay
(explicit `--device`/`--platform` or the single booted one).
- Selector resolution runs against **platform-unified UI trees**: iOS
native view hierarchy (full testID coverage, gated on native-devtools
connecting), Android full accessibility hierarchy (full resource-id
coverage), Chromium CDP DOM, and the Vega automation toolkit. A shared
flatten/text-hoisting pass lets `text` assertions read what an
identified container visibly shows. Vega is remote-driven, so touch
directives fail with guidance to use `tv-remote`/`keyboard`.
- Steps wait for the tree to **settle** (two consecutive identical
reads) before acting; `launch` additionally gates on platform
tree-source readiness instead of silently degrading.

## Visual snapshots

`snapshot` diffs a full-resolution screenshot against baselines in
`.argent/flows/__baselines__/<flow>/`, keyed by platform + resolution,
with configurable `maxMismatch` and `--update-baselines`. The status bar
is pinned for the whole run (iOS `simctl status_bar`, Android demo mode)
so clocks/battery don't drive diffs. Baseline/current/diff images are
returned as artifacts on failure (and the written baseline on first
capture/update); a clean pass carries none, keeping reports and agent
context lean.

## Recording

The record-live workflow (`flow-start-recording` → `flow-add-step` →
`flow-finish-recording`) now emits directives automatically: coordinate
taps are rewritten to portable selector `tap:` steps (selector captured
from the pre-tap tree), a leading `restart-app` becomes `launch:`,
`flow-execute` of a sibling fragment becomes `run:`, and device ids are
stripped. The skill instructs the agent to polish the saved YAML
afterwards (fold tap+keyboard into `type:`, swipes-to-reach into
`scroll-to:`, awaits into sugar form).

## Also included

- `argent flow run` / `argent flow list` CLI with per-step report
rendering, artifact materialization, and `--json` output.
- New structured failure codes for flow validation, device resolution,
composition, and devtools readiness.
- Extensive unit test coverage (tree adapters, selector matching,
scrolling, composition, type focus, snapshots, status bar).
 2026-07-13 17:15:01 +02:00
Ignacy Łątka 2cefe552d3 fix(argent-lens): strip ELECTRON_RUN_AS_NODE from spawned Electron env (#471) 
## Problem

QA report: **Argent Lens preview window never launches** —
`propose_variant` / `await_user_selection` stage variants correctly, but
the native window never appears and `await_user_selection` hangs. The
tool-server log shows the child crashing at launch:

```
dist/preview-window/main.cjs:28
import_electron.app.setName("Argent Lens");
                    ^
TypeError: Cannot read properties of undefined (reading 'setName')
    ...
    at c._load (node:electron/js2c/node_init:2:18095)
```

## Root cause (reproduced & proven)

`import_electron.app` being `undefined` while the JS still executes is
the exact signature of the Electron binary booting in **Node mode**. For
a downloaded, non-fused Electron the only trigger is
`ELECTRON_RUN_AS_NODE` in the environment. In Node mode
`require("electron")` returns the executable *path string* (not the
module), so `.app` is `undefined`.

The tool-server routinely runs under an Electron-based MCP host (VS
Code, Cursor, the Codex desktop app), which sets
`ELECTRON_RUN_AS_NODE=1` in the env of the Node children it spawns.
`createPreviewWindowManager.ensureOpen()` spread `{ ...process.env }`
straight into its Electron child, so the flag leaked in and the child
booted headless as Node → crash before any window.

Verified against the real Electron binary + the real `main.cjs`:

| Env | Result |
|-----|--------|
| `ELECTRON_RUN_AS_NODE=1` | Crashes at `main.cjs:28` — **byte-identical
to the report**, incl. the `node:electron/js2c/node_init` frame |
| unset | Boots as a real Electron GUI, gets past `setName`, only fails
later at `loadURL` |

The wrapper-bundle / code-signing theory in the report was a **red
herring** — the reporter never controlled for the spawn *environment*.
The same node-mode surfaces differently depending on whether the `.app`
wrapper built: plain path → `setName` crash (what they saw); wrapper
path → Node rejects the Electron-only `--no-sandbox` flag (`bad option:
--no-sandbox`). One fix cures both.

## Fix

Strip `ELECTRON_RUN_AS_NODE` before spawning, via a shared
`electronGuiChildEnv()` helper, at **both** GUI-Electron spawn sites:
- `preview-window.ts` — the Lens window (the reported bug).
- `boot-electron.ts` — the same latent flaw would boot a headless CDP
app with **no debug port**, timing out `boot-device --electronAppPath`.

## Verification

- **Root cause** reproduced byte-for-byte, both directions, against the
real binary.
- **End-to-end**: ran the real fixed `ensureOpen()` path via ts-node
with `ELECTRON_RUN_AS_NODE=1` leaked into the parent env against the
real Electron binary — child boots as a real GUI app (`VERDICT: PASS`).
Reverting the one-line fix flips it to `FAIL`, confirming the test
discriminates.
- **Unit tests**: added `electron-env.test.ts` (helper contract) + a
regression case in `preview-window-launch-failure.test.ts` asserting the
spawn env strips the var and keeps the override. 21/21 pass.
- `tsc --noEmit` (build) and `tsc --noEmit -p tsconfig.test.json` (CI
"Unit tests" gate) both clean; prettier + eslint clean on all changed
files.
 2026-07-07 13:36:27 +02:00
Ignacy Łątka 9b027096d7 feat: Electron support (#247) 
## Summary

Adds Electron (Chromium + CDP) as a third device platform alongside iOS
and Android. Argent can boot an Electron app, list it next to
simulators/emulators, drive it with the same tool surface where it makes
sense — taps, swipes, keyboard, screenshots, describe, navigation — and
debug it via the four ported `debugger-*` tools.

Mirrors the sim-server architecture in TypeScript: a per-device server
with HTTP API, MJPEG screencast, refcounted CDP screencast, FPS tracker,
clipboard, and a WebSocket command bus.

## What works on Electron

| group | tools | status |
|---|---|---|
| Lifecycle | `boot-device` (with `electronAppPath`), `launch-app`,
`open-url` | ✅ works |
| Vision | `screenshot`, `describe` (DOM walker w/ shadow-DOM +
same-origin iframes) | ✅ works |
| Input | `gesture-tap`, `gesture-swipe`, `keyboard` (text + named keys)
| ✅ works |
| Sequencing | `run-sequence` (per-step capability check) | ✅ works |
| **JS debugger** | `debugger-connect`, `debugger-status`,
`debugger-evaluate`, `debugger-log-registry` | ✅ **direct CDP** |
| Discovery | `list-devices` (Electron entries shown alongside
sims/emulators) | ✅ works |
| Cleanup | `stop-simulator-server`, `stop-all-simulator-servers`,
`stop-metro` | ✅ works |
| Misc | `update-argent`, `dismiss-update`, `gather-workspace-data`, all
`flow-*` recording tools | ✅ works |

### Debugger port — how it routes

Dispatch lives in `tools/debugger/debugger-service-ref.ts`: an Electron
device id (`electron-cdp-<port>`) routes to a new
`ElectronJsRuntimeDebugger` blueprint that wraps the page CDP session
already opened by `boot-device`. Everything else stays on the
Metro-driven `JsRuntimeDebugger` blueprint. The new blueprint exposes
the same `JsRuntimeDebuggerApi` shape so tool code is unchanged.
`Runtime.consoleAPICalled` events feed into the existing `LogFileWriter`
+ cluster pipeline — `debugger-log-registry` returns the same shape on
Electron as it does on Metro.

## What is cleanly rejected with `Tool 'X' is not supported on electron
app`

HTTP capability gate declines these up-front (HTTP 400):

| group | tools |
|---|---|
| Mobile gestures | `gesture-pinch`, `gesture-rotate`, `gesture-custom`
|
| Mobile controls | `button`, `rotate` |
| App lifecycle | `restart-app`, `reinstall-app` |
| iOS-native | `paste`, all `native-*`, `native-profiler-*` |
| RN-only debugger | `debugger-component-tree`, `debugger-reload-metro`,
`debugger-inspect-element` |
| RN-only network | `view-network-logs`, `view-network-request-details`
|
| RN-only profiler | all `react-profiler-*` (incl. `component-source`),
all `profiler-*` query / combined tools |

## Verification history

Three commits resolved findings from two parallel verification swarms:

- `507b295` — swarm v1 fixes: early-disconnect race in new blueprint,
NaN timestamp crash, disconnect listener leak, missing test coverage for
13 of 15 lockouts, missing disconnect→terminated propagation test, stale
skill + tool descriptions.
- `dd0f185` — pre-existing crash: `spawn()` in boot-electron lacked an
`'error'` event handler; ENOENT escalated to `uncaughtException`. + 4
regression test cases.
- `95161ab` — swarm v2 fixes: orphan promise rejection when pid-less
spawn fires deferred `'error'`, misleading rationale comment, stale
cross-skill references calling this "Metro debugging only".

Branch was also cherry-picked onto v0.8.0 (`e1327b4`) to pick up #245.

## Follow-ups left for separate PRs

- **CPU profiling on Electron** — Chromium's
`Profiler.start`/`Profiler.stop` work fine; only the V8 sample format
differs from Hermes'. A normalizer in the query layer would unlock
`react-profiler-cpu-summary` + `profiler-cpu-query` for arbitrary
Electron renderers (without the React-DevTools commit half).
- **Source-mapped element inspection on Electron** —
`debugger-inspect-element` could be rebuilt on top of
`DOM.getNodeForLocation` + CSS source maps, but it'd be a from-scratch
implementation rather than a port.

## New sim-server-equivalent HTTP surface

Mounted at `/electron-server/:deviceId/*` — not advertised to MCP,
consumed by preview UIs and integration tests:

- `GET /viewport`, `GET /stream.mjpeg` (multipart-JPEG screencast)
- `POST /api/screenshot`, `/api/clipboard/text`, `/api/fps`,
`/api/navigate`, `/api/reload`, `/api/history/back`,
`/api/history/forward`
- `WS /ws` (input commands + event bus)

Sharp is loaded as an optional dependency — screenshots still work
without it (no rotate/resize, just raw PNG).

## Test plan

- [x] 786 vitest cases pass; new coverage for the dispatch helper,
blueprint factory edge cases (early disconnect, NaN timestamp, dispose
symmetry), 15-tool table-driven lockout assertion, and 5 boot-electron
spawn-error scenarios.
- [x] Build + prettier clean.
- [x] E2E swept the original 27 surfaces (11 MCP tools, 8 HTTP
endpoints, MJPEG, WebSocket, 6 capability rejections, session cleanup).
- [x] E2E swept the debugger surface against a live Electron app —
connect / status / evaluate (`1+1`, JSON-stringified globals,
`document.title`) / log-registry (clusters + log file).
- [x] E2E swept the 17 lockouts — every one returns HTTP 400 with `Tool
'X' is not supported on electron app`.
- [x] Two rounds of parallel verification swarm; every real finding
addressed.
- [ ] Reviewer drives an Electron app end-to-end via MCP and confirms
describe / tap / swipe / keyboard / debugger-evaluate land correctly.

---------

Co-authored-by: Filip131311 <contact@filipkaminski.com>
 2026-06-12 17:54:45 +02:00