mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
fix/docs-high-severity
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ignacy Łątka
|
2251c7cf46 |
feat(screen-recording): screen-recording-start/stop tools with per-call stop reminders (#517)
## What Adds a `screen-recording-start` / `screen-recording-stop` tool pair (naming follows `native-profiler-start/stop`) that records the device screen to an mp4 and returns it as a downloadable artifact. Capture runs through **simulator-server** — the same backend `screenshot` and every interaction tool already drive. simulator-server publishes the device screen as an MJPEG stream (`/stream.mjpeg`), so recording needs no platform-specific recorder: the tool-server subscribes to that stream, paces the frames onto a fixed 30 fps timeline, and pipes them into a single ffmpeg process that encodes — and watermarks — straight to the final mp4. The stream is multi-subscriber, so a preview window can stay open on the same device while recording. - **One path for every device simulator-server drives** — iOS simulators, Android emulators, and physical Android devices. - **Native resolution, constant 30 fps** — h264, `+faststart`, no post-processing pass. - Session state lives in a per-device `ScreenRecordingSession` blueprint (mirrors the native-profiler session): time-limit cap with auto-finalize, unexpected-exit recovery (a capture that ended on its own is still retrievable via stop), always-startable state reset on stop, and dispose-on-shutdown cleanup through the registry. ## Why the frames are paced host-side A device only emits a frame when the screen **changes**. Pointing ffmpeg straight at the stream therefore produces a video whose length is the sum of the moments something moved — a still screen collapses to a fraction of a second — and an ffmpeg blocked on a silent socket does not answer a stop signal promptly either (measured: 12 s late). Re-emitting the last frame on a wall-clock schedule fixes both: **video duration always equals real elapsed time**, and stop is a stdin EOF that finalizes in ~60 ms. Duplicate frames cost almost nothing once encoded (a 10 s recording of a static screen is ~1 MB). ## Finishing pass folded into capture The Argent watermark (logo + `By @swmansion`, bottom-left, sized relative to the frame) is stamped by the same ffmpeg invocation that encodes the capture, so there is no second encode and no temp intermediate. Its colour is chosen **per pixel** from the background luma underneath it — the white logo over dark areas, a runtime-derived near-black copy over light areas — so it keeps maximum contrast against whatever is behind it. It sits at 20 % opacity and is opt-out via `argent disable video-watermark` (read when the recording starts). Frame geometry comes from the first JPEG's SOF header, so no `ffprobe` is needed. ffmpeg is now the recorder rather than an optional finisher, so a missing ffmpeg fails `screen-recording-start` up front with an install hint instead of silently degrading the output at stop time. ## Reminder plumbing While a recording is running — or ended on its own but was never retrieved — **every** tool result carries a `note` telling the agent the recording is still going and how to stop it. It rides the same `note` field as the update reminder but is per-call and unsuppressed; both notes compose when they coincide. The note text switches to "already ended, retrieve the file" once the cap fires or the capture dies. ## Skill New `argent-screen-recording` skill (+ routing stanza in `rules/argent.md`): documents the start → interact → stop workflow and instructs the agent to schedule its own reminder/wakeup for the expected end of the recording immediately after starting one (harness reminder tool if available, otherwise a background `sleep` whose completion notification pulls it back). Scores 10/10 on `scripts/grade-skills.mjs`. ## Verification - Unit: 45 tests across `screen-recording.test.ts`, `mjpeg-stream.test.ts`, `watermark.test.ts`, `http-recording-note.test.ts`; full tool-server suite **2808 tests / 280 files green**; `tsc --noEmit -p tsconfig.test.json`, root `tsc --build`, eslint, prettier all clean. - E2E on a live tool-server (`POST /tools/...`), driving real interactions between start and stop: - **iOS sim** (iPhone 16 Pro Max, iOS 18.5): 1320×2868 h264, `r_frame_rate=30/1`, 338 frames, container duration 11.27 s vs reported `durationMs` 11314 — **within 50 ms of wall clock**. Extracted frames show real screen content (mid app-switcher animation) with the watermark rendered light-over-dark. - **Android emulator** (Pixel 3a, API 34): 1080×2220 h264, 30 fps, 353 frames, duration 11.77 s vs `durationMs` 11793 — within 30 ms. 512 KB for a mostly-static home screen. - **Static screen** (no interaction at all, 10 s): 300 frames, duration exactly 10.000 s — the case the previous `recordVideo` implementation reduced to ~0.03 s. - **Long silence then activity** (15 s untouched, then an app launch): stream survived, no warning, frames after the silence show the new screen. This caught a real bug during review — the connect timeout was also acting as a socket *inactivity* timeout and tore the stream down at 10 s of stillness; fixed and covered by a regression test that fails without the fix. - **Time-limit cap**: `timeLimitSeconds: 5` self-finalized at 4.97 s / 149 frames, the per-call note flipped to "already ended (it hit its 5s time limit)", and a stop issued 11 s later still returned the video with the matching warning. ## Not supported (capability-gated with clear errors) tvOS simulators, physical iPhones, Chromium apps, Vega, and remote (`remote:`-prefixed) simulators — none of them expose a readable frame stream. --------- Co-authored-by: Ignacy Łątka <claude-hubert.gancarczyk@swmansion.com> |
||
|
Ignacy Łątka
|
f3d108a5a1 |
feat: Argent Lens (#271)
Adds a UI for proposing and choosing between multiple agent-generated variants on a running app. **MCP tools** (`packages/tool-server/src/tools/variants/`) - `propose_variant` — non-blocking; agent stages a variant for an on-screen element (text/label/identifier/role matcher). - `await_user_selection` — one blocking call; parks until the human submits, then returns the picks + free-form annotations + a global comment. **Preview UI** (`packages/ui`) - Floating cards anchored to live element frames via the existing `describe` tree, with connector lines. - Comment-mode inspector: hover spotlight (blur+dim everything else), right-click any element to pin a comment. - Sticky padded hitbox + trajectory-aware dismissal so focus doesn't drop on cursor jitter. - Footbar [Complete] [Clear], confirm modal for destructive actions, green toast on submit. - Single source of feel constants in `theme.css`. **Electron host** (`packages/preview-window`) - New workspace package. Frameless transparent BrowserWindow at fixed size; the squeeze open/close is a CSS `transform: scaleY` on `<html>`, GPU-composited — no OS-level resize. - Tool-server spawns it on `await_user_selection` park, reuses it across re-awaits, and asks it to animate-close on submit (1s grace so the toast is visible). - Stdio-line IPC (no extra ports). Ephemeral throughout — the renderer hits the tool-server's already-ephemeral `/preview/*` HTTP. --------- Co-authored-by: Filip131311 <contact@filipkaminski.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ignacy Łątka
|
acfe80ab6b |
chore: add ESLint with typescript-eslint (#349)
## What Adds type-aware linting to the workspace. Until now the only static gate on JS/TS was Prettier (formatting) and `tsc` — there was no linter, so the whole class of bugs `tsc` doesn't catch (floating promises, misused promises, throwing non-Errors, unsafe stringification, dead code) went unguarded across ~450 TS files. - **`eslint.config.mjs`** — flat config. `recommendedTypeChecked` runs on the TS sources via an explicit `project` list (each package keeps tests in a separate `tsconfig.test.json` that `projectService` can't auto-discover, so it's listed explicitly). Type-checked rules are disabled for the loosely-typed plain-JS build/dev scripts. The `argent-private` submodule and build artifacts are ignored (mirrors `.prettierignore`). - **`lint` / `lint:fix`** npm scripts. - **`.github/workflows/lint.yml`** — builds the workspace (type-aware rules need the referenced projects' declarations) then runs `eslint .`. Node 24 (ESLint 10 requires `^20.19 || ^22.13 || >=24`). ## Rule calibration This is an adoption PR, so it's deliberately calibrated rather than maximal: - **Kept on (bug-catchers):** `no-floating-promises`, `no-misused-promises` (with `checksVoidReturn: false` to drop the noisy callback variant), `only-throw-error`, `no-base-to-string`, `prefer-promise-reject-errors`, `no-unused-vars` (`_`-prefix escape hatch), plus the ESLint recommended set. - **Parked off as documented debt:** the high-volume style/`any`-family rules — `no-unsafe-*`, `require-await`, `no-unnecessary-type-assertion`, `restrict-template-expressions`, `unbound-method`, `no-explicit-any`, etc. These flag broad classes of pre-existing code and are out of scope here; each is commented as a ratchet target for follow-up passes. All findings from the kept-on rules are fixed in this PR (dead code/imports, error-cause chaining, **3 genuine floating promises**, non-Error throws, unsafe stringification). Stale unused `eslint-disable` directives left over in the tree were removed by `--fix`. ## Verification Locally, all green: `eslint .` (0 errors), `tsc --build`, `typecheck:tests`/`typecheck:scripts`, the full `vitest` suite (no behavior change from the fixes), and `prettier --check`. ## Not included (intentionally) Other static-analysis gates discussed (CodeQL, dependency-audit, ShellCheck, actionlint) are out of scope for this PR. |
||
|
Paweł Fornagiel
|
92cc69a75c |
feat: extract configuration-core and migrate auto-screenshot to feature flag (#295)
## Summary - Introduces `@argent/configuration-core` as the shared source of truth for feature-flag storage, the registry, and `isFlagEnabled`, so MCP and CLI can import the same primitives without coupling runtime code to CLI-only modules. - Replaces `ARGENT_AUTO_SCREENSHOT` with the off-by-default `disable-auto-screenshot` flag (auto-screenshot stays on unless enabled via `argent enable disable-auto-screenshot`); renames `ARGENT_AUTO_SHUTDOWN` to `ARGENT_TOOL_SERVER_SHUTDOWN_ON_MCP_EXIT`. - Keeps `@argent/cli` as the command layer (`enable` / `disable` / `flags`) with backward-compat re-exports, colored flag state output (picocolors), and `--help` listings of available flags. --------- Co-authored-by: Filip131311 <contact@filipkaminski.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Ignacy Łątka
|
c5c3f7af11 |
feat: feature flags (#260)
## Summary
- New top-level CLI commands `argent enable <flag>` / `argent disable
<flag>` / `argent flags`.
- Storage is a small JSON file at `~/.argent/flags.json` by default;
`--scope project` writes to `<project-root>/.argent/flags.json` instead.
- `enable` writes `true`. `disable` removes the entry at the chosen
scope (and deletes the file + empty `.argent/` when it becomes the last
flag).
- `isFlagEnabled` walks project → global, so a project entry shadows the
same key in the global file. To opt a single project out of a
globally-enabled flag, hand-edit `<project>/.argent/flags.json` to
`{"flags":{"name":false}}` — there is no CLI for an explicit override.
- The dispatcher routes the new commands to `@argent/cli` exports;
bundling needs no config changes (only `node:fs|path|os`).
## Behaviour
- `argent enable foo` → writes `{ "flags": { "foo": true } }` to global
file.
- `argent disable foo` → removes `foo` from the global file (clean round
trip with `enable`).
- `argent enable foo --scope project` → writes to
`<project-root>/.argent/flags.json`.
- `argent flags` → lists effective state with the winning scope;
`--json` for scripting.
- Project root is the nearest ancestor with `.argent`, `.git`, or
`package.json` — falls back to cwd when none found.
- POSIX `--` separator: `argent enable -- weird-name` is supported.
- Help (`--help` / `-h`), invalid `--scope`, invalid flag names, missing
arg, unknown flags all produce a clear stderr error and `exit 2`.
- Flag-name validation: `/^[a-zA-Z][a-zA-Z0-9._-]*$/`.
- Writes are atomic (tmp file + rename) so concurrent invocations cannot
tear the JSON.
## Reader API
`isFlagEnabled(name)` is exported from `@argent/cli` for future
consumers; it walks project → global and returns `false` when unset.
## Files
- `packages/argent-cli/src/flags.ts` (new) — storage + CLI handlers
- `packages/argent-cli/src/index.ts` — re-exports `enable` / `disable` /
`flags` / `isFlagEnabled` / `FlagScope`
- `packages/argent-cli/test/flags.test.ts` (new) — 41 tests
- `packages/argent-cli/vitest.config.ts` (new)
- `packages/argent-cli/package.json` — description bump
- `packages/argent/src/cli.ts` — dispatcher routes + help text
- `README.md`, `CONTRIBUTING.md` — command tables
## Test plan
- [x] `npm test --workspaces --if-present` → 1085 tests pass (41 in
`@argent/cli`).
- [x] `npm run typecheck:tests --workspaces --if-present` → clean.
- [x] `npm run build` (root `tsc --build`) → clean.
- [x] `npm run build -w @swmansion/argent` (dispatcher + esbuild
bundles) → clean.
- [x] `npx prettier --check .` → clean.
- [x] End-to-end against the built dispatcher in a tmpdir:
- `argent enable my-feature-flag` (global) writes to
`~/.argent/flags.json`.
- `argent enable proj-only --scope project` writes to
`<proj>/.argent/flags.json`.
- `argent disable my-feature-flag` removes the entry; round trip leaves
a clean tree (file + empty `.argent/` removed when no other state lives
there).
- `argent disable foo` preserves sibling flags in the same file.
- `argent flags` lists each flag with the winning scope.
- `argent enable --` errors cleanly; `argent enable -- name-after`
accepts the literal.
- `argent enable foo --scope project` walks up to `.git` /
`package.json` / `.argent`; stray editor configs (`.claude`, `.cursor`,
…) do NOT count as project roots.
- Error paths (`--scope bogus`, no name, `1bad-name`) exit `2` with a
clear message.
|