4 Commits

Author	SHA1	Message	Date
Ignacy Łątka	2251c7cf46	feat(screen-recording): screen-recording-start/stop tools with per-call stop reminders (#517) ... ## What Adds a `screen-recording-start` / `screen-recording-stop` tool pair (naming follows `native-profiler-start/stop`) that records the device screen to an mp4 and returns it as a downloadable artifact. Capture runs through **simulator-server** — the same backend `screenshot` and every interaction tool already drive. simulator-server publishes the device screen as an MJPEG stream (`/stream.mjpeg`), so recording needs no platform-specific recorder: the tool-server subscribes to that stream, paces the frames onto a fixed 30 fps timeline, and pipes them into a single ffmpeg process that encodes — and watermarks — straight to the final mp4. The stream is multi-subscriber, so a preview window can stay open on the same device while recording. - **One path for every device simulator-server drives** — iOS simulators, Android emulators, and physical Android devices. - **Native resolution, constant 30 fps** — h264, `+faststart`, no post-processing pass. - Session state lives in a per-device `ScreenRecordingSession` blueprint (mirrors the native-profiler session): time-limit cap with auto-finalize, unexpected-exit recovery (a capture that ended on its own is still retrievable via stop), always-startable state reset on stop, and dispose-on-shutdown cleanup through the registry. ## Why the frames are paced host-side A device only emits a frame when the screen **changes**. Pointing ffmpeg straight at the stream therefore produces a video whose length is the sum of the moments something moved — a still screen collapses to a fraction of a second — and an ffmpeg blocked on a silent socket does not answer a stop signal promptly either (measured: 12 s late). Re-emitting the last frame on a wall-clock schedule fixes both: **video duration always equals real elapsed time**, and stop is a stdin EOF that finalizes in ~60 ms. Duplicate frames cost almost nothing once encoded (a 10 s recording of a static screen is ~1 MB). ## Finishing pass folded into capture The Argent watermark (logo + `By @swmansion`, bottom-left, sized relative to the frame) is stamped by the same ffmpeg invocation that encodes the capture, so there is no second encode and no temp intermediate. Its colour is chosen **per pixel** from the background luma underneath it — the white logo over dark areas, a runtime-derived near-black copy over light areas — so it keeps maximum contrast against whatever is behind it. It sits at 20 % opacity and is opt-out via `argent disable video-watermark` (read when the recording starts). Frame geometry comes from the first JPEG's SOF header, so no `ffprobe` is needed. ffmpeg is now the recorder rather than an optional finisher, so a missing ffmpeg fails `screen-recording-start` up front with an install hint instead of silently degrading the output at stop time. ## Reminder plumbing While a recording is running — or ended on its own but was never retrieved — **every** tool result carries a `note` telling the agent the recording is still going and how to stop it. It rides the same `note` field as the update reminder but is per-call and unsuppressed; both notes compose when they coincide. The note text switches to "already ended, retrieve the file" once the cap fires or the capture dies. ## Skill New `argent-screen-recording` skill (+ routing stanza in `rules/argent.md`): documents the start → interact → stop workflow and instructs the agent to schedule its own reminder/wakeup for the expected end of the recording immediately after starting one (harness reminder tool if available, otherwise a background `sleep` whose completion notification pulls it back). Scores 10/10 on `scripts/grade-skills.mjs`. ## Verification - Unit: 45 tests across `screen-recording.test.ts`, `mjpeg-stream.test.ts`, `watermark.test.ts`, `http-recording-note.test.ts`; full tool-server suite **2808 tests / 280 files green**; `tsc --noEmit -p tsconfig.test.json`, root `tsc --build`, eslint, prettier all clean. - E2E on a live tool-server (`POST /tools/...`), driving real interactions between start and stop: - **iOS sim** (iPhone 16 Pro Max, iOS 18.5): 1320×2868 h264, `r_frame_rate=30/1`, 338 frames, container duration 11.27 s vs reported `durationMs` 11314 — **within 50 ms of wall clock**. Extracted frames show real screen content (mid app-switcher animation) with the watermark rendered light-over-dark. - **Android emulator** (Pixel 3a, API 34): 1080×2220 h264, 30 fps, 353 frames, duration 11.77 s vs `durationMs` 11793 — within 30 ms. 512 KB for a mostly-static home screen. - **Static screen** (no interaction at all, 10 s): 300 frames, duration exactly 10.000 s — the case the previous `recordVideo` implementation reduced to ~0.03 s. - **Long silence then activity** (15 s untouched, then an app launch): stream survived, no warning, frames after the silence show the new screen. This caught a real bug during review — the connect timeout was also acting as a socket *inactivity* timeout and tore the stream down at 10 s of stillness; fixed and covered by a regression test that fails without the fix. - **Time-limit cap**: `timeLimitSeconds: 5` self-finalized at 4.97 s / 149 frames, the per-call note flipped to "already ended (it hit its 5s time limit)", and a stop issued 11 s later still returned the video with the matching warning. ## Not supported (capability-gated with clear errors) tvOS simulators, physical iPhones, Chromium apps, Vega, and remote (`remote:`-prefixed) simulators — none of them expose a readable frame stream. --------- Co-authored-by: Ignacy Łątka <claude-hubert.gancarczyk@swmansion.com>	2026-07-23 21:14:37 +02:00
filip131311	8d2a499eaf	Project-scope configuration with schema-driven merge + argent config (#534) ... ## Summary Adds a general, **scoped** configuration system on top of the shared `config.json`, plus a universal `argent config` command to manage it. Motivated by the discussion on #401 (iOS device set): config that needs to work naturally with the MCP server should be read off disk at runtime like telemetry consent is — not threaded through CLI flags/env and a kill-and-respawn dance. This lands the reusable machinery; the device-set becomes a one-line schema entry + a runtime reader on top of it. Four pieces, matching the requirements: ### 1. Project-scope config files `config.json` now resolves at two scopes: - `~/.argent/config.json` (**global**, default) - `<project-root>/.argent/config.json` (**project**, resolved by the same `.argent`/`.git`/`package.json` marker walk as feature flags) `configFilePath` / `readConfigObject` / `updateConfig` take an optional `scope` (default `global`), so **every existing caller is unchanged** (telemetry, notices, lens). Per-scope lock + atomic write are preserved. ### 2. Schema-driven merged reads `getConfigValue(def)` reads each scope the value's schema entry allows, merges them under the entry's policy, and falls back to the entry's default. This is the reader consumers should call instead of poking at a single global file. The existing **lens** getters (`getRememberedAgent` etc.) are migrated onto it as the first live consumer — a repo can now pin its lens agent, falling back to the user's global choice. ### 3. A registration file for configurations (`config-schema.ts`) Adding a configuration is a one-liner: `key`, `description`, allowed `scopes`, a `parse` validator, and a `merge` policy. Merge presets: - `prioritize-local` (project wins) · `prioritize-global` (global wins) - `prioritize-restrictive` (booleans/numbers — opt-out/stricter wins) - `union` / `intersection` (arrays) - …plus an escape hatch: pass your **own** `(inputs) => value` function for a bespoke rule. ### 4. `argent config` — one universal command `argent config <list | get | set | unset>` with `--scope global|project` and `--json`. New configs are added as **schema entries**, not new top-level `argent` commands, to avoid bloat. **Telemetry stays backward-compatible** on its dedicated `argent telemetry` command (which also drains/resets the live client). It's surfaced *read-only* under `argent config` via a `manageCommand` guard — so `argent config set telemetry.enabled` refuses and points at `argent telemetry`, and a committed project file can never re-enable a global opt-out (`prioritize-restrictive`, global-only). ## Testing - **Unit / integration**: 71 configuration-core tests (merge presets + custom fn, dotted-path + prototype-pollution guards, scope merge, scope/validation/manage-command errors, `listConfig`, `coerceCliValue`) and 11 real-filesystem CLI tests driving the actual `config()` entry point across sandboxed HOME + project cwd. Full suites green: configuration-core 71, argent-cli 268, telemetry 281, argent 60. - **Live, on the built binary** (`node packages/argent/dist/cli.js …`): set global + project (from a deep subdir) → effective get returns the project value; per-scope gets return each file; both files land in the right place; `argent telemetry disable/enable` writes `config.json` and `argent config get telemetry.enabled` reflects it read-only; unknown-key / invalid-value / bad-scope errors exit non-zero with hints. - `tsc --build` across the workspace, prettier, and eslint (`--max-warnings 0`) all clean. ## Notes / follow-ups - Config-file reads are not yet fingerprint-cached the way telemetry's consent read is; a long-lived tool-server re-reads on each `getConfigValue`. Fine for CLI/low-frequency use; a hot runtime consumer (e.g. a future device-set reader) may want the same mtime/inode cache.	2026-07-22 12:47:42 +02:00
Stanisław Chmiela	a7775e4ba6	feat(tool-server): add tool-server event log (#367) ... We want to display a nicely parsed and rendered log of events that happened over a span of an Argent session. Current log file is not designed to be machine-readable, so adding a JSONL-formatted log of events (gated by `tool-server-event-log`). I eventually want this 😋 https://x.com/j3nnings/status/2069072177584849192 When enabled, the tool server writes Bunyan-compatible records to `ARGENT_EVENT_LOG` or `~/.argent/tool-server-events.jsonl`. The log captures tool-server lifecycle events and registry lifecycle events, including tool invocations and others. To make messages nicer, added `interaction` object to tool definition. This allows tools to define a nicer message in log (print "Screenshot capture completed." instead of "Tool `screenshot` completed.") Adding these to the existing tools is a follow up PR matter I think.	2026-07-10 11:52:28 +02:00
Paweł Fornagiel	92cc69a75c	feat: extract configuration-core and migrate auto-screenshot to feature flag (#295) ... ## Summary - Introduces `@argent/configuration-core` as the shared source of truth for feature-flag storage, the registry, and `isFlagEnabled`, so MCP and CLI can import the same primitives without coupling runtime code to CLI-only modules. - Replaces `ARGENT_AUTO_SCREENSHOT` with the off-by-default `disable-auto-screenshot` flag (auto-screenshot stays on unless enabled via `argent enable disable-auto-screenshot`); renames `ARGENT_AUTO_SHUTDOWN` to `ARGENT_TOOL_SERVER_SHUTDOWN_ON_MCP_EXIT`. - Keeps `@argent/cli` as the command layer (`enable` / `disable` / `flags`) with backward-compat re-exports, colored flag state output (picocolors), and `--help` listings of available flags. --------- Co-authored-by: Filip131311 <contact@filipkaminski.com> Co-authored-by: Cursor <cursoragent@cursor.com>	2026-06-10 15:01:25 +02:00