Files
Ignacy Łątka 2251c7cf46 feat(screen-recording): screen-recording-start/stop tools with per-call stop reminders (#517)
## What

Adds a `screen-recording-start` / `screen-recording-stop` tool pair
(naming follows `native-profiler-start/stop`) that records the device
screen to an mp4 and returns it as a downloadable artifact.

Capture runs through **simulator-server** — the same backend
`screenshot` and every interaction tool already drive. simulator-server
publishes the device screen as an MJPEG stream (`/stream.mjpeg`), so
recording needs no platform-specific recorder: the tool-server
subscribes to that stream, paces the frames onto a fixed 30 fps
timeline, and pipes them into a single ffmpeg process that encodes — and
watermarks — straight to the final mp4. The stream is multi-subscriber,
so a preview window can stay open on the same device while recording.

- **One path for every device simulator-server drives** — iOS
simulators, Android emulators, and physical Android devices.
- **Native resolution, constant 30 fps** — h264, `+faststart`, no
post-processing pass.
- Session state lives in a per-device `ScreenRecordingSession` blueprint
(mirrors the native-profiler session): time-limit cap with
auto-finalize, unexpected-exit recovery (a capture that ended on its own
is still retrievable via stop), always-startable state reset on stop,
and dispose-on-shutdown cleanup through the registry.

## Why the frames are paced host-side

A device only emits a frame when the screen **changes**. Pointing ffmpeg
straight at the stream therefore produces a video whose length is the
sum of the moments something moved — a still screen collapses to a
fraction of a second — and an ffmpeg blocked on a silent socket does not
answer a stop signal promptly either (measured: 12 s late). Re-emitting
the last frame on a wall-clock schedule fixes both: **video duration
always equals real elapsed time**, and stop is a stdin EOF that
finalizes in ~60 ms. Duplicate frames cost almost nothing once encoded
(a 10 s recording of a static screen is ~1 MB).

## Finishing pass folded into capture

The Argent watermark (logo + `By @swmansion`, bottom-left, sized
relative to the frame) is stamped by the same ffmpeg invocation that
encodes the capture, so there is no second encode and no temp
intermediate. Its colour is chosen **per pixel** from the background
luma underneath it — the white logo over dark areas, a runtime-derived
near-black copy over light areas — so it keeps maximum contrast against
whatever is behind it. It sits at 20 % opacity and is opt-out via
`argent disable video-watermark` (read when the recording starts). Frame
geometry comes from the first JPEG's SOF header, so no `ffprobe` is
needed.

ffmpeg is now the recorder rather than an optional finisher, so a
missing ffmpeg fails `screen-recording-start` up front with an install
hint instead of silently degrading the output at stop time.

## Reminder plumbing

While a recording is running — or ended on its own but was never
retrieved — **every** tool result carries a `note` telling the agent the
recording is still going and how to stop it. It rides the same `note`
field as the update reminder but is per-call and unsuppressed; both
notes compose when they coincide. The note text switches to "already
ended, retrieve the file" once the cap fires or the capture dies.

## Skill

New `argent-screen-recording` skill (+ routing stanza in
`rules/argent.md`): documents the start → interact → stop workflow and
instructs the agent to schedule its own reminder/wakeup for the expected
end of the recording immediately after starting one (harness reminder
tool if available, otherwise a background `sleep` whose completion
notification pulls it back). Scores 10/10 on `scripts/grade-skills.mjs`.

## Verification

- Unit: 45 tests across `screen-recording.test.ts`,
`mjpeg-stream.test.ts`, `watermark.test.ts`,
`http-recording-note.test.ts`; full tool-server suite **2808 tests / 280
files green**; `tsc --noEmit -p tsconfig.test.json`, root `tsc --build`,
eslint, prettier all clean.
- E2E on a live tool-server (`POST /tools/...`), driving real
interactions between start and stop:
- **iOS sim** (iPhone 16 Pro Max, iOS 18.5): 1320×2868 h264,
`r_frame_rate=30/1`, 338 frames, container duration 11.27 s vs reported
`durationMs` 11314 — **within 50 ms of wall clock**. Extracted frames
show real screen content (mid app-switcher animation) with the watermark
rendered light-over-dark.
- **Android emulator** (Pixel 3a, API 34): 1080×2220 h264, 30 fps, 353
frames, duration 11.77 s vs `durationMs` 11793 — within 30 ms. 512 KB
for a mostly-static home screen.
- **Static screen** (no interaction at all, 10 s): 300 frames, duration
exactly 10.000 s — the case the previous `recordVideo` implementation
reduced to ~0.03 s.
- **Long silence then activity** (15 s untouched, then an app launch):
stream survived, no warning, frames after the silence show the new
screen. This caught a real bug during review — the connect timeout was
also acting as a socket *inactivity* timeout and tore the stream down at
10 s of stillness; fixed and covered by a regression test that fails
without the fix.
- **Time-limit cap**: `timeLimitSeconds: 5` self-finalized at 4.97 s /
149 frames, the per-call note flipped to "already ended (it hit its 5s
time limit)", and a stop issued 11 s later still returned the video with
the matching warning.

## Not supported (capability-gated with clear errors)

tvOS simulators, physical iPhones, Chromium apps, Vega, and remote
(`remote:`-prefixed) simulators — none of them expose a readable frame
stream.

---------

Co-authored-by: Ignacy Łątka <claude-hubert.gancarczyk@swmansion.com>
2026-07-23 21:14:37 +02:00

394 lines
16 KiB
TypeScript

import { describe, it, expect, beforeEach, afterEach } from "vitest";
import * as fs from "node:fs";
import * as path from "node:path";
import * as os from "node:os";
import { disable, enable, flags as flagsCmd } from "../src/flags.js";
import {
getFlagsPath,
isFlagEnabled,
readFlags,
setFlag,
type FlagDefinition,
} from "@argent/configuration-core";
// Hermetic registry for the CLI tests so they never depend on which flags ship
// in the production FLAG_REGISTRY. enable()/flags() take an injectable registry
// for exactly this reason.
const TEST_REGISTRY: readonly FlagDefinition[] = [
{ name: "my-feature-flag", description: "Primary test flag." },
{ name: "proj-flag", description: "Project-scoped test flag." },
{ name: "g", description: "Global explicit-scope test flag." },
{ name: "after-dashdash", description: "Enabled after the -- separator." },
{ name: "x", description: "Round-trip test flag." },
{ name: "a", description: "Listing flag A." },
{ name: "b", description: "Listing flag B." },
{
name: "opt-out-flag",
description: "On by default; disable persists off.",
defaultEnabled: true,
},
];
// All tests redirect global+project storage into tmp dirs by mutating
// process.env.HOME (consumed by os.homedir()) and process.cwd().
let tmpHome: string;
let tmpProject: string;
let originalHome: string | undefined;
let originalUserProfile: string | undefined;
let originalCwd: string;
beforeEach(() => {
// realpath unwraps macOS's /var → /private/var tmpdir symlink so the path
// we hand back from getFlagsPath matches what process.cwd() reports after
// chdir().
tmpHome = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "argent-flags-home-")));
tmpProject = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "argent-flags-proj-")));
// Drop a marker so resolveProjectRoot stops here instead of walking up to
// the actual user's repo and writing into it.
fs.writeFileSync(path.join(tmpProject, "package.json"), "{}");
originalHome = process.env.HOME;
originalUserProfile = process.env.USERPROFILE;
originalCwd = process.cwd();
process.env.HOME = tmpHome;
process.env.USERPROFILE = tmpHome;
process.chdir(tmpProject);
});
afterEach(() => {
process.chdir(originalCwd);
if (originalHome === undefined) delete process.env.HOME;
else process.env.HOME = originalHome;
if (originalUserProfile === undefined) delete process.env.USERPROFILE;
else process.env.USERPROFILE = originalUserProfile;
fs.rmSync(tmpHome, { recursive: true, force: true });
fs.rmSync(tmpProject, { recursive: true, force: true });
});
interface CapturedConsole {
stdout: string;
stderr: string;
}
// Replaces console.log / console.error / process.exit with capturing stubs
// for the duration of `fn`. If `fn` throws (the process.exit stub throws
// `__test_exit_<code>` to escape the call site without killing the test
// runner), `expectExit` propagates the throw through `expect(...).toThrow`
// — the captured streams are returned either way.
function withMockedConsole(fn: () => void): { stdout: string; stderr: string; threw: unknown } {
const originalLog = console.log;
const originalError = console.error;
const originalExit = process.exit;
let stdout = "";
let stderr = "";
let threw: unknown = undefined;
console.log = ((...args: unknown[]) => {
stdout += args.join(" ") + "\n";
}) as typeof console.log;
console.error = ((...args: unknown[]) => {
stderr += args.join(" ") + "\n";
}) as typeof console.error;
process.exit = ((code?: number) => {
throw new Error(`__test_exit_${code ?? 0}`);
}) as typeof process.exit;
try {
fn();
} catch (err) {
threw = err;
} finally {
console.log = originalLog;
console.error = originalError;
process.exit = originalExit;
}
return { stdout, stderr, threw };
}
function captureConsole(fn: () => void): CapturedConsole {
const { stdout, stderr, threw } = withMockedConsole(fn);
if (threw !== undefined) throw threw as Error;
return { stdout, stderr };
}
function expectExit(code: number, fn: () => void): CapturedConsole {
const { stdout, stderr, threw } = withMockedConsole(fn);
expect(threw).toBeInstanceOf(Error);
expect((threw as Error).message).toBe(`__test_exit_${code}`);
return { stdout, stderr };
}
describe("enable / disable CLI", () => {
it("enable writes flag=true globally by default", () => {
const out = captureConsole(() => enable(["my-feature-flag"], TEST_REGISTRY));
expect(out.stdout).toContain('Enabled flag "my-feature-flag" (global)');
expect(readFlags("global")).toEqual({ "my-feature-flag": true });
expect(readFlags("project")).toEqual({});
});
it("disable removes the flag entry from the chosen scope", () => {
setFlag("my-feature-flag", true, "global");
const out = captureConsole(() => disable(["my-feature-flag"]));
expect(out.stdout).toContain('Disabled flag "my-feature-flag" (global)');
expect(readFlags("global")).toEqual({});
});
it("disable on an unset flag is a no-op (still succeeds)", () => {
const out = captureConsole(() => disable(["never-set"]));
expect(out.stdout).toContain('Disabled flag "never-set" (global)');
expect(readFlags("global")).toEqual({});
});
it("disable persists an explicit false for an opt-out flag (not a no-op unset)", () => {
// An opt-out flag is on by default, so `disable` must write `false` — an
// unset would just revert it to its on default.
const out = captureConsole(() => disable(["opt-out-flag"], TEST_REGISTRY));
expect(out.stdout).toContain('Disabled flag "opt-out-flag" (global)');
expect(readFlags("global")).toEqual({ "opt-out-flag": false });
expect(isFlagEnabled("opt-out-flag")).toBe(false);
});
it("enable then disable on an opt-out flag round-trips true → false", () => {
captureConsole(() => enable(["opt-out-flag"], TEST_REGISTRY));
expect(readFlags("global")).toEqual({ "opt-out-flag": true });
captureConsole(() => disable(["opt-out-flag"], TEST_REGISTRY));
expect(readFlags("global")).toEqual({ "opt-out-flag": false });
});
it("the shipped `argent disable video-watermark` writes false", () => {
// Uses the production FLAG_REGISTRY (no injected registry) — the real UX.
const out = captureConsole(() => disable(["video-watermark"]));
expect(out.stdout).toContain('Disabled flag "video-watermark" (global)');
expect(readFlags("global")).toEqual({ "video-watermark": false });
});
it("--scope project targets project storage", () => {
captureConsole(() => enable(["proj-flag", "--scope", "project"], TEST_REGISTRY));
expect(readFlags("project")).toEqual({ "proj-flag": true });
expect(readFlags("global")).toEqual({});
});
it("supports --scope=project syntax", () => {
captureConsole(() => enable(["proj-flag", "--scope=project"], TEST_REGISTRY));
expect(readFlags("project")).toEqual({ "proj-flag": true });
});
it("supports --scope=global syntax (explicit)", () => {
captureConsole(() => enable(["g", "--scope=global"], TEST_REGISTRY));
expect(readFlags("global")).toEqual({ g: true });
});
it("-- ends flag parsing and treats the next token as the flag name", () => {
captureConsole(() => enable(["--", "after-dashdash"], TEST_REGISTRY));
expect(readFlags("global")).toEqual({ "after-dashdash": true });
});
it("-- with no following positional still surfaces the usage error", () => {
const out = expectExit(2, () => enable(["--"]));
expect(out.stderr).toContain("Usage: argent enable <flag-name>");
});
it("rejects an invalid scope value", () => {
const out = expectExit(2, () => enable(["x", "--scope", "everywhere"]));
expect(out.stderr).toContain('--scope must be "project" or "global"');
});
it("requires a flag name", () => {
const out = expectExit(2, () => enable([]));
expect(out.stderr).toContain("Usage: argent enable <flag-name>");
});
it("rejects invalid flag names", () => {
const out = expectExit(2, () => enable(["1-bad"]));
expect(out.stderr).toContain('Invalid flag name "1-bad"');
});
it("rejects unknown flags", () => {
const out = expectExit(2, () => enable(["good", "--bogus"]));
expect(out.stderr).toContain("Unknown flag: --bogus");
});
it("rejects extra positional arguments", () => {
const out = expectExit(2, () => enable(["a", "b"]));
expect(out.stderr).toContain('Unexpected extra argument: "b"');
});
it("--help prints usage without writing anything", () => {
const out = captureConsole(() => enable(["--help"]));
expect(out.stdout).toContain("Usage: argent enable");
expect(readFlags("global")).toEqual({});
});
it("--help lists every available flag with its description", () => {
const out = captureConsole(() => enable(["--help"], TEST_REGISTRY));
expect(out.stdout).toContain("Available flags:");
expect(out.stdout).toContain("my-feature-flag");
expect(out.stdout).toContain("Primary test flag.");
expect(out.stdout).toContain("proj-flag");
expect(out.stdout).toContain("Project-scoped test flag.");
// -h is an alias for --help and shows the same listing.
const short = captureConsole(() => disable(["-h"], TEST_REGISTRY));
expect(short.stdout).toContain("Available flags:");
expect(short.stdout).toContain("Listing flag A.");
expect(readFlags("global")).toEqual({});
});
it("enable → disable round trip leaves a clean tree (no stub entry)", () => {
captureConsole(() => enable(["x"], TEST_REGISTRY));
captureConsole(() => disable(["x"]));
expect(readFlags("global")).toEqual({});
expect(fs.existsSync(getFlagsPath("global"))).toBe(false);
expect(isFlagEnabled("x")).toBe(false);
});
it("rejects enabling a flag that is not in the registry", () => {
const out = expectExit(2, () => enable(["not-registered"], TEST_REGISTRY));
expect(out.stderr).toContain('Unknown feature flag "not-registered"');
expect(out.stderr).toContain("argent flags");
expect(readFlags("global")).toEqual({});
});
it("rejects enabling the legacy auto-shutdown flag name (never wired)", () => {
// ARGENT_AUTO_SHUTDOWN stayed an env var (renamed), so `auto-shutdown` was
// never added to the registry — `argent enable auto-shutdown` must fail.
const out = expectExit(2, () => enable(["auto-shutdown"]));
expect(out.stderr).toContain("Unknown feature flag");
expect(readFlags("global")).toEqual({});
});
it("disable stays lenient — clears a stored flag that is not in the registry", () => {
// Simulates a flag that was enabled once and later removed from the registry.
setFlag("legacy", true, "global");
const out = captureConsole(() => disable(["legacy"]));
expect(out.stdout).toContain('Disabled flag "legacy" (global)');
expect(readFlags("global")).toEqual({});
});
});
describe("flags (list) CLI", () => {
it("ships the argent-lens flag in the production registry", () => {
// Guards the gate: setup-registry.ts reads isFlagEnabled("argent-lens"),
// so that exact name must stay registered (and discoverable via `argent flags`).
const out = captureConsole(() => flagsCmd([]));
expect(out.stdout).toContain("argent-lens");
expect(out.stdout).not.toContain("No feature flags are defined.");
});
it("lists every registry flag with its description and effective scope", () => {
setFlag("a", true, "global");
setFlag("b", true, "global");
setFlag("b", false, "project");
const out = captureConsole(() => flagsCmd([], TEST_REGISTRY));
expect(out.stdout).toContain("Feature flags");
// value + winning scope
expect(out.stdout).toMatch(/a\s+enabled\s+\(global\)/);
expect(out.stdout).toMatch(/b\s+disabled\s+\(project\)/);
// descriptions are shown
expect(out.stdout).toContain("Listing flag A.");
expect(out.stdout).toContain("Listing flag B.");
});
it("shows unset registry flags as disabled (with no scope)", () => {
// Nothing stored; every registry flag should still be listed as disabled.
const out = captureConsole(() => flagsCmd([], TEST_REGISTRY));
expect(out.stdout).toMatch(/my-feature-flag\s+disabled/);
expect(out.stdout).toContain("Primary test flag.");
expect(out.stdout).not.toContain("(global)");
expect(out.stdout).not.toContain("(project)");
});
it("--json emits the raw scopes plus the registry view with descriptions", () => {
setFlag("a", true, "global");
setFlag("a", false, "project");
const out = captureConsole(() => flagsCmd(["--json"], TEST_REGISTRY));
const parsed = JSON.parse(out.stdout);
// raw scope maps preserved for back-compat
expect(parsed.global).toEqual({ a: true });
expect(parsed.project).toEqual({ a: false });
expect(parsed.effective).toEqual({ a: { value: false, scope: "project" } });
expect(parsed.paths.global).toBe(getFlagsPath("global"));
expect(parsed.paths.project).toBe(getFlagsPath("project"));
// registry view carries description + resolved state
const a = parsed.flags.find((f: { name: string }) => f.name === "a");
expect(a).toEqual({
name: "a",
description: "Listing flag A.",
enabled: false,
scope: "project",
});
const unset = parsed.flags.find((f: { name: string }) => f.name === "my-feature-flag");
expect(unset).toMatchObject({ enabled: false, scope: null });
});
it("--help prints usage", () => {
const out = captureConsole(() => flagsCmd(["--help"]));
expect(out.stdout).toContain("Usage: argent flags");
});
it("--help lists every available flag with its description", () => {
const out = captureConsole(() => flagsCmd(["--help"], TEST_REGISTRY));
expect(out.stdout).toContain("Available flags:");
expect(out.stdout).toContain("my-feature-flag");
expect(out.stdout).toContain("Primary test flag.");
expect(out.stdout).toContain("Listing flag B.");
});
});
describe("deprecating a flag (removed from FLAG_REGISTRY)", () => {
// A flag that was enabled before being deprecated still lives in flags.json
// but is absent from the registry. Loading/reading it must never throw.
it("`argent flags` lists a deprecated stored flag under 'unrecognized' (no throw)", () => {
setFlag("deprecated-flag", true, "global");
let out!: CapturedConsole;
expect(() => {
out = captureConsole(() => flagsCmd([], TEST_REGISTRY));
}).not.toThrow();
expect(out.stdout).toContain("Stored but no longer recognized");
expect(out.stdout).toMatch(/deprecated-flag\s+enabled\s+\(global\)/);
});
it("--json surfaces it under `unrecognized` and keeps it out of the registry view", () => {
setFlag("deprecated-flag", true, "global");
const out = captureConsole(() => flagsCmd(["--json"], TEST_REGISTRY));
const parsed = JSON.parse(out.stdout);
expect(parsed.unrecognized).toEqual([
{ name: "deprecated-flag", enabled: true, scope: "global" },
]);
expect(parsed.global).toEqual({ "deprecated-flag": true });
expect(
parsed.flags.find((f: { name: string }) => f.name === "deprecated-flag")
).toBeUndefined();
});
it("an empty registry with stored flags still loads cleanly (nothing crashes or is hidden)", () => {
setFlag("legacy-a", true, "global");
setFlag("legacy-b", false, "project");
const out = captureConsole(() => flagsCmd([], []));
expect(out.stdout).toContain("No feature flags are defined.");
expect(out.stdout).toContain("Stored but no longer recognized");
expect(out.stdout).toMatch(/legacy-a\s+enabled\s+\(global\)/);
expect(out.stdout).toMatch(/legacy-b\s+disabled\s+\(project\)/);
});
it("disable cleans up a deprecated flag and it leaves the listing", () => {
setFlag("deprecated-flag", true, "global");
captureConsole(() => disable(["deprecated-flag"])); // lenient — no registry needed
expect(readFlags("global")).toEqual({});
const out = captureConsole(() => flagsCmd([], TEST_REGISTRY));
expect(out.stdout).not.toContain("deprecated-flag");
});
it("a deprecated flag with a prototype-style name still loads without error", () => {
setFlag("toString", true, "global"); // valid per FLAG_NAME_RE, also on Object.prototype
expect(() => captureConsole(() => flagsCmd([], TEST_REGISTRY))).not.toThrow();
expect(isFlagEnabled("toString")).toBe(true);
const out = captureConsole(() => flagsCmd(["--json"], TEST_REGISTRY));
const parsed = JSON.parse(out.stdout);
expect(parsed.unrecognized).toContainEqual({
name: "toString",
enabled: true,
scope: "global",
});
});
});