Files
Ignacy Łątka 2251c7cf46 feat(screen-recording): screen-recording-start/stop tools with per-call stop reminders (#517)
## What

Adds a `screen-recording-start` / `screen-recording-stop` tool pair
(naming follows `native-profiler-start/stop`) that records the device
screen to an mp4 and returns it as a downloadable artifact.

Capture runs through **simulator-server** — the same backend
`screenshot` and every interaction tool already drive. simulator-server
publishes the device screen as an MJPEG stream (`/stream.mjpeg`), so
recording needs no platform-specific recorder: the tool-server
subscribes to that stream, paces the frames onto a fixed 30 fps
timeline, and pipes them into a single ffmpeg process that encodes — and
watermarks — straight to the final mp4. The stream is multi-subscriber,
so a preview window can stay open on the same device while recording.

- **One path for every device simulator-server drives** — iOS
simulators, Android emulators, and physical Android devices.
- **Native resolution, constant 30 fps** — h264, `+faststart`, no
post-processing pass.
- Session state lives in a per-device `ScreenRecordingSession` blueprint
(mirrors the native-profiler session): time-limit cap with
auto-finalize, unexpected-exit recovery (a capture that ended on its own
is still retrievable via stop), always-startable state reset on stop,
and dispose-on-shutdown cleanup through the registry.

## Why the frames are paced host-side

A device only emits a frame when the screen **changes**. Pointing ffmpeg
straight at the stream therefore produces a video whose length is the
sum of the moments something moved — a still screen collapses to a
fraction of a second — and an ffmpeg blocked on a silent socket does not
answer a stop signal promptly either (measured: 12 s late). Re-emitting
the last frame on a wall-clock schedule fixes both: **video duration
always equals real elapsed time**, and stop is a stdin EOF that
finalizes in ~60 ms. Duplicate frames cost almost nothing once encoded
(a 10 s recording of a static screen is ~1 MB).

## Finishing pass folded into capture

The Argent watermark (logo + `By @swmansion`, bottom-left, sized
relative to the frame) is stamped by the same ffmpeg invocation that
encodes the capture, so there is no second encode and no temp
intermediate. Its colour is chosen **per pixel** from the background
luma underneath it — the white logo over dark areas, a runtime-derived
near-black copy over light areas — so it keeps maximum contrast against
whatever is behind it. It sits at 20 % opacity and is opt-out via
`argent disable video-watermark` (read when the recording starts). Frame
geometry comes from the first JPEG's SOF header, so no `ffprobe` is
needed.

ffmpeg is now the recorder rather than an optional finisher, so a
missing ffmpeg fails `screen-recording-start` up front with an install
hint instead of silently degrading the output at stop time.

## Reminder plumbing

While a recording is running — or ended on its own but was never
retrieved — **every** tool result carries a `note` telling the agent the
recording is still going and how to stop it. It rides the same `note`
field as the update reminder but is per-call and unsuppressed; both
notes compose when they coincide. The note text switches to "already
ended, retrieve the file" once the cap fires or the capture dies.

## Skill

New `argent-screen-recording` skill (+ routing stanza in
`rules/argent.md`): documents the start → interact → stop workflow and
instructs the agent to schedule its own reminder/wakeup for the expected
end of the recording immediately after starting one (harness reminder
tool if available, otherwise a background `sleep` whose completion
notification pulls it back). Scores 10/10 on `scripts/grade-skills.mjs`.

## Verification

- Unit: 45 tests across `screen-recording.test.ts`,
`mjpeg-stream.test.ts`, `watermark.test.ts`,
`http-recording-note.test.ts`; full tool-server suite **2808 tests / 280
files green**; `tsc --noEmit -p tsconfig.test.json`, root `tsc --build`,
eslint, prettier all clean.
- E2E on a live tool-server (`POST /tools/...`), driving real
interactions between start and stop:
- **iOS sim** (iPhone 16 Pro Max, iOS 18.5): 1320×2868 h264,
`r_frame_rate=30/1`, 338 frames, container duration 11.27 s vs reported
`durationMs` 11314 — **within 50 ms of wall clock**. Extracted frames
show real screen content (mid app-switcher animation) with the watermark
rendered light-over-dark.
- **Android emulator** (Pixel 3a, API 34): 1080×2220 h264, 30 fps, 353
frames, duration 11.77 s vs `durationMs` 11793 — within 30 ms. 512 KB
for a mostly-static home screen.
- **Static screen** (no interaction at all, 10 s): 300 frames, duration
exactly 10.000 s — the case the previous `recordVideo` implementation
reduced to ~0.03 s.
- **Long silence then activity** (15 s untouched, then an app launch):
stream survived, no warning, frames after the silence show the new
screen. This caught a real bug during review — the connect timeout was
also acting as a socket *inactivity* timeout and tore the stream down at
10 s of stillness; fixed and covered by a regression test that fails
without the fix.
- **Time-limit cap**: `timeLimitSeconds: 5` self-finalized at 4.97 s /
149 frames, the per-call note flipped to "already ended (it hit its 5s
time limit)", and a stop issued 11 s later still returned the video with
the matching warning.

## Not supported (capability-gated with clear errors)

tvOS simulators, physical iPhones, Chromium apps, Vega, and remote
(`remote:`-prefixed) simulators — none of them expose a readable frame
stream.

---------

Co-authored-by: Ignacy Łątka <claude-hubert.gancarczyk@swmansion.com>
2026-07-23 21:14:37 +02:00

352 lines
13 KiB
TypeScript

import { describe, it, expect, beforeEach, afterEach } from "vitest";
import * as fs from "node:fs";
import * as path from "node:path";
import * as os from "node:os";
import {
FLAG_REGISTRY,
findProjectRoot,
getFlagDefinition,
getFlagsPath,
isFlagEnabled,
isFeatureEnabled,
readFlags,
resolveProjectRoot,
setFlag,
unsetFlag,
type FlagDefinition,
} from "../src/flags.js";
// Hermetic registry for getFlagDefinition's injectable-registry path so the
// test never depends on which flags ship in the production FLAG_REGISTRY.
const TEST_REGISTRY: readonly FlagDefinition[] = [
{ name: "my-feature-flag", description: "Primary test flag." },
{ name: "opt-out-flag", description: "On by default.", defaultEnabled: true },
];
// All tests redirect global+project storage into tmp dirs by mutating
// process.env.HOME (consumed by os.homedir()) and process.cwd().
let tmpHome: string;
let tmpProject: string;
let originalHome: string | undefined;
let originalUserProfile: string | undefined;
let originalCwd: string;
function readJsonFile(filePath: string): unknown {
return JSON.parse(fs.readFileSync(filePath, "utf8"));
}
beforeEach(() => {
// realpath unwraps macOS's /var → /private/var tmpdir symlink so the path
// we hand back from getFlagsPath matches what process.cwd() reports after
// chdir().
tmpHome = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "argent-flags-home-")));
tmpProject = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "argent-flags-proj-")));
// Drop a marker so resolveProjectRoot stops here instead of walking up to
// the actual user's repo and writing into it.
fs.writeFileSync(path.join(tmpProject, "package.json"), "{}");
originalHome = process.env.HOME;
originalUserProfile = process.env.USERPROFILE;
originalCwd = process.cwd();
process.env.HOME = tmpHome;
process.env.USERPROFILE = tmpHome;
process.chdir(tmpProject);
});
afterEach(() => {
process.chdir(originalCwd);
if (originalHome === undefined) delete process.env.HOME;
else process.env.HOME = originalHome;
if (originalUserProfile === undefined) delete process.env.USERPROFILE;
else process.env.USERPROFILE = originalUserProfile;
fs.rmSync(tmpHome, { recursive: true, force: true });
fs.rmSync(tmpProject, { recursive: true, force: true });
});
describe("getFlagsPath", () => {
it("defaults global path under ~/.argent/flags.json", () => {
expect(getFlagsPath("global")).toBe(path.join(tmpHome, ".argent", "flags.json"));
});
it("project path lives at <project-root>/.argent/flags.json", () => {
expect(getFlagsPath("project")).toBe(path.join(tmpProject, ".argent", "flags.json"));
});
it("respects explicit cwd / homeDir overrides", () => {
const altHome = fs.realpathSync(
fs.mkdtempSync(path.join(os.tmpdir(), "argent-flags-alt-home-"))
);
const altProj = fs.realpathSync(
fs.mkdtempSync(path.join(os.tmpdir(), "argent-flags-alt-proj-"))
);
fs.writeFileSync(path.join(altProj, "package.json"), "{}");
try {
expect(getFlagsPath("global", { homeDir: altHome })).toBe(
path.join(altHome, ".argent", "flags.json")
);
expect(getFlagsPath("project", { cwd: altProj })).toBe(
path.join(altProj, ".argent", "flags.json")
);
} finally {
fs.rmSync(altHome, { recursive: true, force: true });
fs.rmSync(altProj, { recursive: true, force: true });
}
});
});
describe("resolveProjectRoot", () => {
it("walks up to the nearest marker", () => {
const nested = path.join(tmpProject, "a", "b", "c");
fs.mkdirSync(nested, { recursive: true });
expect(resolveProjectRoot(nested)).toBe(tmpProject);
});
it("returns startDir when no marker exists in ancestry", () => {
// A bare tmpdir guaranteed to have no project markers between it and /
const bare = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "argent-flags-noroot-")));
try {
expect(resolveProjectRoot(bare)).toBe(bare);
} finally {
fs.rmSync(bare, { recursive: true, force: true });
}
});
it("treats existing .argent as a marker", () => {
fs.rmSync(path.join(tmpProject, "package.json"));
const argentDir = path.join(tmpProject, ".argent");
fs.mkdirSync(argentDir);
const nested = path.join(tmpProject, "sub");
fs.mkdirSync(nested);
expect(resolveProjectRoot(nested)).toBe(tmpProject);
});
it("treats existing .git as a marker", () => {
fs.rmSync(path.join(tmpProject, "package.json"));
fs.mkdirSync(path.join(tmpProject, ".git"));
const nested = path.join(tmpProject, "sub");
fs.mkdirSync(nested);
expect(resolveProjectRoot(nested)).toBe(tmpProject);
});
});
describe("findProjectRoot", () => {
it("returns the marker directory when one exists", () => {
const nested = path.join(tmpProject, "a", "b");
fs.mkdirSync(nested, { recursive: true });
expect(findProjectRoot(nested)).toBe(tmpProject);
});
it("returns null when no marker exists in ancestry (unlike resolveProjectRoot)", () => {
const bare = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "argent-flags-noroot-")));
try {
expect(findProjectRoot(bare)).toBeNull();
expect(resolveProjectRoot(bare)).toBe(bare);
} finally {
fs.rmSync(bare, { recursive: true, force: true });
}
});
});
describe("setFlag / unsetFlag / readFlags", () => {
it("writes the flag to disk and reads it back", () => {
setFlag("alpha", true, "global");
expect(readFlags("global")).toEqual({ alpha: true });
const file = readJsonFile(getFlagsPath("global"));
expect(file).toEqual({ flags: { alpha: true } });
});
it("preserves other flags when setting one", () => {
setFlag("alpha", true, "global");
setFlag("beta", false, "global");
expect(readFlags("global")).toEqual({ alpha: true, beta: false });
});
it("overwrites a flag with a new value", () => {
setFlag("alpha", true, "global");
setFlag("alpha", false, "global");
expect(readFlags("global")).toEqual({ alpha: false });
});
it("unsetFlag removes the entry and reports whether it existed", () => {
setFlag("alpha", true, "global");
setFlag("beta", true, "global");
expect(unsetFlag("alpha", "global")).toBe(true);
expect(readFlags("global")).toEqual({ beta: true });
expect(unsetFlag("missing", "global")).toBe(false);
});
it("removes the file (and empty .argent dir) when the last flag is unset", () => {
setFlag("alpha", true, "global");
unsetFlag("alpha", "global");
expect(fs.existsSync(getFlagsPath("global"))).toBe(false);
expect(fs.existsSync(path.join(tmpHome, ".argent"))).toBe(false);
});
it("keeps the .argent dir when sibling state lives next to flags.json", () => {
setFlag("alpha", true, "global");
const sibling = path.join(tmpHome, ".argent", "tool-server.json");
fs.writeFileSync(sibling, "{}");
unsetFlag("alpha", "global");
expect(fs.existsSync(getFlagsPath("global"))).toBe(false);
expect(fs.existsSync(sibling)).toBe(true);
expect(fs.existsSync(path.join(tmpHome, ".argent"))).toBe(true);
});
it("project + global live in separate files", () => {
setFlag("alpha", true, "global");
setFlag("alpha", false, "project");
expect(readFlags("global")).toEqual({ alpha: true });
expect(readFlags("project")).toEqual({ alpha: false });
});
it("recovers from malformed JSON by treating storage as empty", () => {
const file = getFlagsPath("global");
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, "not json");
expect(readFlags("global")).toEqual({});
setFlag("alpha", true, "global");
expect(readFlags("global")).toEqual({ alpha: true });
});
it("ignores non-boolean values in the stored flags object", () => {
const file = getFlagsPath("global");
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(
file,
JSON.stringify({ flags: { real: true, bogus: "yes", numeric: 1, nested: {} } })
);
expect(readFlags("global")).toEqual({ real: true });
});
it("treats missing .argent dir as empty without throwing", () => {
expect(readFlags("global")).toEqual({});
expect(readFlags("project")).toEqual({});
});
it("writes are atomic — no .tmp leftover in the .argent dir", () => {
setFlag("alpha", true, "global");
setFlag("beta", true, "global");
const dir = path.join(tmpHome, ".argent");
const leftovers = fs.readdirSync(dir).filter((f) => f.includes(".tmp"));
expect(leftovers).toEqual([]);
});
});
describe("isFlagEnabled", () => {
it("returns false for an unknown flag", () => {
expect(isFlagEnabled("unknown")).toBe(false);
});
it("project value overrides global value (explicit false masks global true)", () => {
setFlag("alpha", true, "global");
setFlag("alpha", false, "project");
expect(isFlagEnabled("alpha")).toBe(false);
setFlag("beta", false, "global");
setFlag("beta", true, "project");
expect(isFlagEnabled("beta")).toBe(true);
});
it("falls through to global when project does not set it", () => {
setFlag("alpha", true, "global");
expect(isFlagEnabled("alpha")).toBe(true);
});
it("falls through to project-only when global is unset", () => {
setFlag("alpha", true, "project");
expect(isFlagEnabled("alpha")).toBe(true);
});
it("respects explicit cwd / homeDir options instead of process state", () => {
setFlag("alpha", true, "global", { homeDir: tmpHome });
expect(isFlagEnabled("alpha", { homeDir: tmpHome, cwd: tmpProject })).toBe(true);
});
});
describe("prototype-named flags (Object.prototype keys)", () => {
// Names like "toString"/"constructor"/"valueOf" also exist on
// Object.prototype. A naive `name in obj` check would treat these as set
// (returning a truthy prototype member) even when storage is empty — these
// guard that hasOwn semantics are used throughout.
const protoNames = ["toString", "constructor", "valueOf", "hasOwnProperty"];
for (const name of protoNames) {
it(`isFlagEnabled("${name}") is false (and a real boolean) when unset`, () => {
const result = isFlagEnabled(name);
expect(result).toBe(false);
expect(typeof result).toBe("boolean");
});
it(`unsetFlag("${name}") on storage without it returns false and is a no-op`, () => {
setFlag("real", true, "global");
expect(unsetFlag(name, "global")).toBe(false);
expect(readFlags("global")).toEqual({ real: true });
});
it(`set/unset round trip works for a flag literally named "${name}"`, () => {
setFlag(name, true, "global");
expect(readFlags("global")).toEqual({ [name]: true });
expect(isFlagEnabled(name)).toBe(true);
expect(unsetFlag(name, "global")).toBe(true);
expect(isFlagEnabled(name)).toBe(false);
});
}
});
describe("FLAG_REGISTRY / getFlagDefinition", () => {
it("getFlagDefinition returns the entry or undefined", () => {
expect(getFlagDefinition("my-feature-flag", TEST_REGISTRY)?.description).toBe(
"Primary test flag."
);
expect(getFlagDefinition("nope", TEST_REGISTRY)).toBeUndefined();
});
it("getFlagDefinition defaults to the shipped registry", () => {
expect(getFlagDefinition("disable-auto-screenshot")?.description).toMatch(/auto/i);
expect(getFlagDefinition("tool-server-event-log")?.description).toMatch(/event/i);
});
it("every shipped registry entry has a non-empty name and description", () => {
// Guards against a half-filled entry being added to the production registry.
for (const def of FLAG_REGISTRY) {
expect(def.name).toMatch(/^[a-zA-Z][a-zA-Z0-9._-]*$/);
expect(def.description.trim().length).toBeGreaterThan(0);
}
});
});
describe("opt-out (default-on) flags", () => {
it("isFlagEnabled honors a caller-supplied default for an unset flag", () => {
expect(isFlagEnabled("opt-out-flag")).toBe(false); // storage-only default is off
expect(isFlagEnabled("opt-out-flag", { default: true })).toBe(true);
});
it("isFeatureEnabled reads an unset opt-out flag as ON via its declared default", () => {
expect(isFeatureEnabled("opt-out-flag", {}, TEST_REGISTRY)).toBe(true);
// an opt-in flag (no defaultEnabled) still reads off when unset
expect(isFeatureEnabled("my-feature-flag", {}, TEST_REGISTRY)).toBe(false);
});
it("isFeatureEnabled reads an explicit false as OFF (the disable case)", () => {
setFlag("opt-out-flag", false, "global");
expect(isFeatureEnabled("opt-out-flag", {}, TEST_REGISTRY)).toBe(false);
});
it("isFeatureEnabled reads an explicit true as ON", () => {
setFlag("opt-out-flag", true, "global");
expect(isFeatureEnabled("opt-out-flag", {}, TEST_REGISTRY)).toBe(true);
});
it("project scope overrides the ON default for an opt-out flag", () => {
setFlag("opt-out-flag", false, "project");
expect(isFeatureEnabled("opt-out-flag", {}, TEST_REGISTRY)).toBe(false);
});
it("the shipped video-watermark flag is declared opt-out (on by default)", () => {
expect(getFlagDefinition("video-watermark")?.defaultEnabled).toBe(true);
expect(isFeatureEnabled("video-watermark")).toBe(true); // unset ⇒ on
});
});