Files
callstack__agent-device/test/integration/smoke-daemon-http.test.ts
Michał Pierzchała 5e48486ad8 test: catch daemon lifecycle and durable-state leaks (#1781 B1) (#1859)
* test: daemon leak oracle around the real-subprocess daemon lanes (#1781 B1)

Adds test/integration/support/daemon-leak-oracle.ts and calls it at the end of
smoke-daemon-clean, smoke-daemon-http and daemon-replace-exit-flush. After
shutdown the oracle asserts that no daemon-owned process survives (ownership:
PPID descendant, PGID = daemon pid, AGENT_DEVICE_STATE_DIR env, state-dir
argv — never global counts) and that the isolated state dir holds only
classified artifacts (no *.tmp, no daemon.json/lock without a live daemon, no
open capture descriptor).

Red-proofs: pre-fix #1324 (a84caa818) leaves simctl recordVideo with ppid 1 /
pgid = dead daemon; pre-fix #1109 (be4bd092b) leaves the agent-browser daemon
plus its Chrome fleet. Both are clean on main.

Refs #1781 #1431

* test: fail the daemon leak oracle on a surviving daemon and pin its rules

Review of #1859 found the oracle reported "clean" when the daemon itself
outlived shutdown: daemon pids were excluded from ownership, liveDaemonPids
never reached hasDaemonLeaks, and a live daemon even flipped daemon.json/lock
from stray to expected. stopProcessForTakeover is best-effort void, so only
smoke-daemon-clean independently asserted death.

- a live daemon pid at phase 'after-shutdown' is now itself a leak, and its
  metadata files stay stray; a live daemon remains legitimate at 'after-close'
- split the pure ownership/residue rules into daemon-leak-model.ts and pin them
  with daemon-leak-model.test.ts, using the real ps rows captured during the
  #1109 and #1324 red-proofs (the lanes' daemons own no children, so the fixture
  test is what guards those shapes in CI)
- exempt the managed tools/ install tree before the .tmp rule, so agent-browser's
  own download temporaries are no longer a false LEAK
- report empty directories as residue (an unswept session scaffold leaves no file)
- reuse src/utils/host-process.ts (expandProcessTree, uniquePositivePids) and its
  /bin/ps convention instead of re-deriving them
- assert in smoke-daemon-http on the success path, not in finally, so the settle
  window cannot replace a primary assertion's diagnostic

Refs #1781 #1431

* test: make the leak oracle's after-close phase name the session that closed

Re-review of #1859: `after-close` accepted every capture descriptor and legacy
marker, so the closed session's unfinalized handle was indistinguishable from
another session's legitimately live one — the phase could not fail, the same
shape as the surviving-daemon blocker, and it left half of B1's stated scope
undelivered.

- the observation carries the session directories closed at the checkpoint; a
  capture handle must be finalized once its owning session is gone (every
  session after shutdown, only the closed ones after close), while another
  session's live handle stays expected and a legacy pid marker is never a
  finish record
- the oracle accepts `closedSessions` and a `sessionsDir` override, normalizing
  entries to the canonical `sessions/<name>/…` shape so an in-process harness
  rooted directly at its own sessions dir is classified the same way
- add a real route regression: a provider-backed session with a live screen
  recording is closed through the daemon route, and the oracle refuses a
  descriptor left `lifecycle: "open"`. Reverting the close-route finalization
  (session-close-lifecycle-teardown.ts, the #1325 fix) turns it red naming
  sessions/default/screen-recording.resource.json; with the pre-fix model it
  stays green, which is the P1 in one line

Refs #1781 #1431

* test: require the closed-session identity at the leak oracle's owning interface

Re-review of #1859: `closedSessions` was optional and defaulted to empty, so any
caller — or the standalone CLI invoked with `--phase after-close` and no
`--closed-session` — silently restored the vacuous checkpoint that accepts every
unfinalized capture handle and reports clean.

- phase and the identity that phase needs are now one discriminated shape:
  { phase: 'after-close'; closedSessions: NonEmpty<string> } | { phase:
  'after-shutdown' }, so the empty case is not expressible and 'after-shutdown'
  is unchanged
- the CLI refuses the same invocation with a typed INVALID_ARGS error and a hint
  naming what the missing identity would have cost, instead of degrading
- daemon-leak-oracle-cli.test.ts drives the real CLI: the unnamed after-close
  invocation must fail without reporting, the named one finds the planted
  unfinalized handle, and after-shutdown is unaffected. Reverting the guard makes
  it print 'clean (after-close)' over that same handle and turns the test red

Refs #1781 #1431

* test: split the leak oracle's unrecorded process arm out of the lane path

Maintainer asked whether the harness can be trimmed. The two arms differ in kind:
the daemon already records captures and state-dir artifacts, so those rules are
short and fire on every lane run; nothing records what a daemon spawned, so the
process arm reconstructs ownership from the OS four ways — and the three CLI
daemon lanes are device-free, so it only ever compared an empty set to an empty
set.

Move that arm to test/integration/support/daemon-owned-process-probe.ts, the
manual script that produced the #1109/#1324 evidence, and keep its rules pinned
by a fixture test (no lane can run the probe, so its fixtures are the only CI
guard on those shapes). The shipped oracle keeps every guarantee it had:
surviving daemon at after-shutdown, the closed-session finalization rule, the
state-dir allowlist, the CLI refusal, and the phase-discriminated types.

Lane-path harness 852 → 548 LOC (-36%); repo total roughly flat, because the
ownership reconstruction can be relocated but not deleted. That is the argument
for the follow-up: once the daemon records owned child pids the way it records
captures, the arm collapses to "read the record, assert they are dead" and moves
back into the oracle.

Re-proved after the move: #1324 against pre-fix a84caa818 on an iPhone 16
simulator still goes red through the probe (simctl recordVideo, ppid 1,
pgid = the dead daemon, 0-byte mp4), and clean once the orphan is reaped.

Refs #1781 #1431

* test: guard the daemon-owned-process arm in CI on the live web lane

Re-review of ce9f0eeb: moving that arm to a manual probe left the two failures
B1 exists to prevent detectable only by hand, and a fixture test proves regexes
recognize synthetic rows, not that the shipped route reaps what it spawned.

Restore the arm to the shipped oracle (the three files return byte-identical to
3a5b9bef3) and give it a lane that can execute it: smoke-web-platform is the one
CI route whose daemon owns real children — the managed agent-browser daemon and
its Chrome fleet. After the normal smoke it reopens a session, stops the daemon
with that session still open (the #1109 shape: an ordinary close reaps the fleet,
so only an unclosed session can strand it), and requires that nothing owned
outlives the browser idle window.

Proven both ways locally: green in 59s, and red when the fleet is stranded
(browser idle window raised past the settle budget) with the oracle naming 15
owned processes — the #1109 signature, in a lane that runs on every PR.

Also: daemon-replace-exit-flush cleared `info` before the oracle ran, so a failed
stop would skip the `finally` retry and remove the state dir while the daemon was
still alive. Clear it only once the checkpoint passes.

Refs #1781 #1431 #1882

* docs: align daemon leak coverage rationale

* refactor(test): narrow daemon oracle to durable state leaks
2026-08-20 10:56:45 +02:00

122 lines
4.0 KiB
TypeScript

import test from 'node:test';
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { DAEMON_RPC_PROTOCOL_VERSION } from '../../src/daemon/http-health.ts';
import { skipWhenLoopbackUnavailable } from '../../src/__tests__/test-utils/loopback.ts';
import { stopProcessForTakeover } from '../../src/daemon/daemon-process.ts';
import { formatResultDebug } from './cli-json.ts';
import { assertNoDaemonLeaks } from './support/daemon-leak-oracle.ts';
import { runCliJson } from './test-helpers.ts';
type DaemonInfo = {
token: string;
pid: number;
processStartTime?: string;
transport?: string;
httpPort?: number;
};
test('daemon HTTP transport starts from CLI and accepts a command RPC', async (t) => {
if (await skipWhenLoopbackUnavailable(t)) {
return;
}
const stateDir = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-http-smoke-'));
try {
const args = [
'session',
'list',
'--json',
'--daemon-transport',
'http',
'--state-dir',
stateDir,
];
const run = () =>
runCliJson(args, {
env: {
...process.env,
AGENT_DEVICE_DAEMON_SERVER_MODE: 'http',
},
});
const cli = run();
assert.equal(cli.status, 0, formatResultDebug('start HTTP daemon', ['session', 'list'], cli));
assert.equal(cli.json?.success, true, JSON.stringify(cli.json));
const info = readDaemonInfo(stateDir);
assert.equal(info.transport, 'http');
assert.equal(typeof info.httpPort, 'number');
assert.ok((info.httpPort ?? 0) > 0);
const health = await fetch(`http://127.0.0.1:${info.httpPort}/health`);
assert.equal(health.status, 200);
const healthPayload = (await health.json()) as Record<string, unknown>;
assert.equal(healthPayload.ok, true);
assert.equal(healthPayload.service, 'agent-device-daemon');
assert.equal(typeof healthPayload.version, 'string');
assert.equal(healthPayload.rpcProtocolVersion, DAEMON_RPC_PROTOCOL_VERSION);
const rpc = await callCommandRpc(info, 'session_list');
assert.equal(rpc.status, 200);
assert.equal(rpc.body.result?.ok, true, JSON.stringify(rpc.body));
const unauthorized = await callCommandRpc({ ...info, token: 'wrong-token' }, 'session_list');
assert.equal(unauthorized.status, 401);
assert.equal(unauthorized.body.error?.data?.code, 'UNAUTHORIZED');
// #1781 B1: the HTTP-mode daemon must exit — leaving nothing it owns and no
// unclassified state-dir residue. Asserted on the success path so the
// oracle's settle window can never replace a primary assertion's
// diagnostic; the `finally` below stays best-effort cleanup.
await stopDaemon(info);
await assertNoDaemonLeaks({ stateDir, daemonPids: [info.pid], phase: 'after-shutdown' });
} finally {
if (fs.existsSync(path.join(stateDir, 'daemon.json'))) {
await stopDaemon(readDaemonInfo(stateDir));
}
fs.rmSync(stateDir, { recursive: true, force: true });
}
});
function readDaemonInfo(stateDir: string): DaemonInfo {
const infoPath = path.join(stateDir, 'daemon.json');
return JSON.parse(fs.readFileSync(infoPath, 'utf8')) as DaemonInfo;
}
async function callCommandRpc(
info: DaemonInfo,
command: string,
): Promise<{ status: number; body: any }> {
const response = await fetch(`http://127.0.0.1:${info.httpPort}/rpc`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({
jsonrpc: '2.0',
id: `rpc-${Date.now()}`,
method: 'agent_device.command',
params: {
token: info.token,
session: 'default',
command,
positionals: [],
flags: {},
},
}),
});
return {
status: response.status,
body: await response.json(),
};
}
async function stopDaemon(info: DaemonInfo): Promise<void> {
if (!Number.isInteger(info.pid) || info.pid <= 0) return;
await stopProcessForTakeover(info.pid, {
termTimeoutMs: 1500,
killTimeoutMs: 1500,
expectedStartTime: info.processStartTime,
});
}