Files
callstack__agent-device/scripts/wire-compat/model.test.ts
Michał Pierzchała 05a1d76f2e test: add daemon RPC wire-surface compatibility gate (#1717)
* test: gate daemon RPC wire compatibility against the last released tag (#1432)

ADR 0006 fixes exactly when DAEMON_RPC_PROTOCOL_VERSION must be bumped, and
nothing checked that it was. The runtime guard (readRemoteDaemonHealth) refuses
a mismatched peer, but only fires when someone remembered the bump — a wire
change that skipped it left both sides advertising protocol 2 while parsing
different payloads, which is the failure ADR 0006 exists to prevent.

Local daemons cannot skew (isReusableDaemonInfo takes over on any package
version mismatch). Cross-machine is skewed by design — proxy, cloud/limrun, a
remote macOS host — and ADR 0006 explicitly rules package version out as the
compatibility gate there, so the one boundary where skew is intended was the
one boundary with no gate.

test/wire-compat/surface.ts declares the wire surface grouped by the ADR bullet
each group serves, quoting it, with an `uncovered` note where a bullet is only
partly digestible (the /health and /rpc literals inside http-server.ts stay
reviewer-owned: a moved route 404s at connect time rather than misparsing).
ledger.json records what each declaration hashes to, at which protocol version.

Two gates, split for the same reason the replay-compat corpus splits:
- unit-core holds the ledger to its source and prints the digest to paste;
- Released-Surface Compatibility reads the ledger at the last RELEASED tag and
  requires the drift since then to carry a bump or a compatibleChanges ack.

From one commit a bumped ledger and an unbumped one are both just an edited
file, so only a released baseline can tell them apart. Acks are keyed by the
digest they cover, so one "added an optional field" cannot launder later
changes. Digests ignore comments and formatting; the manifest's closure is
derived from the AST, so a field typed by an unlisted sibling fails rather than
sitting outside the gate.

CI cost: one added job (checkout + toolchain + two node scripts, ~1 min),
mirroring the existing full-history replay-compat job.

* test: close wire-surface overclaim and make the closure fail closed (#1432)

Addresses both review P1s on #1717.

P1 — the manifest materially overclaimed ADR 0006 coverage. It quoted all four
bullets while digesting only the payload TYPES, so the producer and consumer
seams could break a skewed peer without moving a listed digest. Now listed on
both sides of every boundary: JSON-RPC method sets and the projections that
turn each method's params into a DaemonRequest, createRpcError/sendJson/
writeRpcResponseEnvelope, resolveToken and the auth-hook types, upload
preflight/finalize/308 handlers and the resumable ticket shape, artifact route
and download/inventory framing, REST error mapping, and the client's own
payload builder, lease-method mapping, response parser and error projection.
57 -> 117 declarations.

What stays out is now named rather than implied: createDaemonHttpServer's
dispatch wiring and the /health and /rpc literals inside it. Everything it
dispatches WITH is digested individually, and a moved route 404s at connect
time rather than misparsing — the loud failure, not the silent one.

P1 — imported and re-exported payload shapes escaped the closure.
declarationHomes() scanned only the manifest's own files and the walk
continued silently when a name could not be placed, so a listed type could
gain foo?: ImportedShape from a new module and stay green. Resolution is now
explicit and fails closed: relative imports, workspace specifiers (through the
owning package's own exports map, so a re-pointed export cannot drop a type),
and facade re-export chains. Every referenced name must land on a listed
declaration, a waiver with a written reason, a declared external module, or the
TS/Node global set. Fixed two extractor blind spots the walk exposed: a
declaration's own generic parameters and `as const` were being reported as
references.

Planted-red proofs (wire-mutations.test.ts): 13 cases independently mutate
method naming, response serialization, response parsing, auth projection,
upload ticket shape, 308 framing, artifact framing, REST error mapping, and
progress framing, each asserting the digest moves; 3 probes prove the closure
really reaches across a package boundary, a facade re-export, and a plain
relative import. Mutations apply inside the declaration's own span — a
whole-file replace silently hit a sibling sharing the substring, which is how
the first draft of one case passed vacuously.

The largest waiver pair (InternalRequestOptions, CommandFlags) rests on ADR
0006's own additive rule: they reach the peer inside DaemonRequest's untyped
flags/input bags, and the decision says a new flag needs no bump. Digesting
them would fire the gate on every new CLI flag and train reviewers to
rubber-stamp acks.

* test: list the consumer half of the auxiliary HTTP boundaries (#1432)

Addresses the remaining review P1 on #1717. The manifest claimed both sides of
response/upload/artifact framing while listing nothing from upload-client.ts,
daemon-artifacts.ts, or the health consumer in daemon-client-transport.ts, so
those parsers could narrow without moving a listed digest or protocol 2.

Now listed (117 -> 141 declarations):

- /health consumer: RemoteDaemonHealth, readHealthPayload, readDaemonHttpHealth,
  readRemoteDaemonHealth. This is the sharpest of the three — narrowing the
  reader or the comparison disables the very refusal ADR 0006 exists to
  guarantee, and nothing else in the repo would notice.
- /upload consumer: UploadResponse, UploadPreflightResponse, UploadPreflightResult,
  parseUploadPreflightResult, requestUploadPreflight, uploadDirectArtifact,
  tryDirectUploadWithResume, shouldRetryDirectUpload, finalizeDirectUpload,
  uploadLegacyArtifact, ARTIFACT_HASH_ALGORITHM, isStringRecord, and
  PreparedUploadArtifact — whose sha256/sizeBytes/fileName/artifactType/
  contentType fields ARE the preflight body the daemon parses.
- /artifacts/* consumer: DaemonArtifactEndpoint, buildDaemonArtifactUrl,
  isRemoteDaemon, DownloadRemoteArtifactParams, downloadRemoteArtifact,
  materializeRemoteArtifacts, resolveMaterializedArtifactPath.

Running the closure fail-closed over the new files surfaced three more stops,
each decided rather than skipped: PreparedUploadArtifact listed (it is payload),
UploadProgressSink waived (client-local rendering, never leaves the process),
and src/daemon/types.ts#DaemonArtifact waived as a re-export alias of the listed
kernel type, matching its DaemonRequest/DaemonResponse siblings.

10 more planted-red mutations cover the new seams: health version-read and
mismatch-refusal defeated, RemoteDaemonHealth field dropped, preflight parser
narrowed, preflight/legacy response shapes narrowed, finalize body key renamed,
ticket field renamed, artifact tenant header dropped, artifact URL moved. A
fourth closure probe proves the upload-consumer files are genuinely reached by
the walk rather than merely listed. 22 -> 33 tests.

The README now states the coverage as a producer/consumer table per boundary,
so the claim is checkable at a glance instead of asserted in prose.

* test: list the client half of the resumable 308 contract (#1432)

Addresses the third review P1 on #1717. Listing the daemon's
handleResumableUpload proved it still PRODUCES 308; nothing proved the client
still CONSUMES the released one. src/remote/upload-stream.ts owns that half and
was entirely outside the manifest, so a newer client could stop accepting
`upload-offset`, change how it reads `Range: bytes=0-N`, or emit a different
resumed `Content-Range` without moving one of the 141 listed digests.

Now listed (141 -> 151): UploadStreamResponse, streamFileToHttpRequest,
streamFileToHttpRequestAttempt, buildUploadRequestHeaders, isUploadResumeStatus,
isUploadRedirectStatus, parseUploadResumeOffset, parseNonNegativeIntegerHeader,
firstHeaderValue, MAX_UPLOAD_REDIRECTS.

streamFileToHttpRequestAttempt is listed despite its size, unlike
createDaemonHttpServer which stays in `uncovered`. The distinction is stated at
the declaration: the HTTP server only dispatches to handlers that are each
digested, while the attempt loop IS the resume state machine — it decides
whether a 308 continues the upload and what the next request carries, so its
sequencing alone can break a released daemon while every helper keeps its digest.

6 new planted-red mutations prove the client half moves the ledger: a dropped
`upload-offset` fallback, narrowed Range parsing, a changed resumed
Content-Range, 308 no longer treated as continue, a narrowed UploadStreamResponse,
and dropped header-value coercion. 33 -> 39 tests.

Closure fail-closed surfaced two more stops: UploadStreamProgressOptions waived
(local byte-progress rendering) and URL/URLSearchParams added to the global set.

README now carries a `/upload` resume row in the producer/consumer table, and
names the pattern behind three rounds of review: the coverage sentence kept
getting written ahead of the coverage, so the table and the `uncovered` notes
are the claims to trust — they are checkable against surface.ts, prose is not.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-10 20:52:29 +02:00

128 lines
4.9 KiB
TypeScript

/**
* The acceptance criterion #1432 states in one sentence — "a wire-type change
* without an RPC version bump fails" — proved from fixtures.
*
* It cannot be proved end-to-end yet: no released tag carries a wire ledger
* until this lands and ships, so `run.ts` has nothing to diff against. These
* cases pin the rule now, and `run.ts` becomes a thin git reader over them.
*/
import assert from 'node:assert/strict';
import test from 'node:test';
import type { WireLedger } from '../../test/wire-compat/ledger.ts';
import { compareWireLedgers } from './model.ts';
const RESPONSE = 'packages/kernel/src/contracts.ts#DaemonResponse';
const META = 'packages/kernel/src/contracts.ts#DaemonRequestMeta';
function ledger(overrides: Partial<WireLedger> = {}): WireLedger {
return {
protocolVersion: 2,
declarations: { [RESPONSE]: 'sha256:aaa', [META]: 'sha256:bbb' },
compatibleChanges: [],
...overrides,
};
}
function compare(current: WireLedger, digests: Record<string, string>) {
return compareWireLedgers({
baselineTag: 'v0.20.6',
released: ledger(),
current,
digests: new Map(Object.entries(digests)),
});
}
test('an unchanged wire surface passes', () => {
const result = compare(ledger(), { [RESPONSE]: 'sha256:aaa', [META]: 'sha256:bbb' });
assert.deepEqual(result.failures, []);
assert.deepEqual(result.changed, []);
});
test('a changed wire declaration without a bump or ack fails, naming the symbol', () => {
const current = ledger({ declarations: { [RESPONSE]: 'sha256:zzz', [META]: 'sha256:bbb' } });
const result = compare(current, { [RESPONSE]: 'sha256:zzz', [META]: 'sha256:bbb' });
assert.deepEqual(result.changed, [RESPONSE]);
assert.equal(result.failures.length, 1);
assert.match(result.failures[0]!, /DaemonResponse/);
assert.match(result.failures[0]!, /without bumping DAEMON_RPC_PROTOCOL_VERSION \(still 2\)/);
});
test('the same change passes once the protocol version is bumped', () => {
const current = ledger({
protocolVersion: 3,
declarations: { [RESPONSE]: 'sha256:zzz', [META]: 'sha256:bbb' },
});
const result = compare(current, { [RESPONSE]: 'sha256:zzz', [META]: 'sha256:bbb' });
assert.equal(result.bumped, true);
assert.deepEqual(result.failures, []);
});
test('the same change passes with a compatible-change ack at the new digest', () => {
const current = ledger({
declarations: { [RESPONSE]: 'sha256:zzz', [META]: 'sha256:bbb' },
compatibleChanges: [
{ declaration: RESPONSE, digest: 'sha256:zzz', rationale: 'Added an optional field.' },
],
});
const result = compare(current, { [RESPONSE]: 'sha256:zzz', [META]: 'sha256:bbb' });
assert.deepEqual(result.failures, []);
});
// The ack is keyed by the digest it covers precisely so it expires. Without
// this, one "added an optional field" ack would launder every later change to
// the same declaration.
test('an ack pinned to a superseded digest does not cover the next change', () => {
const current = ledger({
declarations: { [RESPONSE]: 'sha256:yyy', [META]: 'sha256:bbb' },
compatibleChanges: [
{ declaration: RESPONSE, digest: 'sha256:zzz', rationale: 'Covered the previous change.' },
],
});
const result = compare(current, { [RESPONSE]: 'sha256:yyy', [META]: 'sha256:bbb' });
assert.equal(result.failures.length, 1);
assert.match(result.failures[0]!, /DaemonResponse/);
});
test('an ack with an empty rationale does not count as an ack', () => {
const current = ledger({
declarations: { [RESPONSE]: 'sha256:zzz', [META]: 'sha256:bbb' },
compatibleChanges: [{ declaration: RESPONSE, digest: 'sha256:zzz', rationale: ' ' }],
});
const result = compare(current, { [RESPONSE]: 'sha256:zzz', [META]: 'sha256:bbb' });
assert.equal(result.failures.length, 1);
});
test('a removed wire declaration fails even when acked, because only a bump covers it', () => {
const current = ledger({
declarations: { [META]: 'sha256:bbb' },
compatibleChanges: [
{ declaration: RESPONSE, digest: 'sha256:aaa', rationale: 'Nobody used it.' },
],
});
const result = compare(current, { [META]: 'sha256:bbb' });
assert.deepEqual(result.removed, [RESPONSE]);
assert.equal(result.failures.length, 1);
assert.match(result.failures[0]!, /an ack cannot cover it/);
});
test('a removed wire declaration passes with a bump', () => {
const current = ledger({ protocolVersion: 3, declarations: { [META]: 'sha256:bbb' } });
const result = compare(current, { [META]: 'sha256:bbb' });
assert.deepEqual(result.failures, []);
});
test('a newly added wire declaration is additive and needs nothing', () => {
const added = 'packages/kernel/src/contracts.ts#NewEnvelope';
const current = ledger({
declarations: { [RESPONSE]: 'sha256:aaa', [META]: 'sha256:bbb', [added]: 'sha256:ccc' },
});
const result = compare(current, {
[RESPONSE]: 'sha256:aaa',
[META]: 'sha256:bbb',
[added]: 'sha256:ccc',
});
assert.deepEqual(result.added, [added]);
assert.deepEqual(result.failures, []);
});