Files
Michał Pierzchała 05a1d76f2e test: add daemon RPC wire-surface compatibility gate (#1717)
* test: gate daemon RPC wire compatibility against the last released tag (#1432)

ADR 0006 fixes exactly when DAEMON_RPC_PROTOCOL_VERSION must be bumped, and
nothing checked that it was. The runtime guard (readRemoteDaemonHealth) refuses
a mismatched peer, but only fires when someone remembered the bump — a wire
change that skipped it left both sides advertising protocol 2 while parsing
different payloads, which is the failure ADR 0006 exists to prevent.

Local daemons cannot skew (isReusableDaemonInfo takes over on any package
version mismatch). Cross-machine is skewed by design — proxy, cloud/limrun, a
remote macOS host — and ADR 0006 explicitly rules package version out as the
compatibility gate there, so the one boundary where skew is intended was the
one boundary with no gate.

test/wire-compat/surface.ts declares the wire surface grouped by the ADR bullet
each group serves, quoting it, with an `uncovered` note where a bullet is only
partly digestible (the /health and /rpc literals inside http-server.ts stay
reviewer-owned: a moved route 404s at connect time rather than misparsing).
ledger.json records what each declaration hashes to, at which protocol version.

Two gates, split for the same reason the replay-compat corpus splits:
- unit-core holds the ledger to its source and prints the digest to paste;
- Released-Surface Compatibility reads the ledger at the last RELEASED tag and
  requires the drift since then to carry a bump or a compatibleChanges ack.

From one commit a bumped ledger and an unbumped one are both just an edited
file, so only a released baseline can tell them apart. Acks are keyed by the
digest they cover, so one "added an optional field" cannot launder later
changes. Digests ignore comments and formatting; the manifest's closure is
derived from the AST, so a field typed by an unlisted sibling fails rather than
sitting outside the gate.

CI cost: one added job (checkout + toolchain + two node scripts, ~1 min),
mirroring the existing full-history replay-compat job.

* test: close wire-surface overclaim and make the closure fail closed (#1432)

Addresses both review P1s on #1717.

P1 — the manifest materially overclaimed ADR 0006 coverage. It quoted all four
bullets while digesting only the payload TYPES, so the producer and consumer
seams could break a skewed peer without moving a listed digest. Now listed on
both sides of every boundary: JSON-RPC method sets and the projections that
turn each method's params into a DaemonRequest, createRpcError/sendJson/
writeRpcResponseEnvelope, resolveToken and the auth-hook types, upload
preflight/finalize/308 handlers and the resumable ticket shape, artifact route
and download/inventory framing, REST error mapping, and the client's own
payload builder, lease-method mapping, response parser and error projection.
57 -> 117 declarations.

What stays out is now named rather than implied: createDaemonHttpServer's
dispatch wiring and the /health and /rpc literals inside it. Everything it
dispatches WITH is digested individually, and a moved route 404s at connect
time rather than misparsing — the loud failure, not the silent one.

P1 — imported and re-exported payload shapes escaped the closure.
declarationHomes() scanned only the manifest's own files and the walk
continued silently when a name could not be placed, so a listed type could
gain foo?: ImportedShape from a new module and stay green. Resolution is now
explicit and fails closed: relative imports, workspace specifiers (through the
owning package's own exports map, so a re-pointed export cannot drop a type),
and facade re-export chains. Every referenced name must land on a listed
declaration, a waiver with a written reason, a declared external module, or the
TS/Node global set. Fixed two extractor blind spots the walk exposed: a
declaration's own generic parameters and `as const` were being reported as
references.

Planted-red proofs (wire-mutations.test.ts): 13 cases independently mutate
method naming, response serialization, response parsing, auth projection,
upload ticket shape, 308 framing, artifact framing, REST error mapping, and
progress framing, each asserting the digest moves; 3 probes prove the closure
really reaches across a package boundary, a facade re-export, and a plain
relative import. Mutations apply inside the declaration's own span — a
whole-file replace silently hit a sibling sharing the substring, which is how
the first draft of one case passed vacuously.

The largest waiver pair (InternalRequestOptions, CommandFlags) rests on ADR
0006's own additive rule: they reach the peer inside DaemonRequest's untyped
flags/input bags, and the decision says a new flag needs no bump. Digesting
them would fire the gate on every new CLI flag and train reviewers to
rubber-stamp acks.

* test: list the consumer half of the auxiliary HTTP boundaries (#1432)

Addresses the remaining review P1 on #1717. The manifest claimed both sides of
response/upload/artifact framing while listing nothing from upload-client.ts,
daemon-artifacts.ts, or the health consumer in daemon-client-transport.ts, so
those parsers could narrow without moving a listed digest or protocol 2.

Now listed (117 -> 141 declarations):

- /health consumer: RemoteDaemonHealth, readHealthPayload, readDaemonHttpHealth,
  readRemoteDaemonHealth. This is the sharpest of the three — narrowing the
  reader or the comparison disables the very refusal ADR 0006 exists to
  guarantee, and nothing else in the repo would notice.
- /upload consumer: UploadResponse, UploadPreflightResponse, UploadPreflightResult,
  parseUploadPreflightResult, requestUploadPreflight, uploadDirectArtifact,
  tryDirectUploadWithResume, shouldRetryDirectUpload, finalizeDirectUpload,
  uploadLegacyArtifact, ARTIFACT_HASH_ALGORITHM, isStringRecord, and
  PreparedUploadArtifact — whose sha256/sizeBytes/fileName/artifactType/
  contentType fields ARE the preflight body the daemon parses.
- /artifacts/* consumer: DaemonArtifactEndpoint, buildDaemonArtifactUrl,
  isRemoteDaemon, DownloadRemoteArtifactParams, downloadRemoteArtifact,
  materializeRemoteArtifacts, resolveMaterializedArtifactPath.

Running the closure fail-closed over the new files surfaced three more stops,
each decided rather than skipped: PreparedUploadArtifact listed (it is payload),
UploadProgressSink waived (client-local rendering, never leaves the process),
and src/daemon/types.ts#DaemonArtifact waived as a re-export alias of the listed
kernel type, matching its DaemonRequest/DaemonResponse siblings.

10 more planted-red mutations cover the new seams: health version-read and
mismatch-refusal defeated, RemoteDaemonHealth field dropped, preflight parser
narrowed, preflight/legacy response shapes narrowed, finalize body key renamed,
ticket field renamed, artifact tenant header dropped, artifact URL moved. A
fourth closure probe proves the upload-consumer files are genuinely reached by
the walk rather than merely listed. 22 -> 33 tests.

The README now states the coverage as a producer/consumer table per boundary,
so the claim is checkable at a glance instead of asserted in prose.

* test: list the client half of the resumable 308 contract (#1432)

Addresses the third review P1 on #1717. Listing the daemon's
handleResumableUpload proved it still PRODUCES 308; nothing proved the client
still CONSUMES the released one. src/remote/upload-stream.ts owns that half and
was entirely outside the manifest, so a newer client could stop accepting
`upload-offset`, change how it reads `Range: bytes=0-N`, or emit a different
resumed `Content-Range` without moving one of the 141 listed digests.

Now listed (141 -> 151): UploadStreamResponse, streamFileToHttpRequest,
streamFileToHttpRequestAttempt, buildUploadRequestHeaders, isUploadResumeStatus,
isUploadRedirectStatus, parseUploadResumeOffset, parseNonNegativeIntegerHeader,
firstHeaderValue, MAX_UPLOAD_REDIRECTS.

streamFileToHttpRequestAttempt is listed despite its size, unlike
createDaemonHttpServer which stays in `uncovered`. The distinction is stated at
the declaration: the HTTP server only dispatches to handlers that are each
digested, while the attempt loop IS the resume state machine — it decides
whether a 308 continues the upload and what the next request carries, so its
sequencing alone can break a released daemon while every helper keeps its digest.

6 new planted-red mutations prove the client half moves the ledger: a dropped
`upload-offset` fallback, narrowed Range parsing, a changed resumed
Content-Range, 308 no longer treated as continue, a narrowed UploadStreamResponse,
and dropped header-value coercion. 33 -> 39 tests.

Closure fail-closed surfaced two more stops: UploadStreamProgressOptions waived
(local byte-progress rendering) and URL/URLSearchParams added to the global set.

README now carries a `/upload` resume row in the producer/consumer table, and
names the pattern behind three rounds of review: the coverage sentence kept
getting written ahead of the coverage, so the table and the `uncovered` notes
are the claims to trust — they are checkable against surface.ts, prose is not.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-10 20:52:29 +02:00

403 lines
15 KiB
YAML

name: CI
on:
pull_request:
paths-ignore:
- 'docs/**'
- 'website/**'
- 'README.md'
- '.github/actions/build-docs/action.yml'
- '.github/workflows/deploy.yml'
- '.github/workflows/pr-preview.yml'
- '.github/workflows/pr-preview-cleanup.yml'
push:
branches:
- main
permissions:
contents: read
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# Two single-`rg`-assertion jobs (formerly `ios-runner-swift-compat`,
# `no-test-di-seams`, added independently in b79bd8601 / 9eb060406) folded
# into steps here: each was checkout + one grep, paying full job
# scheduling/checkout overhead and its own PR status-check line for what is
# a single assertion. Each step keeps its own failure message. See #1462.
static-checks:
name: Static Checks
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Disallow trailing commas before closing parenthesis in Swift
run: |
if rg -nU --glob '*.swift' ',\s*\n\s*\)' apple/runner; then
echo "Found trailing commas before ')' in Swift files. This syntax requires Swift 6.1+ and breaks older Xcode toolchains."
exit 1
fi
- name: Fail if test-only DI seams reappear in production code
run: |
if rg '\?\s*:\s*typeof\s+' src/ --glob '!**/__tests__/**' --glob '!*.test.ts'; then
echo "Found test-only DI seams (optional typeof params) in production code."
exit 1
fi
swift-runner-unit-compile:
name: Swift Runner Unit Compile
runs-on: macos-26
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Restore and compile Swift runner unit-test surface
uses: ./.github/actions/setup-apple-runner-build
with:
derived-path: ${{ github.workspace }}/.tmp/swift-runner-unit-derived
cache-key-prefix: swift-runner-unit
build-command: AGENT_DEVICE_XCUITEST_INCLUDE_UNIT_TESTS=1 pnpm build:xcuitest:macos
xcuitest-platform: macos
xcuitest-destination: platform=macOS,arch=arm64
lint:
name: Lint & Format
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Run oxlint
run: pnpm lint
- name: Check formatting
run: pnpm format:check
layering-guard:
name: Layering Guard
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# This job used to run with install-deps: false, and R8 still holds every
# remaining zero-dep job to that contract. The layering guard itself opted out
# when R7 (SessionState ownership) started parsing the daemon with `oxc-parser`
# instead of matching assignment operators with a regex: a regex cannot see
# `??=` or a computed `session[key] =` write, so the choice was a real parser or
# a rule with holes in it. Keep install-deps enabled.
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Check import-direction DAG
# Generalizes the former inline commands/-import grep into a structured
# import-direction lint over the resolved graph. See scripts/layering/check.ts
# and CONTEXT.md (Architecture: folder DAG + layering lint).
run: pnpm check:layering
- name: Check the depgraph report agrees with the gate
# scripts/depgraph reads the same model as the gate, so its inversion count must
# reproduce TYPE_INVERSION_BASELINE. Free two-sources check: if the tree changes
# and only one side is updated, this fails and names the difference. Runs here
# rather than in its own job so the two can never be green independently.
run: pnpm depgraph:test
affected-selector:
name: Affected-check Selector
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# The selector's entry closure reaches `@agent-device/kernel` workspace
# specifiers through src/utils/exec.ts and diagnostics.ts (#1490 W0), and
# workspace package resolution needs the pnpm link in node_modules. The
# R8 relative-import exception is reserved for scripts, not production
# src files, so this job installs dependencies instead.
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
# The selector is fail-open and advisory (GitHub CI stays authoritative),
# so the gate only guards the derivation model.
- name: Check affected-selector model
run: pnpm check:affected:test
maestro-conformance:
name: Maestro Conformance Oracle
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# Unlike the layering/affected guards, this job DOES install deps: the
# verifier parses corpus flows with the live engine, and the Maestro parser
# imports the `yaml` package. Keep install-deps enabled.
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
# Layers 1-2 of the conformance oracle: replay the JVM-generated fixtures
# against the live engine. Deterministic and Java-free — the generated
# fixtures are checked in and only regenerated on an upstream-pin bump. The
# device-backed layer 3 runs on the scheduled conformance-differential
# workflow. See scripts/maestro-conformance/README.md.
- name: Verify Maestro conformance fixtures
run: pnpm maestro:conformance
packaged-cli-node-22-12:
name: Packaged CLI Node 22.12
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup build toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Build CLI
run: |
pnpm build
pnpm check:bundle-owner-files
# The build runs on the default toolchain Node and the package is verified on the minimum
# supported Node, so this job covers what a user on `engines.node` floor actually installs.
- name: Setup Node.js 22.12
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version: '22.12'
# Packs, lints the tarball with publint/attw, installs it outside the workspace, and imports
# every published entry point before running the CLI. See scripts/check-package.ts.
#
# Runs the script directly rather than through `pnpm check:package`: the repo's pinned pnpm
# requires Node >= 22.13 and refuses to start on the 22.12 floor this job exists to cover. The
# gate itself only needs `node` and `npm`, so it is the package.json script minus the launcher.
- name: Verify the published package on Node.js 22.12
run: node --experimental-strip-types scripts/check-package.ts
fallow:
name: Fallow Code Quality
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Run Fallow audit
env:
FALLOW_BASE: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
run: pnpm check:fallow --base "$FALLOW_BASE"
- name: Check for production-unused exports
run: pnpm check:production-exports
replay-compat-provenance:
# The frozen replay-compat corpus (#1417) claims each entry was published by
# a released tag. Only a full-history checkout can re-derive that claim, so
# this job exists separately from the shallow-clone-safe unit lane.
name: Replay-Compat Provenance
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
fetch-tags: true
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Verify corpus entries against their released blobs
run: pnpm check:replay-compat
released-surface-compat:
# The daemon RPC wire ledger (#1432) is compared against the ledger as it
# stood at the last RELEASED tag, which only a full-history checkout can
# read. Same split as the replay-compat corpus above: the shallow unit lane
# holds the ledger to its source, this job holds it to the last release.
name: Released-Surface Compatibility
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
fetch-tags: true
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Verify the wire-compat rules
run: pnpm check:daemon-wire-compat:test
- name: Compare the daemon RPC wire surface against the last released tag
run: pnpm check:daemon-wire-compat
coverage:
# Runs the full unit + provider-integration suites under coverage with
# thresholds, so a separate unit-tests job would rerun the same tests.
name: Coverage
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Test changed-line coverage gate
run: pnpm check:coverage-changed:test
# The retry list is an enumerated set of owned waivers, so an expired entry
# must fail before the suite runs rather than quietly keeping its retry.
- name: Check contention retry policy
run: pnpm check:contention-retry
# Wrapped in the single-retry policy (#1419): a timeout-shaped failure in
# an enumerated contention-flaky file reruns that file once and reports it
# in the job summary. Assertion failures fail here on the first run.
- name: Run coverage
env:
OUTPUT_ECONOMY_BASE: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
run: pnpm test:coverage:ci
- name: Upload contention-retry envelope
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: contention-retry-envelope
if-no-files-found: ignore
path: .tmp/contention-retry/lane-envelope.json
# Reuses the lcov the coverage step just wrote (never runs coverage twice)
# and fails when changed-line coverage < the threshold in
# scripts/coverage-changed/model.ts. The `coverage-waiver` PR label maps to
# the waiver env, which skips the failure but still prints the numbers.
- name: Enforce changed-line coverage gate
if: always() && github.event_name == 'pull_request'
env:
AGENT_DEVICE_COVERAGE_WAIVER: ${{ contains(github.event.pull_request.labels.*.name, 'coverage-waiver') }}
run: pnpm check:coverage-changed --base "${{ github.event.pull_request.base.sha }}"
typecheck:
name: Typecheck
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Run typecheck
run: pnpm typecheck
freerange:
name: FreeRange
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
- name: Check numeric ranges
run: pnpm check:freerange
integration:
name: Integration Tests
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
- name: Run integration tests
run: |
pnpm clean:daemon
pnpm test:integration:node
- name: Run seeded concurrency torture lane (fast PR sweep)
# #1416's nightly torture lane lives under test/integration/nightly/, out
# of the test:integration:node glob, so this is a *deliberate* fast PR
# sweep (TORTURE_RUNS default 128 seeds, ~sub-second) — not an accidental
# glob inclusion. The Concurrency Torture Nightly workflow sweeps a much
# larger seed range on schedule.
run: pnpm test:concurrency-torture
- name: Run provider-backed integration tests
run: pnpm test:integration:provider
- name: Check Provider-backed integration architecture progress
run: pnpm test:integration:progress:check
# A build-cache lookup outage must degrade setup-fixture-app to an inline
# build, not fail the caller. This drives that step's real shell against a
# failing `gh`.
- name: Setup-fixture-app cache-failure fallback
run: sh ./test/scripts/setup-fixture-app-fallback-smoke.sh
web-smoke:
name: Web Platform Smoke
runs-on: ubuntu-latest
timeout-minutes: 30
env:
AGENT_DEVICE_WEB_E2E: '1'
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup toolchain
uses: ./.github/actions/setup-node-pnpm
with:
node-version: '24.13'
- name: Run live web smoke
run: |
pnpm clean:daemon
pnpm test:smoke:web
- name: Upload web smoke artifacts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: web-smoke-artifacts
if-no-files-found: ignore
path: |
test/artifacts/web/**