mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
Jordan Ritter
5acbbcd633
Audits each showcase package's manifest-declared demos for matching spec (tests/e2e/*.spec.ts) and qa/*.md coverage, and enforces a monotonic demo-count baseline via fail-baseline.json. Key design: - PackageIssue tagged union (13 variants) cleanly separates MUST errors from warnings; deriveMessage is the single renderer so new variants cannot emit mismatched prose. - ProbeResult tagged union (missing | ok | unreadable) driven by statSync + errno inspection, distinguishing ENOENT from EACCES and surfacing ENOTDIR as a misconfiguration rather than a silent miss. - runParityImpl isolates each slug's audit in try/catch; a crash in one slug surfaces as a crashed PackageIssue variant and forces EXIT_INTERNAL without aborting siblings. - runParity never throws for content errors. InvalidBaselineError covers coerceBaseline failures; unknown errors route to EXIT_INTERNAL via formatErrorChain (walks .cause with cycle guard + depth cap). - parseMainArgs rejects unrecognised flags and duplicate --baseline with EXIT_INVALID_INPUT (2), mirroring audit.ts parseArgs discipline. - BASELINE_DEMO_COUNT default must match fail-baseline.json.baselineDemoCount; enforced by __tests__/baseline-sync.test.ts. - Exit codes: 0 ok, 1 should-warnings-only, 2 invalid-input, 3 unreadable, 4 internal, 5 must-failure. Tests cover every PackageIssue variant, every exit code (in-process + subprocess), per-slug crash isolation, EACCES routing, ENOTDIR classification, cascade suppression when tests/e2e or qa dirs are unreadable, and baseline coercion edge cases (leading zeros, negative, float, hex, non-numeric).
36 lines
1.6 KiB
TypeScript
36 lines
1.6 KiB
TypeScript
/**
|
|
* Sync contract: validate-parity.ts's BASELINE_DEMO_COUNT constant and
|
|
* fail-baseline.json's `baselineDemoCount` field are two declarations of
|
|
* the same number — the per-package e2e-spec-count floor. The workflow
|
|
* .github/workflows/showcase_validate.yml reads the JSON; validate-parity.ts
|
|
* uses the TS constant as its default. A divergence means the validator
|
|
* and the CI gate disagree silently, so we lock them together here.
|
|
*
|
|
* Prior to this test the contract was enforced only by a comment ("keep
|
|
* in sync"), which is trivially violated without any CI signal. R29-7
|
|
* finding #4 / R29-5.
|
|
*/
|
|
|
|
import { describe, it, expect } from "vitest";
|
|
import fs from "fs";
|
|
import path from "path";
|
|
import { fileURLToPath } from "url";
|
|
import { BASELINE_DEMO_COUNT } from "../validate-parity.js";
|
|
|
|
const __filename = fileURLToPath(import.meta.url);
|
|
const __dirname = path.dirname(__filename);
|
|
|
|
describe("baselineDemoCount sync contract", () => {
|
|
it("fail-baseline.json `baselineDemoCount` equals validate-parity.ts BASELINE_DEMO_COUNT", () => {
|
|
const baselinePath = path.resolve(__dirname, "..", "fail-baseline.json");
|
|
const raw = fs.readFileSync(baselinePath, "utf8");
|
|
const parsed = JSON.parse(raw) as { baselineDemoCount?: unknown };
|
|
// Explicit type guard rather than a blind cast — if the field ever
|
|
// disappears or its type flips, we want a clear failure, not
|
|
// `undefined === <n>` silently passing as false.
|
|
expect(typeof parsed.baselineDemoCount).toBe("number");
|
|
expect(Number.isInteger(parsed.baselineDemoCount)).toBe(true);
|
|
expect(parsed.baselineDemoCount).toBe(BASELINE_DEMO_COUNT);
|
|
});
|
|
});
|