Address the CR gaps on the in-process HTTP-probe control-plane:
- Sweep orphaned `running` probe_runs at control-plane boot (boot()'s
sweepStaleRuns never ran in fleet mode → leaked rows forever). Best-effort;
a sweep failure does not abort boot.
- Add a fail-loud BROWSER_KINDS / HTTP-driver disjointness assert at boot plus
a drift-lock test mirroring registerAllProbeDrivers, so a mis-added kind
can't silently go dark.
- Assert the PR's headline guarantees that were unasserted: cron-from-YAML
(exact values), /health loader-failure boot-survival + probes.reload.failed
emit + rules=0 observability, hot-reload add/remove + watcher teardown, a
discovery-backed family (qa) in the in-process schedule, and tightened
/health rule/job counts to exact equality.
- Bump the includeKind skip log to info; make diffHttpProbeSchedules'
unregister-failure post-state explicit (keep config observable); correct the
/health "no longer masks" comment, the discovery-source comment
(image_drift uses railway-services; version_drift uses pnpm-packages), the
reload-failed surface comment (no bus subscriber on the control-plane), and
drop transitional-rot tags.
The fleet control-plane previously ran only the d6 producer, leaving the 8
HTTP-only probe families (smoke, starter_smoke, image_drift, qa, aimock_wiring,
version_drift, pin_drift, redirect_decommission) dark on the fleet. Lift the
legacy boot() probe-loader machinery into runControlPlane so those families run
in-process:
- Add BROWSER_KINDS = {e2e_d6, e2e_smoke, e2e_demos, e2e_deep}; HTTP = every
other kind. Add registerHttpProbeDrivers (HTTP-only driver set, no BrowserPool
drivers).
- In runControlPlane, build an HTTP-only probeRegistry + discovery registry
(railway-services cached + pnpm-packages, mirroring boot()), a probe-loader
scoped to HTTP kinds, and the same diffProbeSchedules/buildProbeInvoker loop
boot() uses — registering one probe:<id> scheduler entry per YAML config.
Crons are driven from the YAML schedule. Browser e2e_* YAMLs route to the
worker producer path and are NOT scheduled in-process.
- Add an includeKind predicate to createProbeLoader so a browser YAML on disk is
SKIPPED (not rejected) against the HTTP-only registry — no spurious
probes.reload.failed.
- /health: ruleCount now reflects the in-process HTTP probe count (was a
hardcoded 0). The control-plane role still drops the rules>0 gate, but the
real count means a silent probe-loader failure is visible on /health rather
than masked. schedulerJobCount already counts the new probe entries.
- Tear down the HTTP-probe file watcher on stop() and on bind failure.
## Summary
Producer-side foundation for fleet framework item 3b. Two
**behavior-preserving** generalizations that make the seam capable of
multiple browser families and multiple producer cadences, while keeping
the d6 case **byte-identical**. No wiring is flipped on yet (see Out of
Scope).
### 1. Parameterized service enumerator
`showcase/harness/src/fleet/control-plane/catalog-enumerator.ts`
- New generic `createServiceEnumerator(params)`
(catalog-enumerator.ts:215) takes the service-set `filter`, the
`driverKind`, and a `probeKeyPrefix` (string prefix → `<prefix>:<slug>`,
or a builder fn).
- `createD6ServiceEnumerator` (catalog-enumerator.ts:280) is
re-expressed as a thin call passing the d6 params: `D6_DRIVER_KIND`
(`e2e_d6`), prefix `"d6"` (→ `d6:<slug>`), and `D6_DISCOVERY_FILTER`. d6
output is unchanged — same services, same filter, same kind, same keys.
### 2. Control-plane accepts an array of producer schedules
`showcase/harness/src/fleet/control-plane/control-plane.ts`
- New `ProducerSchedule` type (`{ scheduleId, cron, producer }`) + a
`schedules?` dep on `ControlPlaneDeps`.
- `createControlPlane` normalizes to an array (control-plane.ts:~232);
omitting `schedules` degenerates to the single d6 schedule on
`FLEET_PRODUCER_SCHEDULE_ID` (`fleet-job-producer`) @ `40 * * * *` —
current behavior preserved exactly.
- `start()` / `stop()` iterate the array, registering/unregistering each
scheduler entry and starting/stopping each producer.
## Out of scope (deferred — gated on other in-flight PRs)
- **No `runControlPlane` wiring** to actually PASS multiple schedules —
that edit conflicts with in-flight **#5284** (which edits
`runControlPlane`) and is deferred. This PR only makes
`control-plane.ts` *capable* of N schedules + generalizes the enumerator
seam; the wiring lands later.
- No `e2e_smoke` / `e2e_demos` / `e2e_deep` enumerators or producers
(Phase 2).
- No changes to `worker-loop.ts` / `payload-mapper.ts` /
`probe-loader.ts` (other PRs own those).
- No driverKind constant / contract changes.
## Test plan
- [x] Red→green TDD: 3 new enumerator tests (generic
kind/keys/filter/fn-prefix) + 2 new control-plane tests (N entries
registered with distinct crons; stop tears all down) failed before impl,
pass after.
- [x] Equivalence: all pre-existing d6-enumerator + single-schedule
control-plane tests pass unchanged.
- [x] Full harness suite green: **2078 passed** (119 files).
- [x] `tsc -p tsconfig.build.json` clean (exit 0).
- [x] Only the 4 intended files changed; no lockfile drift.
Do not merge — producer-side foundation only; wiring follows after #5284
lands.
## Summary
The `showcase_build` workflow's pre-build `verify-image-refs` gate (SSOT
= `showcase/scripts/railway-envs.ts`) was **failing all staging
deploys** with `1 untracked Railway services`. The interim
`harness-legacy` staging service (id
`11279eba-97eb-417e-82a5-7cb4254eb147`, project `showcase`, env
`staging`) exists on Railway but had no SSOT entry, so the Railway→SSOT
drift check failed → build skipped → nothing deployed (this blocked PR
#5283's merge build).
`harness-legacy` is the **interim legacy all-probe harness**
(`HARNESS_ROLE` unset) stood up to keep non-d6 coverage live during the
fleet migration. It is **not CI-built** (runs a pinned pre-fleet image
digest set out-of-band) and will be torn down at migration end.
## Change
- Adds a `harness-legacy` entry to `SERVICES` mirroring the
`showcase-harness-worker` precedent (PR #5280):
- `ciBuilt: false` — not built by `showcase_build`; no dedicated build
slot.
- `gateIgnore: true` — deliberately-untracked for the image-ref gate.
`findUntrackedServices` treats any SSOT entry as known, so this clears
the "untracked" failure; `gateValidated: false` keeps
`findMissingServices` from flagging it.
- `repoNameOverride` → `showcase-harness` for both envs (same image-ref
shape as the control-plane harness).
- Real serviceInstance IDs recorded for both envs (resolved via Railway
GraphQL); probe disabled in both envs.
- Regenerates `railway-envs.generated.json`.
- Updates service-count and gate-ignored carve-out assertions across the
three affected test files (28 → 29 services; `harness-legacy` added to
the two `GATE_IGNORED` sets).
## Verification
- **Live gate (red→green):** without the entry,
`verify-railway-image-refs.ts` exits **1** with `harness-legacy is not
in the SSOT`; with the entry it exits **0** — `✓ 54 env-scoped instances
verified (2 skipped)`.
- **Full scripts test suite green:** 1771 passed (49 files), including
`railway-envs.test.ts`, `verify-railway-image-refs.test.ts`, and
`emit-railway-envs-json.test.ts`.
- **`emit --check`** confirms `railway-envs.generated.json` is in sync.
- **tsc** clean (`tsc --noEmit -p showcase/scripts/tsconfig.json`).
## Test plan
- [x] `npx tsx showcase/scripts/verify-railway-image-refs.ts` exits 0
against live Railway
- [x] `vitest run` in `showcase/scripts` fully green
- [x] `npx tsx showcase/scripts/emit-railway-envs-json.ts --check`
passes
- [x] `tsc --noEmit -p showcase/scripts/tsconfig.json` clean
- [ ] CI green on this PR
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The showcase_build verify-image-refs gate (SSOT = showcase/scripts/railway-envs.ts)
was failing with "1 untracked Railway services" because the interim
harness-legacy staging service (the legacy all-probe harness kept live during
the pool-fleet migration) exists on Railway but had no SSOT entry. That
Railway->SSOT drift check skips the build, so nothing deploys.
Adds a harness-legacy SERVICES entry mirroring the showcase-harness-worker
precedent (PR #5280): ciBuilt:false (not built by showcase_build, runs a pinned
out-of-band digest) and gateIgnore:true (deliberately-untracked for the image-ref
gate). findUntrackedServices treats any SSOT entry as known, so this clears the
untracked failure; gateValidated:false keeps findMissingServices from flagging
it. Real serviceInstance IDs for both envs recorded from Railway GraphQL.
Regenerates railway-envs.generated.json and updates the service-count /
gate-ignored carve-out assertions (28->29 services).
Verified: live verify-railway-image-refs.ts now exits 0 ("54 env-scoped
instances verified, 2 skipped"); without the entry it exits 1 with the
harness-legacy untracked failure. Full scripts test suite green (1771 passed).
## Summary
Generalizes the fleet pool-worker from a single hardwired d6 driver into
a driver **registry** keyed by `payload.driverKind`, so a single worker
can host all four browser driver families (`e2e_d6`, `e2e_deep`,
`e2e_demos`, `e2e_smoke`). Phase-1 framework, item 3a of the harness
fleet migration.
- **`worker-loop.ts`** now accepts `drivers: Map<string, { driver;
payloadToInput }>` and dispatches each claimed job by
`payload.driverKind`. An unknown kind returns a terminal
`worker-protocol-violation` result (the same failure shape an unmappable
payload uses) — the worker never crashes on an unhandled kind. The
legacy single `driver`+`payloadToInput` pair is retained as a fallback
(so the pre-registry behavior and all existing callers keep working),
and construction fails loud if neither a non-empty registry nor the
legacy pair is supplied.
- **`payload-mapper.ts`** generalizes `createD6PayloadToInput` into a
shared `createPayloadToInput` plus per-kind aliases
(`createDeepPayloadToInput`, `createDemosPayloadToInput`,
`createSmokePayloadToInput`) and the four `E2E_*_DRIVER_KIND` constants.
The re-hydration is identical across families; each driver's own zod
schema is the validation gate.
- **`orchestrator.ts` `runWorker`** builds all four pooled drivers on
the shared `BrowserPool` and registers them by kind (lifted from the
legacy `registerAllProbeDrivers` pooled construction), then injects the
registry into the fleet worker.
- **`fleet/orchestrator.ts` `runWorker`** threads an optional `drivers`
registry through to `startWorkerLoop` (registry takes precedence; legacy
single-driver and self-contained-boot paths preserved).
**d6 routing is unchanged** — equivalence is the gate.
## Test plan
- [x] Red→green: new tests fail against the reverted implementation,
pass with it (verified by stashing the impl files and re-running).
- [x] worker-loop routing tests: `e2e_smoke`→smoke, `e2e_deep`→deep,
`e2e_d6`→d6 (equivalence), unknown kind→`worker-protocol-violation`,
matched-kind uses its own mapper, `startWorkerLoop` dispatches by kind
end-to-end.
- [x] payload-mapper tests: four driver-kind constants, per-kind mapper
re-hydration + key defaulting.
- [x] Full harness suite green: **2085 tests** across 119 files.
- [x] `tsc --noEmit` clean except the known pre-existing `toReversed`
error (present on `main`).
Do NOT merge — pending 7-agent CR.
The multi-schedule seam (createServiceEnumerator + the schedules[] capability
in control-plane) didn't meet the file's own best-effort/fail-loud bar. Harden
it now since Phase 2 builds on it (no production caller yet):
- stop(): guard each producer.stop() per-entry so one rejection no longer aborts
teardown of later schedules (leaked cron handlers + running producers).
- start(): pre-validate every schedule's cron up-front before starting any
producer, throwing an aggregated error naming the offending scheduleId — no
more half-started plane with `started` latched true.
- normalization: throw on duplicate scheduleId (replace-semantics would silently
collapse two producers onto one entry) and on an explicitly-empty schedules:[]
(distinct from omitted, which keeps the d6 default).
- createServiceEnumerator: require a non-empty filter.namePrefix (an absent
prefix would enumerate ALL services) and reject an empty probeKey from a
function-form prefix, naming the slug.
Also: narrow the d6 "byte-identical" docstring (specs identical; the
catalog-enumerated log adds driverKind), pluralize the start()/stop() +
module-header producer comments, drop the Phase 2 marker, and extract the shared
ServiceSetFilter type.
Producer-side foundation for fleet item 3b — two behavior-preserving
generalizations, byte-identical for the d6 case:
1. Generalize the d6 service enumerator into a parameterized
`createServiceEnumerator(params)` carrying the service-set filter, the
driverKind, and the probeKey prefix builder. `createD6ServiceEnumerator`
is now a thin wrapper passing the d6 params (e2e_d6 kind, d6:<slug> keys,
D6_DISCOVERY_FILTER), so d6 behavior is unchanged.
2. Generalize createControlPlane to accept an array of
{ scheduleId, cron, producer } entries and register each on the scheduler.
The single-d6 case degenerates to a one-element array on
fleet-job-producer @ 40 * * * *, preserving current behavior.
Out of scope (gated on in-flight PRs): orchestrator runControlPlane wiring
to pass multiple schedules (conflicts with #5284), the e2e_smoke/demos/deep
families (Phase 2), and worker-loop/payload-mapper/probe-loader.
CR fixes for the fleet worker driverKind→driver registry (PR #5283):
- Fix default (self-contained) worker boot: build the default d6 as a
registry entry { driver, payloadToInput, aggregateSlugKey } instead of a
bare driver with no mapper, so startWorkerLoop's construction guard no
longer throws "Fleet worker has no drivers".
- Thread aggregate-key derivation through DriverRegistryEntry
(aggregateSlugKey?), defaulting to d6:<slug> so the d6 cell-capture filter
stays byte-identical while non-d6 kinds can supply their own scheme.
- Add construction-time fail-loud assert that each registry entry's factory
kind matches its key; raise unknown-driver-kind log to error to match the
sibling protocol-violation logs.
- Extract shared buildPooledBrowserDrivers consumed by both
registerAllProbeDrivers and the worker registry; collapse no-op per-kind
payload-mapper aliases to the single createPayloadToInput.
- Introduce typed DriverKind union (contained to worker/payload-mapper);
keep contracts.driverKind a string wire boundary.
- Tests: new fleet/orchestrator.test.ts (default-boot equivalence), driver
construction-guard + custom aggregateSlugKey coverage, and lock-step +
registry-wiring pins (factory kind == constant).
Generalize the fleet worker from a single hardwired d6 driver into a
driver REGISTRY keyed by payload.driverKind, so one worker can host all
four browser driver families (e2e_d6, e2e_deep, e2e_demos, e2e_smoke).
- worker-loop: accept a `drivers: Map<kind, { driver, payloadToInput }>`
and dispatch each claimed job by `payload.driverKind`. Unknown kind →
terminal `worker-protocol-violation` (same shape as an unmappable
payload), never a worker crash. Legacy single `driver`+`payloadToInput`
pair retained as a fallback for back-compat. Fail-loud at construction
when neither a registry nor the legacy pair is supplied.
- payload-mapper: generalize createD6PayloadToInput into a shared
createPayloadToInput plus per-kind aliases and the four driver-kind
constants (the input re-hydration is identical across families; each
driver's own zod schema is the validation gate).
- orchestrator runWorker: build all four pooled drivers on the shared
BrowserPool and register them by kind, lifted from the legacy
registerAllProbeDrivers pooled construction.
d6 routing is unchanged (equivalence gate). Red-green tests cover
routing e2e_smoke/e2e_deep/e2e_d6 to their drivers and unknown-kind →
protocol-violation. Full harness suite green (2085 tests); tsc clean
except the known pre-existing toReversed error.
## Summary
Removes `examples/integrations/langgraph-python-threads/` (87 files) and
its two entries in `.github/config-allowlist.txt`.
The standalone "LangGraph Python + durable threads" template is defunct:
- The ENT-679 threads rollout was fully reverted from main on 2026-06-04
(#5215/#5216/#5217); the settled model is base `langgraph-python` +
Intelligence activation overlay, not a duplicate `-threads` template.
- The Intelligence CLI no longer scaffolds from it — verified zero
references on Intelligence `main` (a806a7e0) **and** in the published
`copilotkit@3.0.2` npm tarball.
## Verification
- `git grep langgraph-python-threads` across the tracked tree returns
nothing after this change (no hits in `pnpm-workspace.yaml`,
`pnpm-lock.yaml`, `_parity` manifest, CI workflows, or READMEs).
- The example was standalone (npm-based, not a pnpm workspace member,
not an Nx project) — zero workspace projects structurally affected.
- Lefthook commit gates green: check-binaries, sync-lockfile, lint-fix,
`nx run-many -t test --projects=packages/**` (25/25).
- `nx run-many -t lint,build --projects=packages/**`: all green except
pre-existing `@copilotkit/vue:lint` failures on main (files untouched by
this diff, introduced in 913c36b8b5).
Linear: ENT-800
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The first merge of main accidentally committed local lint/format
auto-fixes to 28 files unrelated to the example removal. Restore
them byte-for-byte from origin/main.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The standalone threads template is retired: the ENT-679 rollout was
reverted from main (#5215/#5216/#5217) and the settled model is base
langgraph-python + Intelligence activation overlay. The Intelligence
CLI (main and published copilotkit@3.0.2) no longer references it.
Also drops its two entries from .github/config-allowlist.txt.
## Summary
The pool-fleet cutover manually created a new staging-only Railway
service `showcase-harness-worker` (HARNESS_ROLE=worker, 2 replicas) and
flipped the existing `harness` service to HARNESS_ROLE=control-plane.
The new service was **not in the SSOT**
(`showcase/scripts/railway-envs.ts`), so `verify-railway-image-refs.ts`
failed the "Showcase: Build & Push" workflow on every push to main:
```
✗ [railway] showcase-harness-worker
reason: Railway service "showcase-harness-worker" is not in the SSOT.
```
Because the `build` job `needs: [verify-image-refs]`, that gate failure
**skipped the harness build** — meaning recent harness fixes (e.g.
#5279's chat-rung probe) never rebuilt to staging. The secondary
`Aggregate build results` ENOENT failure was a downstream consequence of
the skipped build.
This PR adds `showcase-harness-worker` to `SERVICES`, matching the live
Railway service:
- **`ciBuilt: false`** — the worker runs the SAME `showcase-harness`
GHCR image the existing harness build slot produces; there is no
separate worker build slot.
- **`gateIgnore: true` / `gateValidated: false`** — the worker is
staging-only (no prod serviceInstance) and domain-less (it pulls jobs
from the control-plane queue rather than serving HTTP), so it does not
fit the symmetric dual-env / public-domain shape the image-ref gate
validates. `gateIgnore` clears the "untracked Railway service" failure
(any SSOT entry counts as known) **without** triggering a false "missing
from prod" failure (`findMissingServices` only checks `gateValidated:
true` entries).
- **`repoNameOverride` → `showcase-harness`** so the image-ref shape
resolves correctly.
- **probe disabled in both envs** — no externally-reachable health
endpoint.
The existing `harness` (now control-plane) entry is unchanged and
remains correct (it still builds + serves the `showcase-harness` image
on its public domain).
Also regenerates `railway-envs.generated.json` and updates the
SSOT-count / gate-coverage test invariants (27 → 28 services; the worker
is the sole intentional `gateIgnore`/`gateValidated:false` entry).
## Proof
Against live Railway (CLI auth):
- Before: `✗ Railway image-ref drift detected (... 1 untracked Railway
services ...)`
- After: `✓ 54 env-scoped instances verified (1 skipped)` — worker is
the 1 skipped (gateIgnore), 0 untracked.
- `emit-railway-envs-json.ts --check`: up to date.
## Test plan
- [x] `verify-railway-image-refs.ts` passes against live Railway (0
untracked)
- [x] `emit-railway-envs-json.ts --check` passes (generated JSON
regenerated)
- [x] Full `showcase/scripts` vitest suite: 1772 passed (49 files)
- [ ] CI green on "Showcase: Build & Push" after merge (the gate failure
that skipped the harness build is removed)
The pool-fleet cutover manually created the staging-only
`showcase-harness-worker` Railway service (HARNESS_ROLE=worker, 2
replicas) and flipped the existing `harness` service to
HARNESS_ROLE=control-plane. The new service was untracked in the SSOT,
so verify-railway-image-refs.ts failed the "Showcase: Build & Push"
workflow on every push to main (1 untracked Railway service), which
skipped the harness `build` job and blocked harness image rebuilds.
Add the worker to SERVICES as a staging-only, domain-less queue worker:
- ciBuilt:false — it runs the SAME `showcase-harness` image the existing
harness build slot produces; there is no separate worker build.
- gateIgnore:true / gateValidated:false — no prod instance and no public
domain, so it does not fit the symmetric dual-env shape the image-ref
gate validates. gateIgnore clears the "untracked Railway service"
failure (any SSOT entry counts as known) without tripping a false
"missing from prod" failure.
- repoNameOverride → showcase-harness so the image-ref shape resolves.
- probe disabled in both envs (no externally-reachable health endpoint).
Regenerate railway-envs.generated.json and update the SSOT-count and
gate-coverage test invariants (27→28 services; worker is the sole
intentional gateIgnore/gateValidated:false entry).
## Summary
The harness `starter-smoke` **chat** rung 503'd on staging because its
probe is a raw `fetch()` POST that sends only `Content-Type` — it omits
`X-AIMock-Context`. The scoped per-integration "Hello" fixtures in
`showcase/aimock/d4/<integration>/chat.json`
(`{match:{userMessage:"Hello", context:"<ctx>"}}`) only match under
aimock **strict mode** when the request carries
`x-aimock-context:<ctx>`. The local browser e2e passes only because each
integration's `playwright.config.ts` injects `extraHTTPHeaders:{
"X-AIMock-Context": "<ctx>" }`, forwarded by the integration's
`HeaderForwardingMiddleware` to aimock.
**Fix:** send `X-AIMock-Context: <columnSlug>` on the chat POST **only**
(the GET health/agent/interaction rungs hit the runtime `/info` route +
app shell, not aimock — matching what the browser does). The driver
already resolves `columnSlug` via `starterToColumnSlug(...)` before the
chat rung runs, and that column slug is exactly the fixture/playwright
context value, so the slug↔context skews resolve for free.
### Definitive starter → X-AIMock-Context map
The context token equals the **dashboard column slug** for every mapped
starter. Verified two ways: each integration's
`showcase/integrations/<col>/playwright.config.ts`
`extraHTTPHeaders["X-AIMock-Context"]`, and the
`showcase/aimock/d4/<col>/chat.json` `match.context` field — both equal
the column slug.
| probe starter slug | X-AIMock-Context (= column slug) | skew? |
|---|---|---|
| `langgraph-js` | `langgraph-typescript` | yes |
| `adk` | `google-adk` | yes |
| `strands-python` | `strands` | yes |
| `ms-agent-framework-dotnet` | `ms-agent-dotnet` | yes |
| `ms-agent-framework-python` | `ms-agent-python` | yes |
| `crewai-crews` | `crewai-crews` | no |
| `langgraph-fastapi` | `langgraph-fastapi` | no |
| `langgraph-python` | `langgraph-python` | no |
| `agno` | `agno` | no |
| `llamaindex` | `llamaindex` | no |
| `mastra` | `mastra` | no |
| `pydantic-ai` | `pydantic-ai` | no |
No genuinely-absent contexts: all 12 mapped starters have a `{Hello,
<col>}` fixture, and an **unmapped** starter returns a red aggregate
*before* any chat probe fires (so `aimockContext` is always populated
when the chat POST runs).
## Red → Green proof
**Unit (harness):** 5 new tests assert the chat POST carries
`x-aimock-context = column slug` (direct + skew maps), only on the chat
rung, alongside `Content-Type`/`Accept`. RED first (5 failed against
unfixed source: header `undefined`), GREEN after the fix.
**Live staging aimock**
(`https://aimock-staging.up.railway.app/v1/chat/completions`, `Hello`
turn):
| starter | RED (no header) | GREEN (`X-AIMock-Context: <col>`) |
|---|---|---|
| `langgraph-js` → `langgraph-typescript` | 503 `no_fixture_match` | 200
|
| `adk` → `google-adk` | 503 `no_fixture_match` | 200 |
| `strands-python` → `strands` | 503 `no_fixture_match` | 200 |
| `langgraph-python` | 503 `no_fixture_match` | 200 |
| `mastra` | 503 `no_fixture_match` | 200 |
## Test plan
- [x] RED: 5 new context tests fail against unfixed source
- [x] GREEN: `starter-smoke.test.ts` 47/47 pass
- [x] Full harness suite: 2073/2073 pass (119 files)
- [x] tsc clean (only the pre-existing `toReversed` lib error remains)
- [x] oxfmt clean, oxlint 0 errors
- [x] Live-staging RED→GREEN for 5 starters incl. 3 skewed names
The harness starter-smoke CHAT rung 503'd on staging because its raw fetch()
POST sent only Content-Type — omitting X-AIMock-Context. The scoped
per-integration "Hello" fixtures in showcase/aimock/d4/<integration>/chat.json
only match (under aimock strict mode) when the request carries
X-AIMock-Context:<context>, and that context token IS the dashboard column
slug (verified against each integration's
showcase/integrations/<col>/playwright.config.ts extraHTTPHeaders and the
fixture's match.context). The local browser e2e passes only because Playwright
injects the header, forwarded by HeaderForwardingMiddleware to aimock.
Send X-AIMock-Context: <columnSlug> on the chat POST only (matching the
browser; the GET rungs hit the runtime /info route, not aimock). columnSlug is
already resolved via starterToColumnSlug before the chat rung runs, so the skew
cases (langgraph-js->langgraph-typescript, adk->google-adk,
strands-python->strands, ms-agent-framework-*->ms-agent-*) get the correct
context for free.
Proven RED->GREEN against live staging aimock
(https://aimock-staging.up.railway.app): without the header -> 503
no_fixture_match; with the mapped column-slug context -> 200 matching the
scoped fixture.
Framework landing heroes (e.g. /langgraph-typescript) now show the same
two StartCommandCards as the home hero, and the "new project" command
pre-fills the framework via the CLI's --framework flag (e.g.
--framework langgraph-js). The mapping from docs slug to CLI framework is
verified against the CLI's AGENT_FRAMEWORKS enum; slugs with no 1:1
template fall back to a bare `npx copilotkit@latest create`. Frameworks
with bespoke setup (a2a git clone, ms-agent-dotnet) keep their own single
copy-command chip. Drops the old "Start the quickstart" button and the
legacy `npx copilotkit@latest init` chip.
Replace the home landing hero's Quickstart dropdown + command-copy chip
with a two-card <HeroStartCommands> block, framed by situation:
- Start a new project -> npx copilotkit@latest create
- Add to an existing project -> npx copilotkit@latest skills onboard
Both cards are equal weight with no pre-selected default. Each command
row copies on click with an aria-live announcement and a clipboard-blocked
fallback. Commands pin @latest so npx never reuses a stale cached CLI.
Removes the now-unused hero-quickstart-dropdown and hero-command-copy
components.
## Summary
- Bumps all starter integration examples under `examples/integrations/*`
to the latest published `@copilotkit` packages: `react-core`,
`react-ui`, `runtime`, `runtime-client-gql` (where present), `sdk-js`,
and the peer `a2ui-renderer` → **1.59.5**.
- Real prior pins were a mix of 1.59.3 / 1.59.1 / 1.56.5; all unified to
1.59.5.
- Regenerated each affected `package-lock.json` (starters use npm
per-app lockfiles and are intentionally excluded from the root pnpm
workspace — `pnpm-lock.yaml` untouched).
- Representative starter (`adk`) `next build` compiles successfully
against 1.59.5.
## Test plan
- [ ] CI build-checks green for the touched starters
- [ ] Starter dashboard greens on staging after image rebuild (chat rung
also depends on the aimock greeting fixture landing)
## Summary
Rewrites the `starter-smoke` probe to the **v2 path-based multi-route**
runtime protocol the deployed starters actually speak — proven via a
local build+curl gate against `examples/integrations/crewai-crews` at
1.59.5 (the runtime mounts `createCopilotEndpoint` in default
`mode:"multi-route"`, identical at 1.59.3 and 1.59.5).
- **agent** → `GET /api/copilotkit/info` requiring 200 + `version` (was:
"any non-404", which a health-JSON/HTML error page wrongly passed)
- **chat** → `POST /api/copilotkit/agent/<id>/run` (`Accept:
text/event-stream`), asserting ≥1 `TEXT_MESSAGE_CONTENT` delta **+**
terminal `RUN_FINISHED` **+** no `RUN_ERROR` (was: "non-empty body")
- **health** → repointed to `GET /api/copilotkit/info` (the starters
serve no `/api/health` route)
- **interaction** → `GET /` (unchanged)
- **drop the trailing slash** on runtime POSTs — `POST /api/copilotkit/`
308-redirects and drops the POST → 404, the core cause of the red rungs
Plus a **unified abort/body-read error classification** (one
`abortOutcome()` helper keyed on the local `externallyAborted` flag): a
self-timeout or a non-abort error → `transport-error` (soft), only a
genuine external abort → `aborted`; a cut-short body read softens to
`transport-error` instead of hard `smoke-failed`; the level loop
short-circuits on external abort. Reviewed across 3 CR rounds
(path-based rewrite + 2 abort-classification fix rounds + confirmation).
> **NOTE:** the content-level chat rung will correctly read **RED**
against the deployed starters until the Python-side `ag-ui-crewai`
`RUN_ERROR` (a real-LLM, in-process agent failure — *not* a probe bug)
is fixed. That is the truthful signal.
## Deferred follow-ups (tracked, not in scope)
- **Verify the alert/staleness consumer branches on `errorClass`, not
`state`** — else the soft/hard (`transport-error` vs `smoke-failed`)
split is cosmetic at the dashboard layer
- `res.text()` resolve-partial (truncated-but-resolved body) precision
edge → could mis-class as `smoke-failed`
- `redirect:"follow"` POST→GET edge; health/interaction success-path
body drain; health/agent share `/info` (correlated signals)
- `CLASS_RANK` ranks `aborted` lowest (design call); unmapped-starter
`columnSlug` invariant + prototype-key collision
- test-coverage gaps: `resolveTimeoutMs` env parsing,
`deriveStarterSlug` fallback, real-timer abort path
## Test plan
- [x] Harness unit suite — 1829 tests pass (105 files); starter-smoke
35/35; red-green verified for protocol + all abort/body-read fixes
- [x] Typecheck (no new errors), oxlint (0 errors), oxfmt, harness build
- [ ] Post-merge: redeploy harness, re-probe; chat rung greens once the
Python-side RUN_ERROR is fixed (separate workstream)
## Summary
Splits the showcase test harness into a **pool-fleet**: ONE Docker image
whose runtime role is selected at boot by `HARNESS_ROLE` into either a
**control-plane** (scheduler / job-producer / result-consumer /
result-aggregator / fleet-health) or a **worker** (BrowserPool +
claim/run/report loop). Roles communicate over a **PocketBase-backed
pull queue** (`probe_jobs`, `workers`) with a JSVM compare-and-set claim
hook (`fleet-claim.pb.js`) guaranteeing exactly-one-winner per claim.
Workers self-bound their own BrowserPool under the cgroup PID ceiling;
the control-plane runs no Chromium.
Key capabilities:
- **REQ-B — pool communication errors surface to the dashboard.** Worker
crashes, lost results, and unreachable-pool conditions are turned into a
`⚡` "unreachable" overlay on the affected cells (never silently
dropped). Three legs feed it: producer lease-driven sweep,
heartbeat-driven fleet-health reclaim, and the result-consumer's
resultless-past-grace latch — each preserving the prior observed colour
rather than fabricating green.
- **Browser-pool PID-ceiling hardening** carried over from the
single-process harness.
- **Role-dispatch is fail-loud:** an unset or invalid `HARNESS_ROLE` (or
`HARNESS_POOL_COUNT`) throws on boot, so a misconfigured fleet member
dies immediately instead of mis-running.
Reviewed via 5 CR fix rounds + 2 full 9-agent confirmation rounds
(cr-loop). 60 files, harness 2041 tests + dashboard 847 tests green,
both production builds green.
## Deferred follow-ups (bucket-(b): non-triggerable at N=2 / design
tensions, tracked not blocking)
- `isWorkerStale` PB-date parse works on Node V8 but doesn't share the
fleet-wide `PB_DATE_SEP_RE` normalization — align for goja-parity /
robustness.
- Never-observed-service comm-error coloring (`aggregateCommError`
writes `"error"` → history-only vs `aggregate()` using `"degraded"`) —
reviewers disagree on the correct direction; revisit as a deliberate
design decision.
- Recovered service can show "unreachable" until the next successful
aggregate run (self-clears within a producer cycle) — consider a
positive clear-on-green for the aggregate row.
- `terminalJobStatus` maps `degraded` → `failed` in run-history
(run-history widget over-reports failures for amber).
- Lease-renew guard uses strict `<` rather than a safety margin (only
matters at near-lease heartbeat config; defaults are 4× safe).
- Dashboard scans `health:`/`chat:`/`tools:` keys for comm-errors that
no harness leg writes (dead/defensive scope).
- `showcase test --isolate <name>` arg-order ambiguity in the CLI.
- Local `docker-compose.local.yml` worker `restart: unless-stopped` can
auto-revive a killed worker during the local REQ-B demo (use `docker
stop` / `restart: no`).
- Result-lost latch-failure can re-surface the same comm-error each
cycle (benign churn; needs result-lost + persistent PB write failure).
## Test plan
- [x] Harness unit suite — 2041 tests pass (119 files)
- [x] Dashboard unit suite — 847 pass / 1 skipped (53 files)
- [x] Typecheck (harness + dashboard), oxlint (0 errors), oxfmt
- [x] Production builds — harness `tsc` + dashboard `next build`
- [ ] Staging cutover to N=2 (gated; plan published separately on
Notion)
The chat rung hardcoded `/api/copilotkit/agent/default/run`, which 404s for
mastra — it registers dynamic non-`default` agent keys via
`MastraAgent.getLocalAgents` rather than `agents:{default}`. Resolve the
agent id per-starter from the first key of the `/info` `agents` map (the same
info response the health + agent rungs already fetch), falling back to
`default` only when that map is empty/unreadable, and surface the resolved id
on the chat row signal for drilldown.
Rewrite the starter-smoke probe to the v2 path-based multi-route runtime
protocol the deployed starters actually speak (proven via a local
build+curl gate against examples/integrations/crewai-crews at 1.59.5):
- agent rung: GET /api/copilotkit/info -> 200 + version (was: any non-404)
- chat rung: POST /api/copilotkit/agent/<id>/run (Accept: text/event-stream)
asserting >=1 TEXT_MESSAGE_CONTENT delta + terminal RUN_FINISHED + no
RUN_ERROR (was: non-empty body)
- health rung: repointed to GET /api/copilotkit/info (the starters serve
no /api/health route)
- interaction rung: GET / (unchanged)
- drop trailing slash on all runtime POSTs (the old /api/copilotkit/ 308s
and drops the POST -> 404, the core cause of the red rung)
Plus a unified abort/body-read error classification (one abortOutcome()
helper keyed on the local externallyAborted flag): a self-timeout or a
non-abort error is transport-error (soft), only a genuine external abort
is 'aborted'; a cut-short body read softens to transport-error instead of
hard smoke-failed; the level loop short-circuits on external abort.
NOTE: the content-level chat rung will correctly read RED against the
deployed starters until the Python-side ag-ui-crewai RUN_ERROR (a
real-LLM, in-process agent failure, NOT a probe bug) is fixed.
Deferred follow-ups: verify the alert/staleness consumer branches on
errorClass (not state) so the soft/hard split is actually consumed;
res.text() resolve-partial precision edge; redirect:follow POST edge;
health/interaction body-drain; health/agent shared /info; CLASS_RANK
aborted-lowest; unmapped-starter columnSlug/prototype-key; test-coverage
gaps (resolveTimeoutMs, deriveStarterSlug, real-timer abort).
## What
Updates the README to reflect CopilotKit's expansion beyond React:
- **New "Works With Your Stack" section** — platform table for
React/Next.js, Angular, Vue, and React Native
- **New "Beyond the Browser" section** — Slack & Microsoft Teams support
with an early access CTA
- Tagline updated to "on any framework, on any surface"
- "Chat UI" feature bullet made framework-neutral (was "React-based")
- useAgent section no longer anchors to the legacy useCoAgent hook
- AG-UI section: LangGraph → LangChain, added Mastra and PydanticAI to
named integrations
- New "How it works" bullet: one agent across web, mobile, and chat
platforms
## Before merging (draft)
- [ ] Replace `EARLY_ACCESS_FORM_LINK` placeholder with the real early
access form URL
- [ ] Point Angular / Vue / React Native quickstart links at their
framework-specific docs pages
- [ ] Confirm status labels (GA vs Supported vs Beta) for Angular / Vue
/ React Native
- [ ] Confirm `npx copilotkit create -f` accepts the new framework
values
Draft content also lives in Notion:
https://app.notion.com/p/copilotkit/New-GitHub-README-3763aa38185280788920fc81780c8ad6
## Summary
Follow-up to #5256 (the concurrent-paged initial-fetch perf fix). Trims
the live-status payload and hardens the realtime/SSE path of the
showcase Coverage dashboard.
- **Trim initial projection** — drop the heavy `signal` field from the
initial PocketBase fetch via a compiler-guarded `STATUS_LIST_FIELDS`
projection + `skipTotal` + length-based bounded-concurrency pagination.
Consumers adapted: `CellDrilldown` lazy-loads `signal` on demand
(genuine failures only), `DiscoveryAuthBanner` self-fetches detail
(identity-tagged cache), legacy `LiveBadge` degrades gracefully when
`signal` is absent.
- **Harden the realtime path** — `flushPending` teardown guards,
keyless-delete `id→key` buffer collapse (per-row last-write-wins, pure
ref+effect mirror), SSE-delta signal-presence upsert, and a `degraded`
flapping flag (clears on heartbeat success, resets on terminal error).
- **Grid/UX** — a `stale` (muted, non-authoritative) tally state
distinct from the no-rows loading state, an amber-chip/green-D5
dimension-classification fix, and `degraded` surfaced in the header
`LiveIndicator` (offline outranks degraded).
## Test plan
- [x] `tsc --noEmit` clean
- [x] `vitest run` — 879 passed / 1 skipped (56 files); +63 new
behavioral tests across the changed areas (real-SDK socket tests for
fetch/autocancel/lazy-signal, fake-timer tests for the flapping detector
and reconnect chain, exhaustive resolveCell truth-tables)
- [x] `next build` — compiled successfully, 5/5 static pages
- [x] oxfmt + oxlint clean
- [ ] CI green
- [ ] Visual verification on staging after deploy
## Known follow-ups (deferred — pre-existing / out-of-subject, not
introduced by this PR)
These were surfaced during review, verified pre-existing on `main` (or
unreachable in the shipping config), and intentionally not fixed here to
keep the PR scoped:
- Column-header tally counts deprecated features hidden from the grid
body (`computeColumnTally` iterates the full `features` list while the
body renders `visibleFeatures`) — pre-existing, header/body over-count
identical before and after. Strong candidate for a dedicated follow-up.
- `resolveCell` rollup does not suppress amber/degraded cells when
`connection === "error"` (only green is suppressed) — pre-existing
precedence.
- `formatTooltip` offline branch renders "degraded since
<transitioned_at>" for a staleness-downgraded green row.
- `LiveBadge` arms the `useLastTransition` fetch for
staleness-downgraded amber badges (no `isGenuineFailure` filter, unlike
the drilldown).
- `CellStatus` passes the raw `featureId` as the CV-badge transition
key, so mapped D5 families query a non-existent history row.
- Cosmetic: `extractSignalFields` renders `{}` for an empty-object error
field; `degraded` lacks an effect-cleanup reset on the
(production-unused) `dimension` param path; minor comment-accuracy nits
(reconnect backoff tiers, rollup-precedence JSDoc).
The check-config-files job blocked the strands-crm showcase PR
(examples/showcases/strands-crm/frontend/next.config.ts was not in
the build-config allowlist). Adding it next to strands-file-analyzer.
github-actions[bot]