PR #5952 (9a8cf615) added explicit `.js` extensions to the relative imports
inside the harness's shared cell-model fold
(showcase/harness/src/shared/cell-model/{cell-model,live-status,staleness}.ts)
— REQUIRED for the harness's pure-Node-ESM runtime and correct as-is.
But the dashboard re-exports that fold via shims
(showcase/shell-dashboard/src/lib/{cell-model,live-status,staleness,format-ts}.ts
`export * from "../../../harness/src/shared/cell-model/*"`), pulling the fold
into the dashboard's `next build`. `export *` does not rewrite the fold's
INTERNAL `.js` edges, and the dashboard's empty next.config.ts had no
extensionAlias, so webpack resolved `./live-status.js` literally, found only
the `.ts` source, and failed:
Module not found: Can't resolve './live-status.js'
Module not found: Can't resolve './staleness.js'
Module not found: Can't resolve './format-ts.js'
> Build failed because of webpack errors
Two-part fix (one coherent subject):
1. Resolution: add `webpack.resolve.extensionAlias` to
showcase/shell-dashboard/next.config.ts so `.js`/`.mjs` specifiers resolve
to `.ts`/`.tsx`/`.mts` sources — the bundler complement to TS NodeNext's
`.js`-import convention. Covers the `next build` (webpack) path CI uses.
The harness fold's `.js` imports are left untouched (they are correct).
2. CI gap: the dashboard build did not run on #5952 because the build matrix
is path-filtered and #5952 only touched `showcase/harness/**`, which
selects `showcase_harness` but not `shell_dashboard`. Add
`showcase/harness/src/shared/**` to the `shell_dashboard` paths-filter so
any change to the shared fold the dashboard compiles in also selects the
dashboard build — a fold change can never again ship an unbuilt dashboard.
Local red-green proof:
- RED (main, before fix): `next build` in showcase/shell-dashboard emitted the
4 fold-resolve errors above.
- GREEN (after extensionAlias): same build → 0 fold-resolve errors; the fold
resolves. Remaining `@/data/*.json` errors are the prebuild-generated files
(generate-registry/probe-docs) skipped in the local repro, produced in CI's
Docker build — unrelated to this fix.
## Summary
Routes the ~20 showcase demo backends to **aimock** (the record/replay
LLM proxy) over Railway **private networking** (`*.railway.internal`)
instead of aimock's **public** `*.up.railway.app` host.
Railway bills traffic to a public domain as **egress even
intra-project**, while `*.railway.internal` private networking is
**free** and **env-scoped**. The 240-concurrent-browser harness fleet
drives every demo continuously, so every LLM SSE stream from aimock back
to a demo backend is currently billed egress.
- aimock ≈ **89% of showcase egress**, ≈ **92% of the 13TB→78TB/mo
increase**.
- Estimated impact: avoids the ≈ **$602/mo → $3,856/mo** growth on the
aimock path.
## Change (config-only, reversible; SSOT-driven)
1. **SSOT** (`showcase/scripts/railway-envs.ts`): add an env-scoped
`internalDomain: "showcase-aimock.railway.internal"` to the aimock entry
in **both** envs. The public `domain` is **kept** (health probes /
external reachability).
2. **Emitter** (`showcase/scripts/emit-railway-envs-json.ts`): emit
`internalDomains` (additive, after `domains`) into the generated JSON.
Every non-aimock service keeps its frozen shape.
3. **Generated JSON** regenerated (oxfmt-canonical; 4-line additive
diff, only the aimock entry).
4. **Promote preflight** (`showcase/bin/railway`): `ssot_target_host`
now **prefers** the private `internalDomains[env]` over the public
`domains[env]`, so the Stage-2 (U5) serviceRef assertion requires demo
backends' `OPENAI_BASE_URL`/etc. to point at the private host.
Non-aimock targets (no `internalDomains`) fall back to their public host
unchanged.
5. **Harness** wiring probe needs **no code change** (it matches on
hostname); added a discriminating test pair + updated the drift-alert
Fix text to the private host.
**Target:** aimock binds `0.0.0.0:4010` (per
`showcase/aimock/RAILWAY.md`); demo backends resolve to
`http://showcase-aimock.railway.internal:4010`.
Deployed env vars, both envs (before → after):
| key | before (public, billed egress) | after (private, free) |
|---|---|---|
| `OPENAI_BASE_URL` | `https://<aimock>.up.railway.app/v1` |
`http://showcase-aimock.railway.internal:4010/v1` |
| `ANTHROPIC_BASE_URL` | `https://<aimock>.up.railway.app` |
`http://showcase-aimock.railway.internal:4010` |
| `GOOGLE_GEMINI_BASE_URL` | `https://<aimock>.up.railway.app` |
`http://showcase-aimock.railway.internal:4010` |
| `AIMOCK_URL` | `https://<aimock>.up.railway.app` |
`http://showcase-aimock.railway.internal:4010` |
`<aimock>` = `aimock-staging` (staging) / `showcase-aimock-production`
(prod). `railway.internal` is env-scoped, so staging demos reach the
staging aimock and prod demos reach prod aimock automatically — the same
private DNS name in both envs.
---
## Red-green proof (verbatim)
### RED — live staging today (billed public egress)
Deployed `showcase-langgraph-fastapi` (staging, service `06cccb5c-…`)
via Railway `variables(...)` GraphQL:
```
OPENAI_BASE_URL = https://aimock-staging.up.railway.app/v1
ANTHROPIC_BASE_URL = https://aimock-staging.up.railway.app
GOOGLE_GEMINI_BASE_URL = https://aimock-staging.up.railway.app
AIMOCK_URL = https://aimock-staging.up.railway.app
```
Pre-fix generated JSON aimock entry — **no** `internalDomains`:
```json
{ "domains": { "staging": "aimock-staging.up.railway.app",
"prod": "showcase-aimock-production.up.railway.app" },
"internalDomains": "ABSENT" }
```
### RED — Ruby U5 serviceref resolver, with the resolver reverted to
public-only
The three new U5 tests FAIL when `ssot_target_host` returns the public
host:
```
7 runs, 15 assertions, 3 failures
1) test_serviceref_prod_pointing_at_public_aimock_host_refuses:
expected REFUSE for prod serviceRef on the public egress host, got []
2) test_serviceref_prod_pointing_at_private_aimock_passes:
prod private aimock ref must not REFUSE, got ["REFUSE: §5.2 (showcase-ag2): prod
OPENAI_BASE_URL="http://showcase-aimock.railway.internal:4010/v1" does NOT point at
aimock's env-LOCAL prod host "showcase-aimock-production.up.railway.app" ..."]
3) test_ssot_target_host_prefers_internal_over_public:
expected "showcase-aimock.railway.internal",
actual "showcase-aimock-production.up.railway.app"
```
### GREEN — after the fix
Post-fix generated JSON aimock entry:
```json
{ "domains": { "staging": "aimock-staging.up.railway.app",
"prod": "showcase-aimock-production.up.railway.app" },
"internalDomains": { "staging": "showcase-aimock.railway.internal",
"prod": "showcase-aimock.railway.internal" } }
```
Ruby U5 serviceref tests (fixed resolver — prefers `internalDomains`):
```
7 runs, 20 assertions, 0 failures, 0 errors, 0 skips
```
Full Ruby spec suite:
```
184 runs, 715 assertions, 0 failures, 0 errors, 0 skips
```
Harness aimock-wiring probe (hostname-match; internal host with `:4010`
+ `/v1` → green, demo still on public host while harness on private →
red):
```
src/probes/aimock-wiring.test.ts 29 passed (was 27; +2 new: internal-host green, public-host drift red)
src/probes/drivers/aimock-wiring.test.ts 16 passed
src/rules/rule-loader.test.ts 61 passed (aimock-wiring-drift.yml parses after Fix-text update)
renderer + render-red-tick + orchestrator 157 passed (no alert-text snapshot broke)
```
Scripts test suite (emitter golden + everything): `2147 passed, 7
skipped` (one pre-existing `/tmp` lockfile flake in
`integration-smoke-registry.test.ts`, green on rerun after clearing the
stale lock). `emit --check` idempotent + oxfmt-canonical. Harness `tsc
--noEmit`: clean.
### GREEN — live infra confirmation
- aimock **staging** deployment status = `SUCCESS` (running), binds
`0.0.0.0:4010` — so `showcase-aimock.railway.internal:4010` resolves to
a live listener for any peer in the staging env.
- aimock serving LLM-shaped responses on `:4010`: `GET /health` → `200`;
`GET /v1/models` → `200` `{gpt-4o, gpt-4o-mini}`.
## What was vs wasn't live-validated
**Validated live:** the RED (deployed staging vars still on the public
egress host); aimock staging is deployed/running and serving on `:4010`;
the full unit/wiring/promote-preflight test surface passes with the new
internal-host values.
**NOT live-validated in-session:** the in-Railway-network DNS resolution
of `showcase-aimock.railway.internal:4010` from a peer service, and a
full staging deploy that flips the four keys + redeploys a demo backend.
Reason: the in-network vantage needs `railway ssh` (requires registering
a persistent account SSH key — a stateful, human-gated change I declined
to make unsupervised) or a staging deploy (the local Railway access
token was expired; the CLI refreshed it for read/GraphQL but a deploy is
a separate gated action). Railway private networking
(`*.railway.internal`) is a standard platform feature; the local
`docker-compose.local.yml` already runs the identical
`http://aimock:4010` internal-host pattern, and the wiring probe's
hostname match is exercised by the new tests. The staging deploy +
in-network curl is the first step of the rollout plan below and must be
run before prod.
## Irreducible egress remains
This does **not** zero showcase egress. Still billed: real browse users
hitting the public demo/shell domains; and aimock in **record mode**
proxying to real providers (the outbound prompt to
OpenAI/Anthropic/Google still bills).
## Rollout plan (reversible config change, staging-first, user-gated)
1. Land this branch (SSOT + generated JSON + assertions).
2. **Staging first:** set the four keys on staging demo backends +
`AIMOCK_URL` on the harness to
`http://showcase-aimock.railway.internal:4010` (`/v1` on
`OPENAI_BASE_URL`); redeploy one demo backend + aimock; from inside a
staging service curl
`http://showcase-aimock.railway.internal:4010/health` (expect 200) and
run a real demo LLM turn / aimock-wiring probe (expect green); confirm
the aimock egress path stops accruing
(`usage(measurements:[NETWORK_TX_GB])`).
3. **User-gated** promote to prod (staging→prod), same key flip.
4. **Rollback** = flip the keys back to the public host (no code revert
needed).
## Follow-ups (out of scope — do NOT bundle)
- Fleet right-sizing (240-concurrent-browser harness).
- `OPENAI_API_KEY` consolidation.
---
Draft — do not merge. Do not deploy to prod.
## What broke
The showcase control-plane harness crash-looped at boot on `origin/main`
HEAD, breaking the staging auto-deploy:
```
BOOT_ERR ERR_MODULE_NOT_FOUND Cannot find module '.../dist/shared/cell-model/live-status' imported from .../dist/shared/cell-model/cell-model.js
```
## Root cause
The harness ships as **pure Node ESM**:
- `showcase/harness/package.json` has `"type": "module"`
- build is `tsc -p tsconfig.build.json` with `moduleResolution:
"bundler"`, which **preserves extensionless import specifiers at emit**
(it does not rewrite `./live-status` → `./live-status.js`)
- the Docker `CMD` and the `start` script both run `node
dist/orchestrator.js`
Under pure Node ESM, relative import specifiers **must** carry the `.js`
extension. The harness already honors this convention everywhere —
`orchestrator.ts` has 79 relative imports, 79/79 ending in `.js`.
The relocated `showcase/harness/src/shared/cell-model/` fold broke the
convention: `cell-model.ts`, `live-status.ts`, `staleness.ts`, and the
equivalence fixtures/test imported their siblings extensionless
(`"./live-status"`, `"./staleness"`, etc). `tsc`, `vitest`, and `tsx`
all resolve extensionless specifiers fine, so it built green and passed
every test — then `node dist/orchestrator.js` threw
`ERR_MODULE_NOT_FOUND` at boot and crash-looped.
## The fix
Add the `.js` extension to every offending relative import in the
cell-model fold, matching the harness convention. **No tsconfig change**
(deliberately not switching to `nodenext`) — the minimal,
convention-matching fix is the extensions.
Files fixed (7 relative import specifiers across 5 files):
- `cell-model.ts` — 4 specifiers (`./live-status` ×3, `./staleness` ×2
counting the re-export)
- `live-status.ts` — `./format-ts`, `./staleness`
- `staleness.ts` — `./live-status`
- `cell-model.equivalence-fixtures.ts` — `./live-status` ×2,
`./cell-model`, `./staleness`
- `cell-model.equivalence.test.ts` — `./cell-model`,
`./cell-model.equivalence-fixtures`
The shell-dashboard re-export shims
(`showcase/shell-dashboard/src/lib/{cell-model,live-status,staleness,format-ts}.ts`)
were intentionally **left unchanged**: that package is a Next.js build
(not `node dist`) and uses extensionless relative imports as its own
convention, including reaching into harness `src`. Adding `.js` there
would break it.
## Local RED / GREEN proof
Same `node dist/orchestrator.js` boot probe, before and after the fix.
**RED** (unmodified branch code, after `pnpm --filter
@copilotkit/showcase-harness build`):
```
BOOT_ERR ERR_MODULE_NOT_FOUND Cannot find module '/…/showcase/harness/dist/shared/cell-model/live-status' imported from /…/showcase/harness/dist/shared/cell-model/cell-model.js
```
**GREEN** (after the fix, rebuilt):
```
BOOT_OK
```
The orchestrator module graph now loads fully — `ERR_MODULE_NOT_FOUND`
is gone.
## Docker boot result
Built the harness image locally from repo root (`docker build -f
showcase/harness/Dockerfile -t harness-esm-test .`) — build succeeded
(exit 0). Ran the container (`docker run harness-esm-test`, no env
supplied). It did **not** crash-loop on module resolution: the full ESM
module graph loaded and boot reached `bootFleet` in `orchestrator.js`,
then exited cleanly on the expected missing-env application error — **no
`ERR_MODULE_NOT_FOUND`**:
```
{"level":"error","msg":"showcase-harness.boot-failed","err":"HARNESS_ROLE must be set to one of: control-plane, worker (got: <unset>). ...","stack":"Error: HARNESS_ROLE must be set ...
at resolveFleetRoleConfig (file:///app/dist/fleet/role-config.js:72:15)
at bootFleet (file:///app/dist/orchestrator.js:3680:20)
at file:///app/dist/orchestrator.js:3762:5
at ModuleJob.run (node:internal/modules/esm/module_job:343:25)"}
```
Every `dist/` module resolved (note
`file:///app/dist/fleet/role-config.js` and
`file:///app/dist/orchestrator.js`); the crash was the intended
env-validation guard, exactly the "later runtime error is acceptable"
case. On pre-fix code this same container would have thrown
`ERR_MODULE_NOT_FOUND` at import time and crash-looped.
## CI regression guard
CI missed this because `tsc` (bundler resolution), `vitest`, and `tsx`
all resolve extensionless specifiers — **no existing CI step ever ran
the real `node dist` module graph** that the container boots.
Added a **harness ESM boot-smoke** step to the `Showcase: Validate` job
(already gated on `showcase/harness/**`, a required PR check): after
building the harness dist, it loads `dist/orchestrator.js` via `node
import()` and fails hard on `ERR_MODULE_NOT_FOUND`. A later runtime
error from missing env/PocketBase is expected and passes — only a
module-resolution failure reddens the build.
**Guard red-green** (the guard's own proof, run locally against both
code states):
- **pre-fix** (extensionless imports): `BOOT_FAIL: ERR_MODULE_NOT_FOUND
…/dist/shared/cell-model/live-status` → exit **1** ✅ caught
- **post-fix** (with `.js`): `BOOT_OK: orchestrator module graph loaded`
→ exit **0** ✅ passes
## Verification
- `pnpm typecheck` / `tsc --noEmit`: clean
- `cell-model.equivalence.test.ts`: 7/7 pass
- lefthook lint + commitlint: pass on both commits
Do not merge — merge is user-gated after code review.
The boot-smoke previously failed only when the thrown error carried a
module-RESOLUTION code (ERR_MODULE_NOT_FOUND + siblings, walked through
the cause chain / AggregateError members) and passed everything else.
That defaults-to-pass on module-EVALUATION crashes — a top-level throw,
an await-rejection, a bad named binding, or a SyntaxError — none of which
carry a resolution code, so a real boot-crashing regression of that class
would ship green.
The smoke runs `node -e "import('./dist/orchestrator.js')"` with
process.argv[1] UNSET, so bootFleet() (the env/PocketBase validation that
legitimately throws) never runs — only the module graph is linked and
evaluated. A clean build therefore loads with no thrown error, so ANY
error thrown by import() here is a boot regression and must fail the gate.
Now: any rejection -> BOOT_FAIL / exit 1 (resolution AND
evaluation/link/binding/syntax/top-level-throw). Successful load ->
BOOT_OK / exit 0. The collectErrorCodes cause/AggregateError walk is
retained ONLY to label the failure ("module-resolution failure" vs
"boot/evaluation failure") — both exit 1, richer diagnostics preserved.
Kept process.exit(0) on success, the timeout 120s wrapper, and
timeout-minutes: 5 (a hang still fails).
No-false-red proof: built the real harness dist and ran the strict guard
against the real dist/orchestrator.js under node -e (argv[1] unset) —
BOOT_OK, exit 0, ~0.28s, no hang, confirming a clean graph loads without
throwing and the strict guard does not false-red real CI.
The boot-smoke gate classified pass/fail using only the top-level `e.code`.
A module-resolution error that arrives WRAPPED — nested in `e.cause`
(possibly a chain), bundled inside an `AggregateError` (`e.errors[]`), or
rethrown without preserving `.code` at the top — showed no code to the
`MODULE_RESOLUTION_CODES.has(e.code)` check and was misclassified as
BOOT_OK, defeating the gate.
Add a `collectErrorCodes` helper that gathers every code reachable from
the thrown error: the error itself, its cause chain (recursively), and any
AggregateError members (recursively), with a depth cap to bound cause
cycles. If ANY collected code is a module-resolution code -> BOOT_FAIL /
exit 1. Purely additive to the FAIL set: direct top-level codes still
redden, and a benign non-resolution runtime error (e.g. the
`HARNESS_ROLE must be set` guard, which carries no such code anywhere)
still passes as BOOT_OK / exit 0. The `timeout 120s` wrapper,
`timeout-minutes: 5`, and success/expected-error `process.exit(0)` are
unchanged.
Local red-green (classifier extracted to a temp file, driven against
synthetic errors):
- RED (top-level-only): wrapped cause -> BOOT_OK exit 0 (swallowed);
AggregateError member -> BOOT_OK exit 0 (swallowed).
- GREEN (hardened): wrapped -> exit 1; aggregate -> exit 1; direct
ERR_MODULE_NOT_FOUND -> still exit 1; benign ERR_INVALID_ARG_TYPE and
HARNESS_ROLE error -> BOOT_OK exit 0; real built dist/orchestrator.js ->
BOOT_OK exit 0 in <200ms (prompt exit, no hang).
The aimock-wiring probe matches on hostname, so it needs no code change for
the private-networking migration. Add a discriminating test pair proving the
internal host (http://showcase-aimock.railway.internal:4010, with :4010 port
and /v1 suffix) resolves green while a demo still on the public egress host
goes red. Update the aimock-wiring-drift.yml Fix text to point operators at
the private host instead of the public production URL.
ssot_target_host now prefers the env-scoped internalDomains host over the
public domains host, so the Stage-2 (U5) serviceRef assertion expects demo
backends' OPENAI_BASE_URL/etc. to point at
http://showcase-aimock.railway.internal:4010 (free intra-env networking)
rather than the billed public egress host. Non-aimock targets (no
internalDomains) fall back to their public host unchanged.
Red-green: reverting the resolver makes the three new U5 tests fail (public
host asserted); restoring makes them pass. Full Ruby spec suite green (184
runs, 715 assertions, 0 failures).
Add an env-scoped `internalDomain` (showcase-aimock.railway.internal) to the
aimock SSOT entry in both envs and emit it as `internalDomains` in the
generated JSON. Railway bills public *.up.railway.app traffic as egress even
intra-project, while *.railway.internal private networking is free and
env-scoped. aimock is ~89% of showcase egress; routing the ~20 demo backends'
LLM traffic at the private host over http://showcase-aimock.railway.internal:4010
eliminates the billed path. The public `domain` is retained for health probes.
Serviceref host resolution + assertions to follow in subsequent commits on
this branch.
The boot-smoke step only treated ERR_MODULE_NOT_FOUND as failure, so other
module-resolution regressions (ERR_UNSUPPORTED_DIR_IMPORT,
ERR_PACKAGE_PATH_NOT_EXPORTED, ERR_UNKNOWN_FILE_EXTENSION,
ERR_INVALID_MODULE_SPECIFIER) were swallowed as BOOT_OK/exit 0 — the very
class of bug this gate exists to catch could slip through. It also had no
process.exit(0) on the success/expected-error paths and no bounded timeout,
so a future open handle at import time could hang node -e to the job's
25-minute ceiling.
- Broaden the failure condition to a MODULE_RESOLUTION_CODES set (any of the
five codes => BOOT_FAIL, exit 1). Non-module-resolution runtime errors
(e.g. the HARNESS_ROLE env guard, no such code) stay BOOT_OK/exit 0.
- Add explicit process.exit(0) on both the success and expected-error paths.
- Wrap the node invocation in `timeout 120s` (non-zero on timeout => step
fails) and add step-level timeout-minutes: 5.
Red-green proof (extracted guard logic vs synthetic modules): current logic
passes ERR_UNSUPPORTED_DIR_IMPORT / ERR_PACKAGE_PATH_NOT_EXPORTED at exit 0
(RED gap); hardened logic fails all five codes at exit 1, keeps benign
runtime error at exit 0, and against the real built dist/orchestrator.js
reports BOOT_OK and exits promptly (779ms, no hang).
CI missed the extensionless-import regression because tsc (bundler
resolution), vitest, and tsx all resolve extensionless relative
specifiers fine — no existing step ever ran the real node dist module
graph, which is what the container actually does at boot.
Add a boot-smoke to the Validate Showcase job (already gated on
showcase/harness/**): after building the harness dist, load
dist/orchestrator.js via a node import() and fail hard on
ERR_MODULE_NOT_FOUND. A later runtime error from missing env/PocketBase
is expected and passes — only a module-resolution failure reddens the
build. Verified red-green: the guard exits 1 on the pre-fix
extensionless imports and 0 once the .js extensions are added.
The harness runs as pure Node ESM (package.json "type":"module", built
with tsc moduleResolution:"bundler" which preserves extensionless import
specifiers at emit, launched via node dist/orchestrator.js). Under pure
Node ESM, relative import specifiers must carry the .js extension — a
convention the harness already honors everywhere (79/79 relative imports
in orchestrator.ts end in .js).
The relocated shared/cell-model fold broke that convention: cell-model.ts,
live-status.ts, staleness.ts, and the equivalence fixtures/test imported
sibling modules extensionless ("./live-status", "./staleness", etc). tsc,
vitest, and tsx all resolve those fine, so it built and tested green — but
at container boot node threw ERR_MODULE_NOT_FOUND on
dist/shared/cell-model/live-status and crash-looped the orchestrator,
breaking the staging auto-deploy.
Add the .js extension to every offending relative import to match the
harness convention. Minimal fix — no tsconfig change.
## Problem
Docs-only showcase runs build successfully, but result aggregation fails
with zero `build-result-*` directories. That prevents the Railway
staging redeploy and posts a misleading build-failure alert.
## Why
After `actions/download-artifact` was upgraded from v4 to v8, a pattern
matching exactly one artifact is extracted directly into the configured
directory as `result.json`. The aggregator only accepted the
multi-artifact layout with nested `build-result-*/result.json` files.
## Fix
- Read the root-level `result.json` when no nested artifact directories
exist.
- Keep the existing fail-loud behavior when neither layout is present.
- Add a regression test for the single-artifact layout.
## What
A harness-native monitor that pages **#oss-alerts** when a whole
integration column collapses to **red-D0** ("completely gone" / backend
unreachable) in **production** — the incident class the per-cell alert
rules miss. On 2026-07-13 LGT went fully gone in prod and nothing paged.
Runs in-process on the control-plane on its own `*/15` cron (prod-only,
kill-switchable), reusing the same #oss-alerts webhook + shared
family-summary the family-silence monitor uses.
## Design — single verdict, no re-derivation
Detection runs the dashboard's **own `buildCellModel` fold** (the shared
`showcase/harness/src/shared/cell-model/` module both the dashboard and
the monitor import — relocated in the first commit of this PR) over the
same PocketBase `status` rows, then applies a column-gone predicate over
the resulting `CellModel` fields. Because it is literally the same pure
fold over the same rows, the monitor's per-cell verdict **equals the
DepthChip the dashboard renders, by construction** — no parallel
interpretation that can drift.
- `columnGone` = over wired+supported cells, `every(achievedDepth===0 &&
chipColor==="red" && !isStaleCell && surfaceState ∉
{unreachable,pending})`.
- Wired-cell enumeration mirrors the dashboard `page-stats` iteration
(the generator's `determineCellStatus` rule), read from the generated
`registry.json`.
## Behavior (per spec)
- **Producer-liveness SUSPENDED gate (F1)** — if the fleet producer is
idle/paused (the LGT mitigation state), the tick holds ALL state: no
OPEN, no CLOSE, no false recovery. Reuses the family-silence
inflight-aware `/api/runs` reasoning; idle window = 3× the longest
resolved producer period.
- **60s confirm re-read** (never a re-probe — re-probing a sick pool
deepens the incident); OPEN only if both scans agree; blips logged, not
paged.
- **15m-detect vs 1h-repost** state machine; CLOSE requires **positive
fresh-healthy** evidence, not mere absence-of-gone.
- **ONE aggregated** outage message + consolidated recovery notice;
`lastAlertAt` advances only after a successful Slack send (dedupe
discipline).
- Durable per-slug `{sinceAt,lastAlertAt}` JSON map in `alert_state`
(`getSet`/`putSet`).
- Prod-only gate: `SHOWCASE_ENV ?? RAILWAY_ENVIRONMENT_NAME ===
"production"` + `PROD_D0_MONITOR_ENABLED` kill-switch,
control-plane-only.
## Red-green proof
**No-divergence (the load-bearing test).** A frozen **test-only**
`naiveGone` (`achievedDepth===0` alone, ignoring color/staleness) is the
anti-example. On committed fixtures:
- **RED** — with the real predicate degraded to the naive depth-only
rule, the GREEN assertions fail (the naive rule wrongly labels the
**gray-D0-no-data** and **stale** columns as gone):
```
× GREEN: the real columnGone predicate fires ONLY on the red-D0-fresh
column
Expected false / Received true (stale + gray-D0 columns mislabeled)
Tests 3 failed | 3 passed (6)
```
- **GREEN** — with the real predicate, it fires ONLY on the red-D0-fresh
column and its per-cell inputs equal `buildCellModel`'s own outputs:
```
✓ src/fleet/control-plane/d0-gone-predicate.test.ts (6 tests)
```
**Producer-idle SUSPENDED (F1) proven load-bearing.** Disabling the gate
flips BOTH F1 tests red:
```
× RED (invisible-outage): idle producer + FRESH-gone rows → SUSPENDED (no OPEN)
× RED (false-recovery prevention): open outage, producer pauses, rows spuriously read healthy → NO recovery, HOLD
Tests 2 failed | 13 passed (15)
```
With the gate present: `Tests 15 passed (15)`.
**Full suite (post-fix):**
- Harness `tsc --noEmit`: **exit 0** · production build (`tsc -p
tsconfig.build.json`): **exit 0**
- Shell-dashboard `tsc --noEmit`: **exit 0** (Phase-1 gate — added the
required `isStaleCell`/`observedAtAgeMs` fields to the CellModel test
literal)
- New tests: predicate (6) + monitor behavioral (15) + registration gate
(7) + equivalence (7) = **35 passed**
- All control-plane tests: **382 passed**; orchestrator tests: **133
passed**
## Files
- `d0-gone-predicate.ts` (+test) — pure
`cellGone`/`columnGone`/`columnFreshHealthy` + registry-derived
wired-cell enumeration
- `d0-gone-monitor.ts` (+test, +gate test) — `createD0GoneMonitor`
factory
- `orchestrator.ts` — `internal:prod-d0-gone-monitor` cron registration
(prod-gated)
- `shell-dashboard/…/unified-cell.test.tsx` — CellModel literal fix
(Phase-1 tsc gate)
## Deferred / not in this PR
- **Live staging red-green (§10.8)** — the mandatory producer-paused
staging proof (induce a real comm-error, observe detect+confirm+post,
then pause `FLEET_PRODUCER_CRON` and assert no false recovery). Requires
a live staging harness + test webhook; not run here. **Should be
executed before undraft/rollout.**
- **#5872 field literals**
(`global-error-promotion`/`spec-failed`/`greenCount`) are HARD-GATED on
#5872 merge — the monitor consumes `buildCellModel` output today and
inherits #5872's tightened inputs per-slug automatically; no hard
dependency.
DRAFT — per-slug rollout and undraft/merge are the user's call.
The shell-dashboard Docker build context (repo root) copies scripts,
shared, integrations, shell-docs content, and shell-dashboard — but not
showcase/harness. The dashboard's src/lib/{cell-model,live-status,
staleness,format-ts}.ts re-export barrels forward to
../../../harness/src/shared/cell-model/*, so the isolated next build
failed with Module-not-found (CI build-check (shell-dashboard)). Local
next build passed only because the full monorepo is present.
Copy just the 4-file cell-model fold to the exact relative path the
shims expect, preserving the single-shared-fold invariant (harness
monitor still imports the same canonical copy) without dragging the
whole harness package into the dashboard image.
## Summary
This PR updates thread import documentation and threads navigation
across shell-docs, including a follow-up that makes the supported
CLI-only import journey explicit.
## What changed
- Adds thread import documentation:
- Root generic guide at `/threads-import` for frameworks without a
source-specific importer page.
- Google ADK-specific guide at `/google-adk/threads-import`.
- LangGraph-specific guide at `/langgraph-python/threads-import`, plus
generated LangGraph framework routes through the framework resolver.
- Authored framework wrappers so Mastra, AG2, Agno, Built-in Agent,
CrewAI Flows, DeepAgents, LlamaIndex, Microsoft Agent Framework, and
PydanticAI can surface the generic guide.
- Updates the CLI and import journey:
- Describes the CLI as supporting both cloud-hosted and self-hosted
Enterprise Intelligence.
- States up front that import runs from an app created with the
CopilotKit CLI and Enterprise Intelligence enabled.
- Clarifies that import uses the project already selected for the
current directory.
- Shows the source import commands before the optional `project select`
explanation, while still directing users to change targets before
running the dry run.
- Removes the suggestion that `skills onboard` plus `project select` can
add Enterprise Intelligence to an arbitrary existing app.
- Keeps the CLI import section concise and links to the complete generic
and source-specific guides.
- Clarifies future thread continuity:
- Threads Drawer uses the shared `CopilotChatConfigurationProvider`, so
selecting a thread updates the active `threadId` without separate state
wiring.
- Headless Threads uses `useThreads`; the app stores the selected
`thread.id` and passes it to the chat component as `threadId`.
- ADK and LangGraph guides retain source-specific stable ID mapping
guidance, including the LangGraph UUID caveat.
- Removes source-guide links that rewrote to the current framework page
and created circular navigation.
- Adds and organizes Threads Drawer docs:
- Extracts shared Threads Drawer content into a reusable snippet.
- Adds root and authored-framework wrapper pages.
- Moves Threads Drawer into a new expanded Threads nav grouping
alongside Headless Threads and Import Thread History.
- Renames and reorganizes threads docs:
- Relabels the existing `Threads` guide as `Headless Threads`.
- Removes the root prebuilt-components nav entry for Threads Drawer so
generated frameworks do not show it in two places.
- Updates root, authored, generated, and Built-in Agent nav metadata so
the Threads grouping is consistent.
- Updates nav tests to recognize pages nested inside authored-framework
groups.
## Validation
Run from `showcase/shell-docs`:
- `npm run lint` passes with the existing repository warning set.
- `npm run typecheck` passes.
- `npm run test` passes: 32 files and 167 tests.
- `npm run build` passes and generates 214 static pages; it reports the
existing Turbopack NFT tracing warning.
- Rendered and link-checked locally:
- `/cli`
- `/threads-import`
- `/mastra/threads-import`
- `/google-adk/threads-import`
- `/langgraph-python/threads-import`
- Root and framework-specific Threads Drawer and Headless Threads links
## Redirects
No redirect URLs were added or required:
- `Threads` was relabeled to `Headless Threads`, but the slug remains
`/threads`.
- `CopilotThreadsDrawer` was relabeled to `Threads Drawer`, but the URL
remains `/prebuilt-components/copilot-threads-drawer` and framework
equivalents.
- Threads Drawer moved in navigation, but the route did not move.
- `Import Thread History` is new at `/threads-import` and framework
routes such as `/mastra/threads-import`, `/google-adk/threads-import`,
and `/langgraph-python/threads-import`, so there is no prior URL to
redirect.
- The generated-framework routing change only selects framework-specific
content for the new `threads-import` slug.
- recoveryMessage multi-slug branch now wraps sinceAt in renderSince() so a
corrupt-but-shaped persisted sinceAt renders "unknown", not raw garbage
(matches outage path and single-slug recovery guard).
- summary.get() read failure now logs ERROR with errorId d0-monitor-summary-read
instead of a low-signal WARN (silently blinds the detector, same family as the
other silent-disable guards).
- classifyProducer inflight short-circuit now also requires anyWorkerOnline, so a
stale/orphaned inflight from a dead worker cannot force a blind live scan.
- Extended C6 recovery test to cover the multi-slug arm; split the inflight
predicate test into online/offline-worker cases; fixed C1(ii) rotation comment.
Fixes#5775.
## Problem
After #5099 scoped the streamdown markdown/table styles under
`[data-copilotkit] [data-streamdown="…"]`, the **table action controls**
(copy / download) are still unstyled for hosts that import
`@copilotkit/react-core/v2/styles.css` but don't also ship streamdown's
raw Tailwind utilities. streamdown renders the controls row, per-button
wrappers, trigger buttons, dropdown popovers and menu items with
unprefixed utilities (`flex`, `items-center`, `justify-end`, `gap-1`,
`cursor-pointer`, `p-1`, …) and **no stable `data-streamdown`
attribute**, so CopilotKit's packaged CSS didn't cover them — the
controls rendered as vertically stacked plain icons instead of a
right-aligned row.
## Fix
Add scoped fallback selectors under `[data-copilotkit]
[data-streamdown="table-wrapper"]`, targeting the controls chrome
**structurally** (since it has no `data-streamdown` hook):
- controls row → `> div:first-child:not(:last-child)` (flex,
right-aligned, gap)
- per-button wrapper → `… > div` (relative, positions the popover)
- trigger buttons → `… > div > button` (matches the code-block
copy/download button styling)
- dropdown popover → `… > div > div`
- popover menu items → `… > div > div > button`
The controls row is `table-wrapper`'s first child **only when controls
are enabled**; `:not(:last-child)` leaves a control-less table (whose
single child is the scroll container, already styled by #5099)
untouched.
## Verification
- **Compiles.** Built `globals.css` through the Tailwind v4 CLI — every
`@apply` resolves (e.g. `bg-background` → `var(--background)`,
`shadow-lg` → the shadow vars, `min-w-[120px]` → `min-width:120px`) and
all five rules emit with correct values.
- **Selectors match the real DOM.** A new DOM test
(`streamdown-table-controls.test.tsx`) renders a real `<Streamdown>`
table and asserts the controls row is `table-wrapper`'s first-non-only
child, carries no `data-streamdown` attribute, and contains the trigger
buttons under `> div > button` — i.e. the scoped selectors target real
elements. This also guards against streamdown markup drift.
- **Selector presence** guarded by `streamdown-styles.test.ts`
(whitespace-robust).
- Full `styles/__tests__` suite green; `oxlint`/`oxfmt` clean.
## Note (out of scope, discovered while fixing)
At runtime in streamdown `1.6.11` the `<table>` element is stamped
`data-streamdown="table-wrapper"` (not `"table"`): `MarkdownTable`
passes `data-streamdown="table-wrapper"` as a prop that leaks through
`...rest` onto the `<table>`, overriding the intended `"table"`. So
#5099's `[data-streamdown="table"]` selector currently matches nothing,
and the table also matches the `table-wrapper` rules. The controls fix
here is unaffected (its child-combinator selectors don't match the
table's `thead`/`tbody` children), but the `[data-streamdown="table"]`
selector is worth a separate follow-up / upstream report.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Fixes#5819.
## Problem
An MCP app `ui/message` follow-up is queued (via the module-level MCP
request queue) until the agent is idle. The queued closure ran
`copilotkit.runAgent({ agent })` against the **shared registry agent** —
reading `agent.threadId`/`agent.messages` at *execution* time, not
enqueue time.
CopilotKit keeps a single shared agent per id and mutates its
`threadId`/`messages` on a thread switch (`CopilotChat.tsx` sets
`agent.threadId = resolvedThreadId` and `setMessages(...)`). So if the
host switches threads while a follow-up is queued (agent busy), the
follow-up — and its streamed events — execute against and render into
the **now-foreground thread**. That's the leak in #5819.
### Why this is a regression
This class of bug was previously prevented by the per-thread clone
routing in the activity renderers (#3630); commit `762370a4e5`
**deliberately** reverted that ("remove per-thread agent cloning,
restore single registry agent per id"), re-opening CPK-7155/#2957.
Re-introducing implicit clones would undo that decision.
## Fix — skip stale follow-ups
Capture the thread the follow-up is enqueued for, and at execution
compare it against the shared agent's current thread
(`ɵrunMcpFollowUp`):
- **Same thread** (common case) → run on the shared agent, unchanged
(still streams into the visible chat).
- **Thread changed** → the shared agent has moved on and can no longer
run the follow-up in its originating thread's context, so **drop it**
(with a warning) rather than leak it into the current thread. The MCP
app already received its `ui/message` ack at enqueue time; only the
optional agent turn on an abandoned thread is skipped.
### Why not re-home the run to the original thread?
An earlier revision re-homed the stale run onto an isolated
`registerProxiedAgent` proxy pinned to the origin thread. Review (thanks
@tylerslaton) surfaced two P1s that are inherent to *running* a stale
follow-up, both confirmed:
1. **Tools/context leak.** `buildFrontendTools`/`getContextForAgent`
include *unscoped* entries (the `useFrontendTool`/`useAgentContext`
default) for any agent id, read live — so the background thread-A run
received thread-B's current context and could execute global frontend
handlers against B.
2. **Lost continuity.** With multiple queued follow-ups, a fresh
per-item proxy seeded from an enqueue-time snapshot meant run 2 never
saw run 1's result (the runtime replaces the clone with
`input.messages`/`input.state`, so persistence didn't recover it).
Both dissolve if the stale follow-up simply doesn't run. Skip-stale is
also far smaller and needs no proxy lifecycle, so it doesn't fight the
post-revert "single registry agent" architecture.
## Testing
- **TDD.** The e2e regression test reproduced #5819 on the pre-fix code
(`run` invoked once with the switched-to `threadId`) and passes after.
- **Unit** (`mcp-followup.test.ts`) — `ɵrunMcpFollowUp` runs on the
shared agent when the thread matches (incl. the `undefined`→`"default"`
case) and drops (no run, warns) when it changed.
- **e2e** (`MCPAppsUiMessage.e2e.test.tsx`) — "does not run a queued
ui/message follow-up against a thread the host switched to (#5819)".
- 30/30 across the MCP + follow-up suites; existing ui/message tests
unchanged. `tsc` 0 errors, `oxlint` 0 errors (my files), `oxfmt` clean.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
C1 (core): select the shown/named outage slugs by re-post-due-ness + rotation
instead of an alphabetical prefix slice, gate the aggregate post on a DUE slug
actually being named, and advance lastAlertAt only for named-and-due slugs. This
stops a wide (>maxSlugs) outage from re-posting every 15m (overflow slugs whose
clock never advanced stayed perpetually "due") and stops a newly-opened overflow
slug from forcing a per-tick re-post; every open slug is now named within a
bounded number of re-posts.
C2: derive outage onset from the gate-failing (non-green) winner rung, not only a
literal red row — a degraded winner no longer strands earliest at NaN and
re-stamps sinceAt to now.
C3: log a loud errorId when readStatusRows breaks on a finite totalPages while the
last page was full (short/inconsistent read) instead of silently truncating.
C4: floor repostMinutes at min 1 in resolveConfig (0 → repostMs 0 → every-tick
re-post).
C5: classifyProducer distinguishes a fresh deploy (workers online, no run history)
as "no-data / not-yet" from a paused "idle" producer; both HOLD (never page
without data) but the fresh case is no longer a misleading permanent SUSPEND.
C6: validate a persisted sinceAt is a parseable ISO before interpolating into the
outage message (renderSince) so a corrupt-but-shaped state blob renders "unknown",
not garbage.
C7: fix the mis-annotated `unsupported` fixture flag and add a symmetric assertion
pinning every naiveMislabels flag == (naiveGone != expectedGone).
Bucket-b: stamp the outage-duration line with evidenceMs (consistent with the
recovery post + lastAlertAt); skip the alert_state write on a pure no-op tick.
Addresses two P1s on #5940: the re-home path (a) injected unscoped frontend
tools/readable context into the background run — `buildFrontendTools`/
`getContextForAgent` include entries with no agentId, so the thread-A run
received thread-B's live context and could execute global frontend handlers
against B — and (b) lost continuity across multiple queued follow-ups (fresh
per-item proxy + enqueue-time snapshot, so run 2 never saw run 1's result).
Both stem from *running* the stale follow-up. Switch to skip-stale: when the
shared agent's threadId no longer matches the thread the follow-up was enqueued
for, drop it (with a warning) rather than run it against the now-foreground
thread. This removes the proxy/registerProxiedAgent machinery entirely and
resolves both P1s by construction. The MCP app still gets its ui/message ack at
enqueue time; only the optional agent turn on an abandoned thread is skipped.
Removes the re-home unit/integration tests; the e2e regression test (no
cross-thread run after a switch) and simplified unit tests cover the behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drives the real CopilotKitCore + RunHandler + ProxiedCopilotRuntimeAgent via
registerProxiedAgent against a mocked transport (mirrors
proxied-runtime-transport.test.ts). Asserts the re-homed run reaches the runtime
addressed to the ORIGINAL threadId (not the foreground one), carries the captured
message, runs on an isolated instance whose events never reach the shared agent,
and unregisters the transient proxy after. Closes the delegate/replay-lifecycle
gap the mocked-host unit tests couldn't reach — in CI, no live runtime needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Queued MCP app ui/message follow-up work executed against whatever thread the
shared registry agent pointed at when the queue drained. If the host switched
threads while the follow-up was queued (agent busy), the run — and its streamed
events — leaked into the now-foreground thread.
Capture the thread context at enqueue and route the follow-up through
ɵrunMcpFollowUp: same thread runs live on the shared agent (unchanged); a
changed thread re-homes the run onto an isolated registerProxiedAgent sibling
pinned to the original thread (own event stream, persists + reconciles on
return); a changed thread on a non-runtime agent drops the follow-up rather
than leaking it.
Regression from 762370a4e5 (revert of per-thread activity-renderer clone
routing, #3630); uses the sanctioned registerProxiedAgent primitive (#4629)
instead of reintroducing implicit clones.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After #5099 the table wrapper/cells are styled via [data-streamdown] selectors,
but the copy/download controls row, button wrappers, trigger buttons, dropdown
popovers and menu items render with raw Tailwind utilities and no stable
data-streamdown attribute — so hosts that import @copilotkit/react-core/v2
styles without shipping streamdown's own utilities saw them unstyled (icons
stacked vertically instead of a right-aligned row).
Add scoped fallback selectors under [data-copilotkit] [data-streamdown=
"table-wrapper"], targeting the controls chrome structurally. The controls row
is the first child ONLY when controls are enabled, so :not(:last-child) leaves a
control-less table (single child = the scroll container) untouched.
Verified against streamdown 1.6.11's actual rendered DOM (a new DOM test guards
that the structure the selectors assume still holds) and by compiling the CSS.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Central structural lever: derive every monitor state from a single per-cell
classifier (classifyCell → gone | healthy | unknown). Treat UNKNOWN (gray /
no-data / stale / amber / comm-error) as UNKNOWN everywhere — never "gone",
never positive-healthy.
- B-F1: recovery/CLOSE requires POSITIVE green cells (cellHealthy: chipColor
green, achievedDepth>=3, fresh), not the mere absence of red. A gone column
decaying to no-data no longer auto-recovers. Fixed the healthyRows test
fixture to emit a genuine green D5/D6 ladder.
- B-onset: derive sinceAt from the folded verdict's contributing ladder rows,
not a raw row.state==="red" re-scan (removed keyBelongsToSlug).
- B-A5gap: guard self-heal/loud-log on "no slug has any wired cell", not
map.size (every integration slug is keyed even with zero wired cells).
- B-env: normalize the prod gate (trim+lowercase, empty-as-unset) so an empty
SHOWCASE_ENV no longer shadows a prod Railway env and a mis-cased/padded
value no longer silently disables the monitor. Added resolveMonitorEnv +
shouldRegister; orchestrator + gate test both use the real predicate.
- B-flap: an already-open outage keeps its hourly re-post even when a later
confirm scan is inconclusive (confirm gates only OPEN and CLOSE).
- B-cadence: advance lastAlertAt only for slugs actually named in the message
(respect maxSlugsInMessage); overflow slugs keep their clock.
- Cheap: no-wired-cells logs once per tick; MAX_SLUGS floors at 1; lastAlertAt
stamps evidenceMs; resolveConfig negative/NaN/empty coverage.
Bucket-(a) fixes, each with a local red-green test:
- A1: replace the substring `:${slug}` onset match with an anchored
exact slug-segment match (`keyBelongsToSlug`) so a prefix-colliding
sibling (`strands` vs `strands-typescript`) no longer mis-attributes
the earlier sibling's red onset. RED: strands' sinceAt was pulled to
the strands-typescript onset; GREEN: each slug gets its own onset.
- A2: recovery/CLOSE is now SYMMETRIC with OPEN — a recovery requires a
second agreeing fresh-healthy read (confirm scan). RED: a single
transient healthy read fired a false "recovered"; GREEN: held until
two reads agree.
- A3: guard an empty/degenerate schedule set — longestPeriodMs 0/NaN
would make idleWindowMs 0 → isProducerLive permanently false → the
monitor SUSPENDS forever and never pages. Falls back to a 45m default
window (DEFAULT_IDLE_WINDOW_MS) and logs at error.
- A4: bound readStatusRows — guard NaN/undefined totalPages (a `page >=
NaN` break never trips) and add a hard MAX_STATUS_PAGES cap so a full
page + bad totalPages cannot infinite-loop/OOM. RED: OOM; GREEN:
terminates at the cap.
- A5: a registry-load failure logs at error with a stable errorId (not a
silent warn-once permanent no-op), and the monitor accepts a loader
thunk so it re-reads registry.json each tick while the wired-cell set
is empty — a transiently-missing file self-heals without a redeploy.
- A6 (verified, no code change): createSlackWebhookTarget already throws
on every non-2xx (4xx/5xx/429/3xx/network-exhausted); added a test
asserting the monitor does NOT delete recovery state when the post
throws.
Bucket-(b): stamp the recovery message with the confirm-scan instant
(evidenceMs) not tick-start; wrap the scheduler tick handler in a
catch+errorId; log the prod env-gate skip at warn with a reason; add a
clarifying comment that the aggregate lastAlertAt reset is intentional
one-message-one-clock cadence; simplify the three dashboard barrel-shim
comments (drop the rot-prone enumerated symbol lists).
Fixes#5801.
## Problem
`CopilotKitProvider` constructs the core **during render** (the `useRef`
init pattern), and the `CopilotKitCore` constructor synchronously starts
the `/info` network request. React can start-and-discard renders
(concurrent rendering / Suspense / StrictMode), and for a *mounting*
fiber `copilotkitRef.current` is `null` again on each discarded attempt
— so a brand-new core is constructed, and fires its own `/info`, per
discarded render. The reporter observed **70–80 `/info` requests on a
single page load** (production build, self-hosted runtime with real
latency).
Root causes (all confirmed on current `main`):
1. Core constructed during render — `CopilotKitProvider.tsx`.
2. Constructor kicks off network I/O — ctor → `setRuntimeUrl` →
`updateRuntimeConnection()` → `fetch(/info)`.
3. `updateRuntimeConnection()` had no in-flight guard.
The dominant multiplier is **cross-instance** (N discarded renders → N
cores → N fetches), so an in-flight guard alone can't fix it — the fetch
must not fire during construction.
## Fix
Separate **construction** (pure) from **connection** (network I/O), as
the issue recommends:
- **core** — new `deferInitialConnection` option: the constructor
records the runtime config (so `copilotkit.runtimeUrl` stays available
*synchronously* to hooks like `use-threads` / the CopilotChat context —
no first-render regression) but does **not** start the `/info` fetch. A
new `connect()` method starts the single connection; it's idempotent
(no-op unless status is `Disconnected`, which `updateRuntimeConnection`
flips to `Connecting` synchronously), so a StrictMode double-invoked
mount effect collapses to one request. `updateRuntimeConnection` also
gains an in-flight guard keyed by `url + requested transport` so
concurrent same-target calls de-dupe (a change to a different target
supersedes).
- **react-core** — the provider constructs with `deferInitialConnection:
true` and calls `copilotkit.connect()` from its existing commit-phase
mount effect. Renders discarded before commit never reach the effect, so
they never fetch. The subscribe effect runs before `connect()`, so the
status-change event is caught (no reliance on the catch-up read for the
common path).
**Backward compatible:** without `deferInitialConnection`, the
constructor still connects — Vue/Angular/vanilla consumers are
unaffected.
## Testing
- **TDD.** New core tests fail on the pre-fix code (constructor fetches)
and pass after.
- **core** (`core-defer-runtime-connection.test.ts`, 7 tests) — the
authoritative proof of the mechanism: deferred construct fires **zero**
`/info`; `runtimeUrl` still exposed synchronously; `connect()` → exactly
one `/info`; `connect()` idempotent across repeated calls; **N orphaned
cores that never `connect()` → 0 fetches, only the committed one
fetches** (directly models the discarded-render bug); in-flight de-dupe;
backward-compat.
- **react-core** (`CopilotKitProvider.deferWiring.test.tsx`) — asserts
the provider constructs with `deferInitialConnection: true` and drives
`connect()` from an effect; **this is the provider-level test that
distinguishes the fix from the bug** (it fails if the deferral wiring is
dropped — verified by reverting).
- **react-core** (`CopilotKitProvider.deferConnection.test.tsx`, 2
tests) — normal-mount regression guards: exactly one `/info` on mount,
and one under `React.StrictMode`.
*Scope note (found via adversarial review):* the raw dozens-of-`/info`
symptom only manifests with React *discarding* in-progress renders and
constructing multiple orphaned cores — Testing Library can't reproduce
that (a single committed mount yields one `/info` either way, and the
ctor's fetch is several microtasks deep so ordering can't distinguish
it). Hence the multi-instance proof lives in the core test above, and
the provider is guarded by the wiring assertion.
- Full suites green: **core 576/576**, **react-core provider 141/141**
(updated one fake-core mock to implement `connect()`). `tsc` 0 errors
(core + react-core); `oxlint` 0 errors; `oxfmt` clean.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Adversarial review found the original provider tests passed even with the fix
removed: a single committed mount yields exactly one /info whether the ctor or
an effect fires it, and the ctor's fetch is several microtasks deep so ordering
can't distinguish it — only the multi-instance (discarded-render) case differs,
which Testing Library can't reproduce.
- Add CopilotKitProvider.deferWiring.test.tsx (mocked core): asserts the provider
constructs with `deferInitialConnection: true` and calls `connect()` from an
effect. This FAILS if the deferral wiring is dropped (verified).
- Keep the two real-core tests as normal-mount regression guards (one /info on
mount; idempotent under StrictMode) and document that the multi-instance proof
lives in core-defer-runtime-connection.test.ts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CopilotKitProvider constructs the core during React's render phase, and React
can start-and-discard renders (concurrent rendering / Suspense / StrictMode).
Because the constructor fired the `/info` request synchronously, every discarded
-and-recreated core issued its own request — a single page load was observed
firing 70-80 `/info` requests instead of one.
Separate construction (pure) from connection (network I/O):
- core: `deferInitialConnection` lets the constructor record the runtime config
(so `runtimeUrl` stays available synchronously to hooks) WITHOUT starting the
`/info` fetch. `connect()` starts the single connection and is idempotent
(bails unless status is Disconnected), so a double-invoked mount effect
collapses to one request. `updateRuntimeConnection` also gains an in-flight
guard keyed by url+transport so concurrent same-target calls de-dupe.
- react-core: the provider constructs with `deferInitialConnection: true` and
calls `copilotkit.connect()` from its commit-phase mount effect — so renders
discarded before commit never fetch.
Backward compatible: without `deferInitialConnection` the constructor still
connects (Vue/Angular/vanilla unaffected).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a harness-native monitor that pages #oss-alerts when a whole
integration column collapses to red-D0 ("completely gone" / backend
unreachable) in production — the incident class the per-cell alert rules
miss (LGT went fully gone on 2026-07-13 and nothing paged).
Detection runs the dashboard's OWN buildCellModel fold (the shared
cell-model module both the dashboard and the monitor import) over the
same PocketBase status rows and applies a column-gone predicate over the
resulting CellModel fields, so the monitor's verdict equals the DepthChip
the dashboard renders by construction — no parallel re-derivation.
- d0-gone-predicate.ts: pure cellGone/columnGone/columnFreshHealthy over
buildCellModel outputs + registry-derived wired-cell enumeration
(mirrors the dashboard page-stats iteration / determineCellStatus rule).
- d0-gone-monitor.ts: createD0GoneMonitor factory — producer-liveness
SUSPENDED gate (reuses the family-silence inflight-aware /api/runs
reasoning, 3x-longest-period idle window), 60s confirm re-read (never a
re-probe), 15m-detect vs 1h-repost state machine, positive-fresh-healthy
CLOSE gate, ONE aggregated outage / consolidated recovery Slack message,
durable per-slug JSON map in alert_state (getSet/putSet).
- orchestrator.ts: register internal:prod-d0-gone-monitor @ */15, gated on
SHOWCASE_ENV ?? RAILWAY_ENVIRONMENT_NAME === production + kill-switch,
control-plane-only (inside runControlPlane), reusing the oss_alerts
webhook target + shared memoized family summary.
- unified-cell.test.tsx: add the required isStaleCell/observedAtAgeMs
fields to the CellModel test literal (Phase-1 dashboard tsc gate).
Red-green: a frozen test-only naiveGone (achievedDepth===0 alone)
mislabels gray-D0-no-data and stale columns as gone on committed
fixtures (RED); the real predicate fires only on red-D0-fresh and matches
buildCellModel's own outputs (GREEN). Producer-idle SUSPENDED proven
load-bearing (disabling the gate flips both F1 tests red). Plus
confirm-scan blip-rejection, hourly dedup, recovery-clear, failure modes,
and the prod-only/kill-switch registration gate.
Move the pure cell-classification fold cluster (cell-model, live-status,
staleness, format-ts) out of showcase/shell-dashboard/src/lib/ into
showcase/harness/src/shared/cell-model/ so BOTH the dashboard and a new
harness monitor import ONE copy with zero duplication and no behavior change.
The harness builds via tsc -p tsconfig.build.json with rootDir:"src" and
cannot import outside its own src/, so the harness is the correct library
home. The dashboard consumes the cluster via relative path across the package
boundary (established precedent, e.g. d5-cadence-banner.redgreen.test.ts).
- git mv the four files into harness shared/cell-model/; their intra-cluster
relative imports stay valid (they move together, no external coupling).
- Replace the four original shell-dashboard paths with thin export-* barrels
so all ~51 existing dashboard import sites resolve unchanged.
- Repoint commError-contract-drift.test.ts's source-text drift parse at the
new canonical harness location (the barrels carry no derivation body).
- Add cell-model.equivalence.test.ts + committed fixtures + a pre-move
baseline JSON (generated from the original git-HEAD code) proving the move
is byte-identical across a red-D0, gray no-data, stale, mixed, all-green,
and unsupported column.
## Problem
The package and docs routes were renamed from Bots to Channels, but the
docs still presented the old “Bots” and “Bot SDK” product name in
navigation and integration guides. The Channels reference page was
titled only “Channels” instead of “Channels SDK.”
## Why
The stale naming makes the Channels SDK links unclear in launch
communications and creates an inconsistent journey between the package
names, Slack and Teams quickstarts, and API reference.
## Fix
- Title the Channels guide and `/reference/channels` page “Channels
SDK.”
- Update the reference chooser and Slack, Teams, and WhatsApp guides to
use the Channels SDK name.
- Remove the stale wrench icon from the Slack and Teams early-access
sidebar link, with a regression test for the navigation node.
- Preserve the existing `/channels`, `/reference/channels`, `/slack`,
and `/teams` routes and all existing link targets.
- Verify permanent redirects from `/bots` and `/bots/*` to `/channels`,
and from `/reference/bot` and `/reference/bot/*` to
`/reference/channels`, with regression coverage for both exact and
nested routes.
Verification:
- `npm run lint`
- `npm run typecheck`
- `npm test`
- `npm run build`
- Local production smoke checks returned HTTP 200 for `/channels`,
`/reference/channels`, `/slack`, and `/teams`; rendered `/slack`
contains no wrench markup.
Jordan Ritter