Adds a sugar form of the --isolate flag that pins the isolation slot
directly from the command line:
bin/showcase test agno --d5 --isolate=9
# equivalent to:
SHOWCASE_ISO_SLOT=9 bin/showcase test agno --d5 --isolate
The arg parser splits --isolate=<N> into setting use_isolate=true plus
exporting SHOWCASE_ISO_SLOT=<N>; the existing picker
(_claim_isolate_slot in _common.sh) handles all validation — positive
integer, slot 0 reserved, 1<=N<=ISOLATE_MAX_SLOT, port probe, liveness.
No validation logic is duplicated.
Tests:
- replays the parser branch and verifies SHOWCASE_ISO_SLOT export +
picker pinning
- drives the picker's reserved-slot (N=0) and out-of-range (N=99)
rejections through the arg form to pin the parser->env->picker wiring
- drift guard: sources the REAL cmd-test.sh, stubs apply_isolation, and
snapshots SHOWCASE_ISO_SLOT to catch any future regression of the
parser branch
Help text and TESTING.md updated in a follow-up commit.
- showcase-canonical-pins.json: bump canonicalCopilotKitVersion 1.59.4 -> 1.60.2;
remove ms-agent-harness-dotnet override (caught up to canonical in prior commit).
- fail-baseline.json: re-ratchet validatePinsFailCount 39 -> 38 and hash to match
the one-item drop (ms-agent-harness-dotnet override no longer counted).
- @ag-ui/mastra: revert 0.2.4 -> 0.2.1-beta.2. 0.2.4 imports
'@mastra/core/runtime-context' which the pinned @mastra/core@1.41.0 does not
export, breaking 'next build' (failing mastra build-check in CI). Holding
@ag-ui/mastra at the prior pin until a coordinated @mastra/core upgrade lands.
## Summary
Built-in Agent docs now render at bare root URLs, such as `/quickstart`,
instead of under `/built-in-agent`. Legacy `/built-in-agent/*` and
`/integrations/built-in-agent/*` paths continue to redirect to canonical
root or backend URLs.
## Why
The Built-in Agent is the default docs surface, so public docs URLs
should not expose it as a sub-slug. Existing links still need to keep
working, and root-page navigation must stay stable regardless of a
visitor's stored framework selection.
## How
- Root docs resolve Built-in Agent authored pages first while preserving
reserved routes like `/ag-ui` and framework-prefixed docs.
- Redirect rules in `next.config.ts` and `seo-redirects.ts` point
retired Built-in Agent and unselected paths directly at canonical
destinations, with regression coverage that prevents redirect
destinations from targeting `/built-in-agent`.
- The sidebar/framework provider treats bare URLs as the default
Built-in Agent surface instead of letting a stored framework value
rewrite root-page chrome.
- MDX link and search-result href rewriting strip retired Built-in Agent
prefixes on root-rendered pages while preserving explicit
cross-framework links.
- Stale docs links, sitemap, `llms.txt`, markdown exports, and OG
resolution now align with the root-served Built-in Agent surface.
- Showcase generated-data tests now serialize shared fixture restoration
to avoid concurrent drift in CI.
## Verification
- GitHub checks are green on `f655013dd2576a46dc17b5901eb4dc501cb21028`.
- `npm --prefix showcase/shell-docs run test --
src/lib/__tests__/search-hrefs.test.ts
src/lib/__tests__/docs-link-rewrite.test.ts
src/lib/__tests__/seo-redirects.test.ts
src/lib/__tests__/next-config-redirects.test.ts
src/components/__tests__/docs-landing-next.test.tsx
'src/app/[framework]/[[...slug]]/__tests__/framework-root-shell-layout.test.ts'`
- `npm --prefix showcase/shell-docs run test -- --exclude
src/app/__tests__/public-assets.test.ts`
- `npm --prefix showcase/shell-docs run typecheck`
- `npm --prefix showcase/shell-docs run lint`
- `npm --prefix showcase/shell-docs run build`
- `npm --prefix showcase/scripts run test`
- `pnpm exec nx run @copilotkit/bot-slack:build --skip-nx-cache
--verbose`
The python rewriter in apply_isolation previously interpolated $slug
directly into the inline python source via bash. A slug containing a
single quote would break the python literal. Internal-tool risk only
(slug is developer-typed), but cheap to harden.
Pass slug via SHOWCASE_ISO_SLUG env var and read os.environ.get(...)
inside the python heredoc. Defense-in-depth; no behavior change for
valid slugs.
The persistent stack's docker-compose.local.yml hardcodes LOCAL_SERVICES_JSON
to the langgraph-python sample for fast N=1 local demos. When --isolate
spawns an iso1 stack with a different slug (e.g. ms-agent-python), the
iso1 harness container inherited that hardcoded value, causing
discovery.railway-services.local-injection to enumerate the wrong service
(showcase-langgraph-python instead of showcase-<requested-slug>). The iso1
probe then targeted the wrong container, broke red-green verification, and
left D5 cells unwritten.
Inject a per-slug LOCAL_SERVICES_JSON override into the iso1 compose
generator so iso1 always probes the slug passed via --isolate.
* duplicate ceiling 290→291: tool-rendering.json's tightened 'current
price of AAPL' matchers now share two match keys with the existing
tool-rendering-custom-catchall.json entries in the same BIA context,
runtime-disambiguated by feature route.
* shadow ceiling 134→132 (ratchet down): the bare 'AAPL' vs 'current
price of AAPL' shadow pair on the tool-rendering.json side is gone.
* PARITY_NOTES: replaces the 'headless-complete turns 3+4 server-tool
reprompt loop' known-issue section with a resolved-via-sequenceIndex
description; the architectural reprompt loop now converges via the
sequenceIndex-gated emitter + narration-fallback pattern in
gen-ui-headless-complete.json.
The 5 new substring shadows are in d6 fixtures landed by this PR:
- d6/ag2/gen-ui-declarative.json: 'Show me a quick KPI dashboard' inner-call mirror overlaps with pre-existing 'KPI dashboard' entry in render-a2ui.json (same toolName=render_a2ui, context=ag2). Runtime-disambiguated by load order (inner-call mirrors ordered BEFORE outer generate_a2ui fixtures per the _meta._note in that file).
- d6/claude-sdk-typescript/gen-ui-declarative.json: same pattern as ag2.
- d6/claude-sdk-typescript/tool-rendering.json: dropped userMessage/turnIndex gates on toolCallId-keyed follow-up fixtures (per _shape_note in that file — Anthropic /v1/messages shape needs toolCallId-only gating for multi-pill loop safety). The remaining ungated userMessage fixtures ('weather in Tokyo', 'AAPL') now substring-overlap with gen-ui-headless-complete and tool-rendering-reasoning-chain prompts in the same context. Runtime-disambiguated by toolCallId chains and first-match-wins ordering.
These overlaps are exactly the runtime-disambiguated pattern the test comment endorses for ceiling bumps.
Prevents a file-parallelism race where this suite's beforeAll generator
call atomically renamed catalog.json while generate-registry.test.ts was
mid-sentinel-test, clobbering the appended sentinel.
Adds a 689-line integration test that exercises the full queue lifecycle
to the /api/runs projection (enqueue → claim → terminal → projection)
across all four families. Updates the railway-envs golden + verify-deploy
drivers regression test to account for the new fleet-runs route surface.
## Summary
End-to-end hardening of the showcase shell's URL plane:
- Carries `backendHostPattern` + `docsHost` in the shell runtime config
(no longer baked from `registry.json` at Docker build time). Derives
demo backend URLs at runtime from the pattern, so a new pattern via env
reconfigures every integration on the next deploy without a registry
rebuild.
- Issues docs-host 301/308 redirects from middleware with a runtime
`DOCS_HOST`; misconfigured values no longer 500 every docs route — they
fall back to a sentinel that disables the docs-redirect step.
- Hardens the redirect table builder + matcher: first-match-wins for
duplicate exact sources, deduped wildcard prefixes with warn, malformed
entries rejected at lookup-build time, case-insensitive matching parity,
trailing-slash normalization, structural `/integrations` namespace guard
above the docs-host redirect (closes R15/R17 hijack class).
- Hardens runtime-config + backend-url env readers:
scheme/whitespace/control-char normalization, query/fragment/userinfo
rejection on `DOCS_HOST` / `POSTHOG_HOST` / backend-host pattern /
local-override URLs, present-but-empty `posthogKey` rejection, prod
loopback `BASE_URL`/`DOCS_HOST` rejection (no silent `http://` prepend),
once-guarded FATAL logging (no per-request spam).
- Brings the build-time twin in `showcase/scripts/generate-registry.ts`
to parity with the runtime normalizer (scheme/trailing-slash strip,
degenerate fallback, `NEXT_PUBLIC` fallback, slug validation).
- Surfaces PostHog capture failures once per failure class; keeps
capture alive across the redirect via `event.waitUntil`; missing
`POSTHOG_KEY` is `console.error` in production and surfaces at
config-resolution time (not first redirect).
- Open-redirect hardening on `/shared//evil.com` (SU-18); `//` rejected
at both source and destination; root `/` exact-source and `/:path*`
wildcard sources rejected; non-printable-ASCII source/destination
rejected.
Stream: SU (shell-runtime-urls). Subject groups
SU-2/8/11/13/14/15/16/17/18/19/20 (initial), SU2-A/B (env-hardening),
CR2-C (test infra), SU5-A1..A7 (registry/builder/lint/matcher boundary),
SU6-A1..A6 (request-time normalization parity), SU6-B1..B7
(parsed-normalized return forms, SHOWCASE_LOCAL states, generator
`{slug}` validation), SU7-F1..F3 (final-round backend-pattern + POSTHOG
+ table + script-side parity).
## Test plan
- [x] `pnpm test` in `showcase/shell` — 7 files, 235/235 pass
- [x] `pnpm test` in `showcase/scripts` — 51 files, 1857/1857 pass (one
pre-existing flake in `generate-registry.test.ts` unrelated to this
diff; passes on subsequent runs, classic vitest fork-reuse cross-file
pollution)
- [x] `pnpm exec tsc --noEmit` in `showcase/shell` — clean
- [x] `oxlint` on every changed file — 0 warnings, 0 errors
- [x] `oxfmt --check` on every changed file — clean
- [x] `pnpm build` in `showcase/shell` (full Next 15.x build with
registry+demo-content+starter-content+search-index generation) — clean
- [ ] CI green on the PR — confirm via `gh pr checks` after push
SU7-F1 — backend host pattern hardening:
- F1.1 Reject bare trailing ?/# in the backend host pattern
- F1.2 Strip internal tab/CR/LF from the backend host pattern
- F1.3 Warn when ignoring an empty-string local backend override
- F1.4 Reject empty-userinfo @ in the backend host pattern authority
- F1.5 Keep __proto__ keys as data in local-backend maps
- F1.6 Commit the local-backends memo key only after the value computes
- F1.7 Trim local backend overrides before validation and name the real
rejection
- F1.8 Honest FATAL when the pattern host is a stray scheme fragment
- F1.9 Canonicalize the pattern authority for parity with the override
path
- F1.10 Acknowledge the staging-to-prod fail-open in the pattern fallback
- F1.11 Harden backend-url/local-backends-env test hygiene
SU7-F2 — runtime-config & client-config edge cases:
- F2.1 Branch POSTHOG_HOST rejection reasons (scheme/degenerate/parse-
failure) instead of the catch-all mislabel
- F2.2 Reject loopback BASE_URL/DOCS_HOST in production instead of the
silent http:// prepend
- F2.3 Key the DOCS_HOST fallback once-guard on (mode, shellHost, value)
and mode-prefix all value-only guard keys
- F2.4 Reject a present-but-empty posthogKey in the client config reader
- F2.5 Drop the trailing slash from SSR_PLACEHOLDER_URL for structural
parity with server values
- F2.6 Attribute the DOCS_HOST slash-strip to readDocsHost itself
- F2.7 Normalize trailing-dot FQDN spellings in the docs self-host loop
guard (both compare sides)
- F2.8 Harden console spies to capture all log args; pin the full all-env
config shape; converge SSR simulation on vi.stubGlobal
SU7-F3 — script-side parity, table classification & test isolation:
- F3 #1 Handle a missing reference integration per the error contract
- F3 #2 Port the runtime backend-host-pattern normalization into the
generator — scheme/trailing-slash strip, degenerate fallback,
NEXT_PUBLIC fallback
- F3 #3 Treat non-mapping manifest parses (empty/null/scalar/array YAML)
as validation errors, not TypeErrors
- F3 #4 Label a missing/unreadable constraints.yaml per the stderr+exit(1)
error contract
- F3 #5 Align atomic-write tmp naming with the test harness straggler-
sweep convention; guard main() on direct invocation
- F3 #6 Correct the determineCellStatus unshipped docstring; replace
stale hardcoded cell counts with formulas
- F3 #7 Isolate the pattern suite on a per-suite tmpdir harness; snapshot
the generator's full write set
- F3 #8 Classify discarded duplicate wildcards as duplicates — hoist the
owner check above the destination warns
- F3 #9 Reject a root ("/") EXACT seo-redirect source — homepage-hijack
twin of the root-wildcard guard
- F3 #10 Reject seo-redirect entries with non-printable-ASCII source/
destination — close the silent-dead-entry class
- F3 #11 Strip trailing slashes in normalizePosthogHost before the scheme
test
- F3 #12 Message-filter the empty-slug-set error count; pin the single
matcher entry
Round-by-round CR convergence covering the redirect builder, the middleware
matcher, the docs-host self-loop guard, and the runtime-config env readers.
Highlights:
- Clear module-load warns after fresh middleware import
- Validate SET BASE_URL values (scheme-less/degenerate/garbage) with
sentinel fallback + once-guarded FATAL log
- Normalize path/query/fragment-bearing DOCS_HOST to origin; reject
non-http(s) schemes; branch rejection reasons
- Harden POSTHOG_HOST (degenerate-host/scheme rejection); expose
posthogKey via readEnvPair semantics
- Reject a DOCS_HOST equal to the shell's own host (redirect-loop guard,
authority compare)
- Warn on missing local-ports.json under SHOWCASE_LOCAL=1 and validate
TCP port range; extract helper for tests
- backend-url hardening — slug charset guard, frozen local-backends memo,
pattern path-segment warn, local-override URL validation
- Client config fail-loud covers all four URL fields with type checks
- Make RuntimeConfig.posthogKey optional — absence is a valid state, not
a wiring bug
- Drop R15/R17 and guard /integrations from SEO redirects
- Dedup duplicate wildcard prefixes with first-match-wins warn
- Validate malformed SEO entries at lookup-build time
- Restore case-insensitive redirect matching parity
- Normalize trailing slashes before redirect matching
- Keep the framework segment on F13, pin MG3 case fix
- Read posthogKey from runtime config in middleware, not raw process.env
- Fall back to the default backend host pattern for degenerate values
- Disable docs redirects when the default fallback collides with the
shell host
- Bring validateBaseUrl to parity with its sibling readers
- Strip query/fragment from POSTHOG_HOST while keeping reverse-proxy paths
- Restrict local backend overrides to http(s) URLs
- Add server-only guard to runtime-config
- Harden localBackendsEnv failure posture
- Hoist /integrations namespace guard above the docs-host redirect
- Validate seo-redirect sources and cross-kind shadowing in
buildRedirectLookup
- Unify slash normalization for middleware matching
- Lowercase-normalize REGISTRY_FRAMEWORK_SLUGS at construction
- Escalate missing POSTHOG_KEY to console.error in production
- Skip all redirect steps when docs redirects are disabled (sentinel
consumer)
- Reject userinfo credentials in DOCS_HOST, POSTHOG_HOST, and the backend
host pattern
- Branch dev-vs-prod logging in readDocsHost and fatalPatternOnce
- Prepend http:// (not https://) to scheme-less loopback hosts
- Round-5 micro-finding batch across the URL config libs
SU5-A1..A7 — registry safety, // reject, builder lint batch (case-
insensitive :path*, same-destination twin allowlist, original-case
divergence remainder), matcher api boundary, generator+vitest infra, test
hygiene + empty docs-host guard, comment batch.
SU6-A1..A6 — reject miscased :path* tokens, warn on tokenless wildcards,
normalize redirect-destination comparisons like request time, reject
destinations containing "//", surface missing POSTHOG_KEY at config-
resolution time, compile matcher harness like Next's runtime, type
parse/tokensToRegexp in the path-to-regexp shim, keep buildRedirectLookup
JSDoc attached.
SU6-B1..B7 — reject query/fragment/userinfo in pattern and local-override
URL gates, return parsed-normalized URL form from validation success
paths, distinguish unset/blank/padded SHOWCASE_LOCAL states, warn when
SHOWCASE_LOCAL is set to a value other than 1, validate {slug} placeholder
in generate-registry, mirror middleware drop semantics in the wiring
test's registry re-derivation, pin the noStore spy and calls to one fresh
module instance in the Edge-path test.
## Summary
The beautiful-chat demo's "Calculator App (Open Generative UI)" pill
rendered a calculator whose "=" key was inert: the fixture's
`jsFunctions` called `Websandbox.connection.remote.evaluateExpression`,
a host-bridge function only the open-gen-ui-advanced demo registers —
beautiful-chat never does, so the call could never resolve. While fixing
it, review and live verification surfaced four more behavioral defects
in the same fixture family, all fixed here across all 18 integrations:
- **Self-contained evaluation**: `jsFunctions` now evaluates in-sandbox
via an allowlist-gated strict-mode `Function()` (digits/operators/`eE`
only; failures → `err`). No host bridge required.
- **Fixture shadowing**: the specific "with standard buttons" pair is
ordered before the generic "build a modern calculator" pair (aimock is
first-match-wins by load order), so the intended fixture actually serves
the pill.
- **Chained evaluation**: results in exponential notation (e.g.
`1.728e+18`) re-evaluate instead of wiping to `err` (regex allows `eE`).
- **Interleaved pills**: removed the thread-global `hasToolResult` gate
that made the calculator fall through to the live proxy (502) when
clicked after any tool-producing pill; toolCallId-anchored follow-ups
(ordered first) disambiguate legs instead.
- **Repeat clicks**: distinct `tool_call_id`s per click via
`sequenceIndex` variants + a non-sequenced fallback, fixing the
second-widget collapse (duplicate id collapsed both renders into one
slot, wiping state). Scoping caveats (per-X-Test-Id counters,
cross-integration co-increment, DEFAULT_TEST_ID degradation floor =
pre-fix behavior) are documented in the fixture comments and GOTCHAS.
Also: SUPERSEDED annotations on the unreachable recorded.json calculator
entries, GOTCHAS corrections (sequenceIndex scoping, hasToolResult
semantics, statelessness claim), and a routing-invariant unit test
pinning the entry ordering structurally and behaviorally for all 18
integrations (55 tests).
## Test plan
- [x] Local Playwright red-green on the built stack (langgraph-python):
broken "=" reproduced pre-fix; post-fix visual proof of render,
`7+8=15`, chained exponent math, calculator-after-dashboard interleave,
and two independent working widgets across repeat clicks
- [x] aimock version bisect (1.28.0/1.29.0/1.30.0) ruling out an aimock
regression before the fixture root-cause
- [x] `showcase/scripts` vitest suite: 51 files / 1899 tests green
(incl. new `calculator-fixture-routing.test.ts` 55/55, red-proofed via
mutation)
- [x] validate-parity 19/19; harness aimock-fixture-coverage 3/3;
oxfmt/oxlint/commitlint clean
- [ ] CI green on PR HEAD
Follow-ups (readonly-state matcher shadowing parity across 15
integrations, sibling-pill interleave gates, GOTCHAS accuracy pass,
aimock sequenceIndex scoping) are tracked in the showcase follow-up
ledger.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
- structural + behavioral pins for all 18 integrations (four ordering invariants; click walk
_001->_002->_003->_003 with follow-ups mirroring the server match/increment flow)
- fail-loud guards against loader error-swallowing and vacuous ordering passes
- full production pill text
- oxfmt applied
Pin alignment fixes 9 validate-pins FAILs; ratchet the drift baseline
count and hash accordingly. Also tighten the _comment: document the exact
hash recipe (SHA-256 of the stderr-only [FAIL] lines, LC_ALL=C sort -u)
and correct baselineDemoCount semantics (exact expected demo count per
package; deviation either direction warns).
## Summary
Hardens the `--isolate` showcase verification flow across three areas:
**1. XDG state migration.** Isolate slot registry and per-run
rewritten-compose scratch dirs move off `/tmp` (wiped on reboot,
world-writable) to
`${XDG_STATE_HOME:-$HOME/.local/state}/copilotkit/showcase/` (`slots/` +
`runs/<name>/`). `/tmp` clearing silently destroyed a kept stack's
compose file and slot, making `--keep` unreliable. Run dirs are keyed by
the finalized project name (not PID) so a kept run is locatable for
manual teardown.
**2. Slot reaping + registry concurrency.** Since the state dir is now
persistent, slots are reaped by compose-project liveness (`docker ps
--filter label=com.docker.compose.project=<name>`), with PID/age
heuristics as fallback. The registry is made safe under concurrent
claimers: a sweep lock with heartbeat updates, own-pid lock release, and
tombstones; a claim-then-verify duplicate-name guard closing the TOCTOU
window; crash-safe reap ordering with compose-down of reap remnants and
a path-traversal guard. Failed `--isolate` setup no longer tears down
the default stack; half-initialized state is cleaned up on the way out.
Teardown uses `--volumes` everywhere, and a failed compose-down
preserves state for diagnosis. `--isolate` names are validated (must
start with lowercase letter/digit; `showcase` is reserved — it aliases
the default stack), and a fail-loud warning precedes pre-down of an
existing stack.
**3. `--keep` now actually persists an isolated stack.** Previously the
unconditional `trap restore_isolation EXIT` tore the stack down
regardless of `--keep`. Teardown is now gated on the keep flag
(`ISOLATE_KEEP` promoted to a global so it survives `cmd_test` return
into the trap scope): the slot + run dir are retained and a survival
notice prints the project name, the three offset host ports, and the
exact `docker compose -p <name> down` command — no silent port/slot
leak. A kept stack's live containers keep its slot from being reaped.
Shell-only — confined to `showcase/scripts/cli/_common.sh` +
`cmd-test.sh`; the harness TS only reads the env vars the shell exports
(unchanged). Follows up the `--keep` caveat documented in #5346.
## Review hardening
The branch went through an 8-round, 7-agent code-review loop with
red-green-verified fixes — that loop produced the state-machine
hardening commit (trap-scope fix, default-stack guards, registry
concurrency/teardown robustness, name validation) and grew the test
suite to pin every fix. A live end-to-end `--keep` verification run is
what surfaced the trap-scope bug (`--keep` silently not honored),
driving the `ISOLATE_KEEP` global fix.
## Test plan
- [x] `showcase/scripts/__tests__/isolate.bats` — 41 isolate tests
(red→green): XDG path resolution (+`XDG_STATE_HOME` override,
`~/.local/state` fallback, `runs/<name>`), liveness-based reaping (dead
project reaped/reclaimed, live project preserved), real-trap-path
`--keep` tests (no simulated-trap shortcuts), sweep/lock/tombstone race
pins (heartbeat resurrection, lock takeover, duplicate-name TOCTOU),
reap-order probe pinning live-slot protection, root/PID-reuse/DST
guards, and sentinel anti-vacuity discipline so trap tests cannot pass
vacuously.
- [x] Full `bats showcase/scripts/__tests__/` green, matching CI's Shell
script tests invocation.
- [x] shellcheck: no new warnings.
- [x] Live end-to-end: `bin/showcase test <slug> --d6 --isolate <name>
--keep` persists the stack under `~/.local/state/copilotkit/showcase`,
survival notice + manual teardown work, follow-up run reaps the stale
slot.
## Summary
- PR #5352's worker-side flap fixes never reached staging automatically:
`harness-workers` runs the same `showcase-harness` image as the
`harness` scheduler, but the SSOT's `ciBuilt: false` conflated "owns a
build slot" with "should be redeployed when its image is rebuilt" — so
main merges redeployed only the scheduler and the workers silently kept
running a stale image (a manual redeploy was required to ship the
fixes).
- This adds an `imageOf` field to the Railway SSOT so a rebuilt image
redeploys **all** of its consumers: the CI redeploy scope is now built
slots ∪ their `imageOf` consumers that declare the target env.
- Staging default scope becomes 27 (26 ciBuilt + `harness-workers` via
expansion); prod is unchanged at 26 (the worker is staging-only and the
expansion is env-aware).
## Design
- `imageOf: "<ssot-key>"` on consumer entries (`harness-workers` →
`harness`), enforced by a module-load invariant
`assertImageConsumersValid`: dangling targets, non-ciBuilt producers,
consumer chains, and consumer envs not a subset of the producer's all
fail loud at import; lookups are prototype-safe (`Object.hasOwn`).
- `expandImageConsumers` in `redeploy-env.ts` performs the env-aware,
single-level expansion and fails loud on unnormalized env names
(synonyms like `production` must go through `resolveEnv`) — the first
real consumer of `ENV_ID_BY_NAME`.
- Service-name resolution (`resolveTargetServices`/`runRedeploy`) now
rejects inherited `Object.prototype` keys with the proper
Unknown-service operator error.
- The explicit `--services` passthrough (a named service is attempted
even in an env it does not declare) is documented and contract-pinned by
a test.
## Review
- 5 unbiased 7-agent CR rounds plus a diff-attribution triage; every
diff-authored finding fixed with red-green proofs.
- ~30 pre-existing script-hygiene findings (env-registry consolidation,
accessor leniency, fetch timeout, parseArgs edges, coverage gaps in
`makeLiveRedeploy`/summary-JSON, etc.) deferred to the flap-fix
follow-up backlog.
## Test plan
- [x] 82/82 vitest (13 new tests: expansion, env-awareness, invariants
incl. prototype keys and env-subset, contract pins)
- [x] `tsc --noEmit -p showcase/scripts/tsconfig.json` clean
- [x] oxfmt clean on all changed files
- [ ] CI green on this PR
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The webhooks SSOT comment claimed the push-driven default scope is
"guaranteed" to leave webhooks untouched — false: a push touching the
build workflow files trips the workflow_config paths-filter disjunct,
which selects every matrix slot (webhooks included; its skip_build slot
still reports success and enters the redeploy CSV). Reworded to state
the actual behavior. imageOf doc now states the enforced NON-EMPTY
subset constraint; serviceEnvPairs doc now truthfully says it has no
consumers yet; file header notes the probe flag default and
bin/railway's Ruby-only "stage" synonym.
Test hygiene: drop the stale bin/railway line-number citation from a
test name, the _envConfigTypeAnchor (EnvironmentConfig is genuinely
referenced by the shape-compile test), a dead eslint-disable, and a
dead `as never` cast (env is an open string); align the webhooks
dispatch-name pin regex with the extraction regex's whitespace
tolerance.
assertEnvRegistryConsistent gains four clauses: (iv) a key present in
both ENV_IDS and ENV_ID_BY_NAME must carry the same env-id (ENV_IDS.prod
drifted to the staging id previously passed every clause while
resolveEnv("prod") silently returned staging); (v) every ENV_IDS env-id
must be carried by a canonical name (was only caught lazily in
resolveEnv); (vi) registry keys must be trim().toLowerCase()-normalized
(resolveEnv lowercases input, so a non-lowercase spelling is registered
but unreachable); (vii) no registry key may be an Object.prototype
property name.
expandImageConsumers' per-entry env skip-check becomes an own-property
test (Object.hasOwn) for uniformity with every other lookup in the file.
REDEPLOY_SUMMARY_JSON is trimmed before the set-but-empty branch so a
whitespace-only value hits the loud warn path instead of attempting a
JSON write against a garbage path.
- ciBuilt field doc: pocketbase IS showcase-CI-built — only webhooks
remains out-of-band; keep the MUST-NOT-touch claim for webhooks only.
- gateValidated field doc: true for every service EXCEPT the two
gateIgnore entries (harness-workers, harness-legacy), not "every
service".
- CI_BUILT_SERVICES comment: also names the excluded non-CI-built
harness-workers and harness-legacy alongside webhooks.
- legacyJsonCompat doc: it is bin/railway's EXPECTED_DOMAINS derivation
that filters out *.up.railway.app hosts (no "parity test rejects"
claim contradicting the placeholder data below).
- webhooks entry comment: a manual service=all build dispatch MAY
bounce webhooks staging (the skip_build slot still reports success,
entering the matrix ∩ success-set redeploy scope); only the
push-driven default scope is guaranteed to leave webhooks untouched.
- railway-envs: route envsFor/instanceIdFor/domainFor/probeEnabled (and
repoNameFor) through shared getEntry/getEnvCfg own-property helpers so
inherited Object.prototype keys on either axis produce the curated
error (or probeEnabled's contract false) instead of raw TypeErrors,
silent undefined, or a spurious probe=true.
- railway-envs: two new module-load invariants (synthetic-map
injectable): assertEnvRegistryConsistent (per-service env keys are
registered canonical names; ENV_ID_BY_NAME env-ids unique; every
canonical name has an ENV_IDS spelling) and
assertServiceAndInstanceIdsUnique (serviceId unique per entry,
instanceId globally unique).
- redeploy-env: invert the exit-code policy to fail-loud by default —
any env except the documented staging carve-out exits non-zero on
per-service failure (a future preview/canary env inherits fatal
semantics instead of silently swallowing failures).
- redeploy-env: sanitize per-service THROWN error messages through
sanitizeErrorBody before recording; flatten bare \r in the summary
table escape.
- redeploy-env: warn on set-but-empty REDEPLOY_SUMMARY_JSON; reject
flag-like --services CSV parts (both forms) and a flag-like first
argument (missing env); derive usage env lists from ENV_IDS.
- tests: prototype-key sweep across all accessors, invariant
positive/negative coverage, third-env exit-code pin, sanitization and
CLI-guard coverage, makeLiveRedeploy !res.ok and non-true mutation
branches.
Cross-session review fixes for the --isolate machinery (one concern:
source + test + docs).
1) Reaper reserved-name guard (critical): _reap_isolate_slot trusted
slot records — a record naming 'showcase' (corrupt, or written by an
older CLI version before apply_isolation reserved the name) passes
the charset regex, so the reap ran `docker compose -p showcase down
--remove-orphans --volumes` against the LIVE default stack,
destroying the PocketBase named volume. The reserved name now gets
the same treatment as the path-traversal guard: warn (naming the
record and why it is dangerous) and leave the slot intact for manual
inspection — no compose-down, no state removal.
Call-site enumeration: _reap_isolate_slot's sole caller is
_sweep_isolate_slots, at 3 sites (dead-PID reap, project-recorded/
no-owner reap, age-fallback reap), all passing
"$slot_entry" "$slot_proj" — all three flow through the new guard
identically.
Red-green: the new bats test ("a slot whose project record reads the
RESERVED 'showcase' is left intact...") was run against the UNFIXED
code first and FAILED — the sweep logged "Attempting to reclaim
stale slot 0 (project showcase has no live containers and no
recorded owner)" and reaped the slot. It passes with the guard.
2) .iso-bak restore race: two concurrent runs can both see a stale
backup; the loser's mv is the FINAL command of its `[ -f ] && mv`
AND-list, so its failure trips set -e and kills the CLI pre-claim
with a raw error. Both mv's now carry `2>/dev/null || true` — the
survivor's restore wins, the loser proceeds with restored originals.
3) Keep-test absence regexes greped only the `--project-name <name>
down` spelling; the reaper's own downs use `-p <name> down`, so a
keep-branch regression via the -p form passed undetected. Both keep
absence assertions now match `(--project-name|-p) <name> down`.
Mutation-verified: a temporary -p-form compose-down added to the
keep branch made BOTH broadened tests FAIL; reverted, suite green.
(All other absence assertions use the word-matched generic
`compose ... down` regex, which already covers both spellings.)
4) RUNBOOK.md/DEBUGGING.md contradicted shipped code: the manual
teardown was quoted without --volumes plus notes claiming
`down --remove-orphans` leaves named volumes (the shipped survival
notice and every teardown path include --volumes), and the name rule
was documented as `[a-z0-9_-]+` (actual: starts with [a-z0-9], then
[a-z0-9_-], uppercase normalized with a warn, 'showcase' reserved).
Both updated to the shipped semantics; the now-redundant separate
`down --volumes` snippets removed.
Verification: full `bats showcase/scripts/__tests__/` green (60 tests);
shellcheck on _common.sh shows no new warnings vs baseline
(pre-existing SC2034/SC2115 only, line-shifted).
Confirmation-CR bucket-(a) fixes for PR #5353 — one coherent concern:
env-name resolution has exactly ONE authority (the ENV_IDS /
ENV_ID_BY_NAME registries) and SSOT accessors fail loud instead of
silently returning wrong values.
- runRedeploy: resolve envId via ENV_ID_BY_NAME with an Object.hasOwn
guard + fail-loud throw listing the registered envs. Removes the
hardcoded `prod`/`staging` pair check and PRODUCTION/STAGING ternary
that contradicted the SSOT's documented open-env contract ("a new env
needs only a registry entry"); the registry lookup subsumes it.
- resolveEnv: derive resolution entirely from the registries (ENV_IDS
spellings -> env-id -> canonical ENV_ID_BY_NAME name) instead of its
own hardcoded synonym chain. Behavior identical for
prod/production/staging; still throws on unknowns, and now also
throws on a mis-wired registry (a spelling whose env-id has no
canonical name).
- serviceForDispatchName: fix the docstring's false "CI-built service"
claim — it does no ciBuilt filtering and tests pin the unfiltered
behavior (the non-CI-built webhooks resolves).
- repoNameFor: fail loud (consistent with instanceIdFor/domainFor)
instead of silently echoing the service name — the exact
silently-wrong-GHCR-name class this PR's hardening targets. Throws on
unknown service, on an env not registered in ENV_ID_BY_NAME
(unnormalized synonyms like "production"), and on a registered env
the service does not declare. Keeps the documented default (the
service name) for declared envs without an override.
Call-site enumeration confirming nothing relies on the old fallback:
- verify-railway-image-refs.ts:523 — iterates the entry's DECLARED
environments keys, registry-filtered, SSOT-matched service names
- __tests__/railway-envs.golden.test.ts:81 — iterates envsFor(name)
(declared envs only) over real SSOT keys
- railway-envs.test.ts repoNameFor cases — dual-env services,
prod/staging only
- __tests__/verify-railway-image-refs.test.ts:264 — FIVE_NEW keys,
all dual-env
- resolveTargetServices: throw when an explicitly-provided services
list resolves to zero entries (whitespace-only programmatic input)
instead of letting runRedeploy exit 0 having redeployed nothing; the
default undefined -> full CI-built scope is unchanged.
- makeLiveRedeploy: add signal: AbortSignal.timeout(30s) so a hung
Railway API records a per-service FAIL instead of stalling CI, and
pass GraphQL errors[].message through sanitizeErrorBody for
consistency with the HTTP-error path. Exported for direct unit tests.
Red-green: 11 new tests (open-env registry resolution incl. a
runtime-registered hypothetical env, repoNameFor negatives,
empty-resolution throw, abort-signal presence, GraphQL error
sanitization) all failed against the old code; full showcase/scripts
suite green (50 files, 1807 tests) + tsc --noEmit clean.
- ISOLATE_KEEP promoted to a global so --keep survives cmd_test return
into the trap scope
- early-die and default-stack protection: failed --isolate setup no
longer tears down the default stack; half-initialized state is
cleaned up on the way out
- liveness/PID/age reaping signals with a sweep lock: heartbeat
updates, own-pid lock release, tombstones, and a claim-then-verify
duplicate-name guard close slot-registry races (TOCTOU, lock
takeover, reap order)
- teardown robustness: --volumes on every compose down, failed-down
runs preserve state for diagnosis, reap remnants get a compose-down,
path-traversal guard, uniform rm guards under set -e
- name validation: --isolate names must start with a lowercase letter
or digit; reserved name 'showcase' rejected (it aliases the default
stack)
- fail-loud warning before pre-down of an existing stack; help text
updated
Result of an 8-round, 7-agent code-review loop with red-green
verified fixes.
harness-workers runs the SAME showcase-harness GHCR image as the harness
scheduler but has ciBuilt:false (it has no build slot of its own), and the
CI staging redeploy scope was derived purely from ciBuilt — so a main-merge
rebuild of showcase-harness:latest only bounced the scheduler while the
workers silently kept running the stale image (PR #5352's worker-side fixes
never reached staging).
Model image consumption explicitly in the SSOT instead:
- railway-envs.ts: new optional `imageOf` field on ServiceEntry — the SSOT
key of the ciBuilt service whose image this entry runs. Set
`imageOf: "harness"` on harness-workers. New module-load invariant
`assertImageConsumersValid` (fail-loud, same style as
assertDispatchNamesUnique): imageOf must name an existing SSOT key, the
target must be ciBuilt, and the consumer itself must not be ciBuilt.
- redeploy-env.ts: new `expandImageConsumers(names, env)` applied inside
runRedeploy — the redeploy set becomes the resolved scope PLUS any
service whose imageOf points at a service already in scope. Env-aware:
a consumer only joins envs it declares, so the staging-only worker
never enters a prod redeploy (prod behavior unchanged).
No workflow change needed: showcase_build.yml keeps passing the
built-and-successful dispatch_names; the script expands them. Gate
behavior (gateIgnore / image-ref gate), the generated JSON
(emit --check passes byte-identical), and the promote dropdown are all
untouched. harness-legacy deliberately gets no imageOf (pinned pre-fleet
digest; must not follow rebuilds).
Previously the EXIT-trap restore_isolation always tore down the isolated
stack, ignoring --keep. Now restore_isolation reads a keep flag (set in
cmd-test.sh when --keep is parsed): when kept it skips compose down, the run-dir
removal, and the slot release, and instead prints a survival notice with the
project, slot, the three offset host ports, and the exact manual teardown
command. The kept stack's live containers keep its slot from being reaped.
Persist the compose project name into each claimed slot dir, and at claim time
reap any slot whose recorded project has no live containers (queried via
docker ps --filter label=com.docker.compose.project). This correctly leaves a
--keep'd stack's slot alone since its containers are still up. The existing
PID/age heuristics remain as a fallback for slots predating the project file.
Move the --isolate slot registry and per-run scratch dir off /tmp (wiped on
reboot, world-writable) to $XDG_STATE_HOME/copilotkit/showcase (slots/ and
runs/<name>). The run dir is now keyed by the finalized project name instead
of the PID so a kept run is locatable for manual teardown. Adds a bats suite
covering the new state-base helper and run-dir location.
llamaindex, agno, and claude-sdk-python emit AG-UI REASONING_MESSAGE_*
events but pinned @ag-ui/client ^0.0.43, whose @ag-ui/core discriminated
union lacks the REASONING_MESSAGE_* variants — the frontend threw
invalid_union_discriminator and the reasoning demo broke. Pin all three
to exact 0.0.48 (matching the claude-sdk-typescript fix in #5323),
regenerate their lockfiles so @ag-ui/core resolves to 0.0.48 with zero
0.0.43 nodes, and ratchet the validate-pins drift baseline down from 60
to 57 to reflect the now-exact pins. Verified locally on the built-image
showcase rig: D5 green and the D6 reasoning-display probe passes for all
three backends with zero invalid_union_discriminator.
Jordan Ritter