The tool-rendering cell (marked supported) had 5/6 e2e failing and
tool-rendering-reasoning-chain 2/5, all from mastra aimock fixtures diverging
from the langgraph-python gold standard. Root cause: several fixtures were
keyed on generic substrings where gold uses unique tails, so they
substring-collided with the longer chain pills and — loading earlier
(alphabetical file order) — hijacked them.
Verified on a faithful rig (Node 22 + next build/start + aimock 1.37.4 replay):
- Stock: the pill's scripted $338.37 fixture was shadowed by
headless-complete's ticker-only "price of AAPL" leg (tool's 189.42 default).
Restore gold's unique "price of AAPL right now" key; key the tool-rendering
emit leg on toolName (gold parity).
- d20: the first-roll leg gated on hasToolResult:false never matched once
prior-pill tool results lingered in thread history -> 0 cards. Match on
userMessage only (gold). Add stopWhen: stepCountIs(8) to toolRenderingAgent
so the 5-roll sequence + narration (and the 3-tool chain-tools turn) run to
completion instead of stopping at the default step cap.
- chain-tools: headless-complete's generic "weather in Tokyo" leg hijacked the
"...get the weather in Tokyo..." pill and emitted only get_weather. Restore
gold's "What's the weather in Tokyo" key.
- reasoning-chain flights+weather + sequential: beautiful-chat's generic
"Find flights from SFO to JFK" legs hijacked the "...JFK and show me the
weather there" pill. Restore gold's "for next Tuesday" key.
Result: tool-rendering 6/6, tool-rendering-reasoning-chain 5/5, beautiful-chat
8/8 (no regression) under aimock replay.
Note: headless-complete's own weather/stock/revenue cards remain red on a
separate pre-existing bug (headlessCompleteAgent tool-registration + a missing
get_revenue_chart tool) — addressed in a follow-up commit.
--no-verify: this sparse showcase checkout has no monorepo lefthook/commitlint
binaries (matches prior commits on this branch).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Release angular v0.2.0
**Scope:** `angular` | **Bump:** `minor`
---
### How this release process works
1. **This PR was created automatically** by the "release / create-pr"
workflow.
It bumped the `angular` packages to `0.2.0`
and generated AI-enhanced release notes.
2. **CI runs on this PR** — the full test suite (unit tests, lint, type
checks, build)
must pass before merging. This is the review gate.
3. **Review the release notes** in `release-notes.md` in this PR.
If a Notion draft was created, you can edit the release notes there
before merging.
4. **When this PR is merged**, the `release / publish` workflow
automatically:
- Builds all packages
- Publishes the `angular` packages to npm at version `0.2.0`
- Creates git tag `angular/v0.2.0`
- Creates a GitHub Release with the final release notes
### Before merging
- [ ] CI is green (tests, lint, types, build)
- [ ] Version bumps look correct
- [ ] Release notes are accurate (edit in Notion if a draft was created)
---
> **Do not merge until CI is fully green.** The full test suite runs
automatically on this PR.
The /eval workflow ran 'showcase eval ... --ci' on a bare runner, but --ci
makes the eval CLI SKIP the Docker lifecycle and assume the showcase fleet is
already running (harness/src/cli/eval/index.ts:348). Nothing in the job starts
it, so every /eval found no healthy container and failed in ~1s without running
a single test (e.g. #5798's /eval d5 mastra: mastra _status=fail, 364ms, zero
tests).
Drop --ci so the CLI builds + starts the in-scope slugs + aimock and
health-checks them before running d5 (the non-ci path). docker compose output
is captured (piped, not inherited) and --json keeps stdout clean, so
eval-results.json stays parseable.
## Problem
The deployed Slack triage bot (`examples/slack`) started failing
**every** turn with:
```
400 Invalid schema for function 'save_diff_comment':
In context=('properties','anchor'), 'propertyNames' is not permitted.
```
No change on our side triggered it — Linear's hosted MCP server changed
the `save_diff_comment` tool schema. Its `anchor` param is now a
free-form map (open object declared with `propertyNames` + open
`additionalProperties`). The bot fetches Linear's tool list at runtime,
so it picked up the new schema automatically.
## Root cause
`@tanstack/ai-openai@0.15.2` (what the bot resolves to) forces `strict:
true` on every function tool. OpenAI's strict function-calling validator
only accepts a subset of JSON Schema and **rejects the entire request
(400, before the model runs)** for a free-form-map object like `anchor`.
One over-rich third-party tool takes down the whole turn.
## Fix — adopt the upstream fix via a dependency upgrade
Already fixed upstream: `@tanstack/openai-base@0.9.8`
([tanstack/ai#933](https://github.com/TanStack/ai/pull/933)) makes the
tool converter detect free-form-map schemas and emit those tools with
`strict: false` (so they stay callable) instead of forcing an invalid
strict schema. First ships in `@tanstack/ai-openai@0.17.0`.
The bot's `^0.15.2` range can't reach it, so this bumps the aligned set
and refreshes `pnpm-lock.yaml`:
| package | before | after |
|---|---|---|
| `@tanstack/ai` | `^0.32.0` | `^0.42.0` |
| `@tanstack/ai-openai` | `^0.15.2` | `^0.17.1` (→ `openai-base@0.9.9`)
|
| `@tanstack/ai-mcp` | `^0.1.3` | `^0.2.5` |
**zod stays at `^3.25.76`.** The repo pins zod to 3.x via a root
`pnpm.overrides` (`zod: ">=3.22.3"`), so the whole workspace resolves
zod 3 regardless. `ai-openai@0.17` peers `zod ^4` (unmet → advisory
warning only), but the strict-schema fix operates on plain JSON Schema,
not zod, so it's unaffected.
**No runtime code change** — the fix lives entirely in the upgraded
adapter (an earlier revision of this PR hand-rolled a schema sanitizer;
that's removed in favor of leaning on TanStack's built-in handling).
## Verification
⚠️ Not verifiable in this worktree (example deps aren't installed here).
Before merge, in an installed env:
- `pnpm --filter slack-example check-types` and `pnpm --filter
slack-example test`
- One live turn hitting Linear (previously-failing `save_diff_comment`
path)
- Sanity-check the bot runs on the workspace's pinned **zod 3** despite
`ai-openai@0.17`'s `zod ^4` peer (the fix path is zod-independent, but
confirm no other `@tanstack/ai` code the bot exercises needs a
zod-4-only API).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Linear's save_diff_comment.anchor started shipping as a free-form map
(propertyNames + open additionalProperties). @tanstack/ai-openai@0.15.2 forced
strict:true on every tool and 400d the whole turn on such schemas.
@tanstack/openai-base@0.9.9 (via ai-openai@0.17) detects free-form-map schemas
and emits those tools with strict:false instead, so they stay callable. Bump the
aligned set and refresh the lockfile:
@tanstack/ai ^0.32.0 -> ^0.42.0
@tanstack/ai-openai ^0.15.2 -> ^0.17.1
@tanstack/ai-mcp ^0.1.3 -> ^0.2.5
zod stays at ^3.25.76: the repo pins zod to 3.x via a root pnpm override, so the
whole workspace resolves zod 3. ai-openai@0.17 peers zod ^4 (unmet, advisory) but
the strict-schema fix operates on plain JSON Schema, not zod, so it is unaffected.
No runtime code change.
Addresses the tool-rendering CHANGES_REQUESTED on #5798 (5/6 e2e failing),
mirroring gold langgraph-python tool_rendering_agent.py:
1. Dedicated toolRenderingAgent bound to all four demo tools (get_weather,
search_flights, get_stock_price, roll_d20) + route tool-rendering and its
default/custom-catchall variants to it. Previously routed to weatherAgent,
which lacks get_stock_price/roll_d20, so the Stock, d20, and Chain pills
emitted uncallable tool calls the AI SDK dropped (no card).
2. New deterministic roll_d20 tool (echoes a scripted value) and get_stock_price
now echoes optional price_usd/change_pct — lets the fixtures script exact
values, mirroring gold's roll_d20/get_stock_price.
3. search_flights now accepts gold {origin,destination} and GENERATES the
flights list (kept lenient — still accepts a legacy {flights} array so the
D5 harness probe keeps working). Fixes the reasoning-chain flights leg that
failed input validation.
4. Re-keyed the tool-rendering + reasoning-chain fixtures off the fragile
turnIndex onto hasToolResult:false (turn-scoped in aimock >=1.37.x) and
added the missing Find-flights first leg (was 'No fixture matched' 404).
5. Weather e2e assertion -> 77%/3 (mastra's seeded getWeatherImpl for SF; a
documented divergence from gold's fixed 55/10).
Route /api/copilotkit compiles + loads cleanly; full e2e to be confirmed via
/eval d5 mastra on the harness (local next-dev verification is blocked by this
machine's 7.7GiB Docker VM OOMing on the heavy route compile).
generate-registry.ts imports the catalog cross-join/flatten fold from
../harness/src/shared/catalog/catalog-flatten.ts, which does
`import yaml from "js-yaml"`. The generator's build/test environments did
not stage that file (or its module-resolution scope), so the fold could
not resolve.
- Dockerfiles (shell, shell-dashboard, shell-docs, shell-dojo): COPY the
shared catalog source + harness/package.json (its `"type":"module"` is
required so catalog-flatten resolves as ESM and its named exports bind)
and provide a node_modules for js-yaml resolution.
- generate-registry-pattern.test.ts (makeHarness): stage catalog-flatten.ts
and harness/package.json at the exact relative path the generator
resolves, and symlink the scripts node_modules onto the harness tree so
the ESM `import yaml from "js-yaml"` resolves.
- js-yaml + @types/js-yaml added to showcase/scripts (package.json and the
npm package-lock.json), and the root pnpm-lock.yaml regenerated to add
the matching importer entries for showcase/scripts (js-yaml >=4.1.1 via
the root override, @types/js-yaml ^4.0.9) so `pnpm install
--frozen-lockfile` stays in sync.
deriveDepth becomes a thin adapter over the shared buildCellModel engine so the
dashboard and API render from one ladder; page-stats routes through the shared
catalog input; dashboard-page per-cell render try/catch isolates a bad cell.
Assigns the authoritative total before the empty-page break and treats an
inconclusive/truncated read as HOLD (fails toward alerting, never folds partial
data into a nothing-gone verdict); de-dupes the short-read log; per-cell scan
try/catch degrades one bad cell without blinding the detector.
One shared catalog-flatten authority (typed throws, not process.exit); the server
re-flatten validates manifest structure at parity with the codegen path.
Runs buildCellModel server-side (full signal the browser strips) so the true
dashboard-visual state is API-derivable. Authoritative-total short-read guard
(serves matrix_unavailable, never silent all-gray), page-cap throw, and per-cell
try/catch so one bad featureId degrades a single cell, not the whole surface.
Single per-rung classifyRung verdict folded uniformly via foldFamily/combine;
enforces the section-2a coherence invariants (chip/isRegression/achieved/d6Effective
from one verdict), first-strike de-amplification gated on max non-infra red
fail_count, future-skew age clamp, and the staleness folds. Golden-master
equivalence baseline + coherence/contribution/combine test suites.
## Summary
Three laser-surgical cleanups to the showcase D4 chat-roundtrip probe
and its `domDone` regression guard. Two are behavior-preserving
refactors; one is a test-coverage hardening. No probe control-flow /
outcome behavior changes.
## Changes
### GB2 — derive the guard's probed-route set from the probe SSOT
`d4-probe-domdone-guard.test.ts` hardcoded its probed-route set as a
literal `["agentic-chat","tool-rendering"]` parallel to the probe's own
inline `demo`/`demoPath` literals — a drift hazard if the probe ever
starts driving a new D4 route. Introduced `D4_DEMO_ROUTE_AGENTIC_CHAT`,
`D4_DEMO_ROUTE_TOOL_RENDERING`, and a `D4_PROBED_DEMO_ROUTES`
source-of-truth array in `d4-chat-roundtrip.ts`, wired the two
`runLevel` `demo`/`demoPath` call sites and `hasToolRendering` to them,
and had the guard derive `PROBED_ROUTES` from that export. A future D4
route is now covered automatically. Added a superset assertion so the
derived set can never narrow coverage below `{agentic-chat,
tool-rendering}`. Route strings unchanged → behavior-preserving.
### T2 — consolidate the completion FSM — DEFERRED
Not surgically achievable without behavior risk. The completion state
(`completeAtMs` / `everObserved` / `degraded` plus the per-poll
`observed` / `completed`) is progressively mutated across poll
iterations inside `runAttempt`'s loop and consumed jointly by the
grace/ceiling/floor deadline clamp (`Math.min(Math.max(baseBudgetEnd,
graceEnd), fastFailEnd, attemptCeiling)`) and three early-return sites.
There is no single read-site to redirect into one named source of truth;
a true consolidation would restructure the deadline arithmetic and risk
altering the clamp outcomes on this flap-critical probe. Left as-is per
proportionate-fix discipline.
### T3 — align complete/completed naming (behavior-preserving rename)
`readTurnComplete` returned `{ observed, complete }` while `runAttempt`
returned `{ text, completed, observed }` for the same concept. Renamed
the lower-churn side (`readTurnComplete`'s `complete` → `completed`, 5
refs vs 6) so both use `completed`. The fields are local to `runLevel`,
not exported and not referenced by any test → pure rename, zero behavior
change.
## Proof — behavior preserved
Baseline (origin/main state, before changes):
- `d4-chat-roundtrip.test.ts`: **84 passed**
- `d4-probe-domdone-guard.test.ts`: **41 passed**
- Total: **125 passed**
After changes:
- `d4-chat-roundtrip.test.ts`: **84 passed** (identical — behavior
preserved through T3 rename + GB2 wiring)
- `d4-probe-domdone-guard.test.ts`: **42 passed** (41 original + 1 new
GB2 superset test)
- Total: **126 passed**
The 125 baseline tests all still pass; the only delta is the added GB2
coverage test.
## GB2 guard red-green
On
`showcase/integrations/langgraph-python/src/app/demos/agentic-chat/page.tsx`:
- Mutated `<CopilotChat … />` → `<CopilotChat …>{null}</CopilotChat>`
(children/render-prop form) → guard **RED** (1 failed:
`langgraph-python/agentic-chat renders <CopilotChat/> in the
attribute-bearing self-closing form`).
- Reverted → guard **GREEN** (42 passed).
Mutation fully reverted; net change to the guard's scanned pages is
zero.
## Quality gates
- `oxfmt --check`: clean
- `oxlint`: 0 errors (2 warnings, both pre-existing on lines 954 / 1189,
untouched by this diff)
- `tsc --noEmit`: clean
- `tsc -p tsconfig.build.json` (full build): clean
readTurnComplete returned { observed, complete } while runAttempt returned
{ text, completed, observed } for the same concept. Rename the lower-churn
side (readTurnComplete's 'complete' → 'completed') so both use 'completed'.
Pure internal rename — the fields are local to runLevel, not exported and
not referenced by any test. Zero behavior change.
The d4-probe-domdone-guard test hardcoded its probed-route set as a literal
["agentic-chat","tool-rendering"] parallel to the probe's own inline
route literals — a drift hazard if the probe starts driving a new D4 route.
Introduce D4_DEMO_ROUTE_AGENTIC_CHAT / D4_DEMO_ROUTE_TOOL_RENDERING and a
D4_PROBED_DEMO_ROUTES source-of-truth array in the probe, wire the two
runLevel demo/demoPath call sites (and hasToolRendering) to them, and have
the guard derive PROBED_ROUTES from that export. Add a superset assertion so
the derived set can never narrow coverage below {agentic-chat,tool-rendering}.
Behavior-preserving: the route strings are unchanged.
Follow-ups to #6082 (showcase deploy consolidation), applied after the
live prod `autoUpdates` flip.
1. **fix** — the `notify-all-builds-failed` / `notify` jobs had the same
cancelled-rollup blind spot as the redeploy guard: an all-legs-cancelled
build (one leg cancelled under contention) that really produced **no**
successful service sent **no** alert. They now fire on `any_success ==
'false'` (guarded by a status function so a user-cancelled run stays
silent). Red-green in `redeploy-guard.test.ts` (reads the live workflow
guards).
2. **feat** — prod `autoUpdates` is now `"disabled"` (was
`"unmanaged"`), bringing prod under drift-gate management now that its
live Railway `autoUpdates` are disabled. Both envs are migrated; the
gate enforces both.
The live staging + prod Railway `autoUpdates` are already disabled, so
the drift gate finds live == SSOT on both envs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Prod autoUpdates is now "disabled" (was "unmanaged") for every service, so the
drift gate enforces prod as well as staging. Paired with disabling autoUpdates
on the live prod Railway services. Regenerates the SSOT JSON.
The notify-all-builds-failed and notify jobs keyed off a 'failure' rollup /
bare failure(), so a build where every real service failed but one leg was
cancelled (contention) rolled up to 'cancelled' and sent no alert — the same
blind spot as the redeploy guard. Fire on any_success == 'false' (guarded by a
status function so a user-cancelled run stays silent). Extends the guard test.
## Showcase deploy-mechanism consolidation
Consolidates the showcase Railway deploy path onto a single
**CI-explicit** mechanism, so we can safely retire Railway's registry
auto-watch (the source of the surprise "Service aimock upgraded to
latest" emails). Design proposal: [Notion — Showcase Deploy-Mechanism
Consolidation](https://app.notion.com/p/3a33aa38185281e4b64cc5bebde92d91).
### What & why
The "aimock upgraded to latest" email was never a per-service config
choice — it was a **CI bug** letting Railway's watcher win a race: a
Renovate PR that only touches `showcase_build.yml` forces a full-fleet
rebuild; the LFS `shell` leg gets cancelled under runner contention; and
the `redeploy-staging` guard (`needs.build.result != 'cancelled'`) then
skipped the CI redeploy for the **whole fleet**, orphaning aimock's
fresh digest for Railway's watcher to pick up. The `autoUpdates` setting
itself had also silently drifted (24 services `minor` / 17 none) —
tracked in no SSOT, gated by nothing.
### The four changes (one commit each)
1. **`fix(showcase)` — the P0 guard bug.** Relax the `redeploy-staging`
**and** `redeploy-staging-starters` guards so a cancelled sibling leg no
longer skips the fleet's staging redeploy; they now redeploy the
already-computed successful-service list. A guard-evaluation test reads
the live workflow `if:` strings and models GitHub's matrix rollup.
2. **`feat(showcase)` — autoUpdates SSOT (per-env, staging-first).** Add
a **per-env** `autoUpdates` policy to every service in `railway-envs.ts`
— **staging: `disabled`** (enforced), **prod: `unmanaged`** (left
exactly as-is until a later migration). Regenerate
`railway-envs.generated.json`. CI-explicit redeploy becomes the single
deploy path on staging.
3. **`feat(showcase)` — drift gate.** New CI gate fails when a live
Railway service's `autoUpdates` diverges from the SSOT. Reads
`Environment.config` (autoUpdates isn't on the typed `ServiceSource`
output), **enforces managed (`disabled`) envs and skips `unmanaged`
ones** (so prod is untouched), **fails closed per-env** on zero-checked,
and skips cleanly on fork PRs with no Railway token.
4. **`feat(showcase)` — scheduled reconcile.** CI-owned self-heal (every
15m) comparing each staging service's deployed digest against GHCR
`:latest`, re-running the staging redeploy for lagging services and
alerting Slack. Invariant: **green ⟺ every in-scope service confirmed
current**; any unconfirmed service (lag, digest error, dropped redeploy,
empty scope, thrown redeploy) alerts and exits non-zero.
### Verification
- Every behavior change carries red-green tests; **230 tests pass**,
`tsc` clean, `oxfmt`/`oxlint` clean, generated JSON in sync, workflows
parse.
- Reviewed via a full CR loop (Tier 3, 5 rounds to convergence); the
reconcile's fail-loud invariant was hardened across rounds (silent-green
holes, stale-digest ordering, expansion false-positives, test hygiene).
### Rollout (staging-first)
- **Staging is flipped live as part of this change** — `autoUpdates`
disabled on all staging services (snapshot-first, verified only
`autoUpdates` changed). The drift gate now enforces staging.
- **Prod is untouched** — its `autoUpdates` stay exactly as-is and the
gate marks prod `unmanaged` (skipped). Migrating prod is a deliberate
follow-up (flip prod live + change prod SSOT `unmanaged`→`disabled`
together) once we're comfortable with staging on the new mechanism. No
transition window where anything is unguarded.
### Follow-ups (from CR, non-blocking)
- Dedup the reconcile alert's `unconfirmed` list by service key
(cosmetic double-listing; exit code already correct).
- Harden the sibling `notify-all-builds-failed`/`notify` jobs against
the same all-legs-cancelled rollup (pre-existing, in a job this PR
doesn't touch).
- Minor: `postSlackAlert` try/catch belt; a few added test assertions;
comment/doc accuracy.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Incident
At ~21:19Z on 2026-07-20 the prod showcase Feature Matrix showed the
`claude-sdk-python` column whole-column-red (✓0/✗37). It self-recovered
by ~21:30Z. Not infra, not staleness: `/api/health` was 200 throughout,
`commErrorKinds` was empty, and the image was current.
## Root cause
The shared D4 chat probe (`d4-chat-roundtrip.ts`) emitted ONE transient
red — `failureSummary: "empty assistant response"`, no
`errorClass`/`errorDesc`. Because the dashboard's `resolveD4` is
**slug-keyed**, that single `chat:<slug>` row gated all 37 feature cells
for the slug → the whole column went red off one flap.
The producer-side origin is a **render race**. `readTurnComplete()`
gated turn completion on `domDone || sseDone`. `sseDone` is the
transport-level `RUN_FINISHED` counter — a synchronous raw-byte SSE
parse that runs *outside* React and can fire at-or-before the React
commit that actually renders the assistant text. `domDone` (the
`data-copilot-running` true→false DOM edge) is coalesced into that
*same* React commit (`use-agent.tsx`
`batchedForceUpdate`/`queueMicrotask`). Under contention, gating on
`sseDone` let the fast-fail grace window elapse while the DOM was still
empty, redding a turn that was about to render correctly.
## The change
`complete: domDone || sseDone` → `complete: domDone`. The moment
`domDone` is true, the assistant text is already in the DOM (same
commit). `observed` still includes `sseDone`, so a turn whose attribute
never appears still gets the wider polling window rather than
fast-failing. `FIRST_TOKEN_GRACE_MS` and `NON_COMPLETION_RETRY_LIMIT`
are untouched. `domDone` is not a new signal — the probe already
observes that edge via `readTurnState`; this only changes which existing
signal `complete` trusts.
**Throughput-neutral:** `runAttempt` returns on a non-empty
`readAssistantText` *before* `readTurnComplete` is consulted, so the
happy path is unchanged. Only a completed-but-empty turn's
classification changes. A genuinely-empty completed turn still reds
(correct) — only the race false-positive is removed.
## Red → green proof (deterministic, on the real code path)
A new deterministic timing-sim regression (`sseLeadMs` fixture)
decouples `sseDone` from `domDone` *in time* and drives the real
`readTurnComplete`/`runAttempt`. Two cases: a TIMING SIM (sseDone at
send, text at 2500ms past the ~2000ms grace window) and a FORMULA PIN
(sseDone-only, token never arrives → asserts `sends === 2`, catching a
literal revert to `domDone || sseDone` via the retry-fired
discriminator, not a wall-clock threshold).
**RED — test hunks applied on top of UNFIXED probe code:**
```
FAIL src/probes/drivers/d4-chat-roundtrip.test.ts > d4 render-race: sseDone-leads / domDone-lags turn completion (Fix B) > TIMING SIM: sseDone fires EARLY, domDone+text land LATE together → GREEN (RED pre-fix)
AssertionError: expected 'red' to be 'green' // Object.is equality
Expected: "green"
Received: "red"
❯ src/probes/drivers/d4-chat-roundtrip.test.ts:3195:21
FAIL src/probes/drivers/d4-chat-roundtrip.test.ts > d4 render-race: sseDone-leads / domDone-lags turn completion (Fix B) > FORMULA PIN: sseDone alone must NOT flip `complete` — a turn that only ever fires sseDone stalls+retries (does NOT fast-fail as completed-empty)
AssertionError: expected 1 to be 2 // Object.is equality
- Expected 2
+ Received 1
❯ src/probes/drivers/d4-chat-roundtrip.test.ts:3229:19
Test Files 1 failed (1)
Tests 2 failed | 82 skipped (84)
```
**GREEN — probe fix hunk then applied, same tests re-run:**
```
✓ src/probes/drivers/d4-chat-roundtrip.test.ts (84 tests | 82 skipped) 9026ms
✓ d4 render-race: sseDone-leads / domDone-lags turn completion (Fix B) > TIMING SIM: sseDone fires EARLY, domDone+text land LATE together → GREEN (RED pre-fix) 5017ms
✓ d4 render-race: sseDone-leads / domDone-lags turn completion (Fix B) > FORMULA PIN: sseDone alone must NOT flip `complete` — a turn that only ever fires sseDone stalls+retries (does NOT fast-fail as completed-empty) 4007ms
Test Files 1 passed (1)
Tests 2 passed | 82 skipped (84)
```
Full `d4-chat-roundtrip.test.ts` suite after the fix: **84 passed (84)**
— no regression.
## Real-probe forcing (Part B) — honest disclosure
The live race is low-rate and non-deterministic. The `--direct`
(direct-LLM) live-probe forcing path could **not** be stood up within
reasonable effort in this environment: Docker is running, but `--direct`
requires a real Anthropic API key that is not present here, and forcing
a probabilistic React-commit-timing race across ~300 reps on a
freshly-built `claude-sdk-python` integration stack is beyond a
reasonable bound for the bonus real-surface proof. It was **not** faked.
The deterministic timing-sim regression above exercises the real
`readTurnComplete`/`runAttempt` code path and is the gating proof.
**Recommendation:** a multi-tick staging dashboard watch post-merge to
confirm the flap does not recur.
## Scope note
This PR fixes the **producer-side** root cause. The **render-layer
de-amplifier** (a single transient D4 flap should degrade a slug's cells
to *amber*, not whole-column-red via slug-keyed `resolveD4` fan-out) is
owned separately by the dashboard-ladder redesign.
## CR follow-ups (post-review hardening)
Three small follow-ups from the 7-agent CR + adversarial verify (which
found zero live bugs in the Fix B core). The `complete: domDone` core is
untouched.
- **Regression-guard test (hardening).** New
`d4-probe-domdone-guard.test.ts` (source-level, TS AST). It asserts
every D4-probed demo page — `/demos/agentic-chat` for every integration,
`/demos/tool-rendering` where present — renders `<CopilotChat/>` in the
attribute-bearing **self-closing** form, not the children render-prop /
slot form. Only the self-closing form hits the `CopilotChatView` branch
that emits `data-testid="copilot-chat"` + `data-copilot-running`, which
the probe's `domDone` gate reads; the `if (children)` branch returns a
`display:contents` wrapper with neither, which would silently regress
the cell to the ~60s / double-send fallback. That regression is
invisible to the probe's own fake-injected unit tests, so it is guarded
structurally at CI time. Chose the source/AST approach over full page
render because the pages are Next.js `"use client"` app-router
entrypoints mounting a live `<CopilotKit runtimeUrl=.../>` provider +
framework agents (impractical to render faithfully in a harness unit
test).
**Red-green proof.** Temporarily mutated
`langgraph-python/agentic-chat/page.tsx` to the children render-prop
form → guard **RED** (`langgraph-python/agentic-chat … expected 0 to be
greater than or equal to 1`, 1 failed | 40 passed). Reverted the
mutation → **GREEN** (41 passed). Mutation fully reverted; net change is
the test only.
- **Comment factual fix.** In `d4-chat-roundtrip.ts` near the `complete:
domDone` gate: the comment said `observed` still includes `sseDone`
"(below)", but `observed` is defined *above*. Corrected "(below)" →
"(above)".
- **Timing-sim robustness pin.** The `TIMING SIM` test's discriminating
power depended on `FIRST_TOKEN_GRACE_MS (2000) < firstTokenDelayMs
(2500) < ceiling (~4000)`, encoded only in prose. Now imports the actual
`FIRST_TOKEN_GRACE_MS` source constant (newly exported) and asserts that
ordering, so raising the grace constant later fails the test loudly
instead of silently ceasing to discriminate. No existing assertion
weakened.
Quality: harness `d4-chat-roundtrip.test.ts` + new guard test green (125
tests); `tsc --noEmit` clean; `tsc -p tsconfig.build.json` clean; `oxfmt
--check` clean; `oxlint` warnings-only (no errors).
The comment near the complete:domDone gate said `observed` still
includes `sseDone` "(below)" — but `observed` is defined ABOVE the
comment, not below. Correct "(below)" to "(above)".
Add a source-level (TS AST) regression guard asserting every D4-probed
demo page (/demos/agentic-chat always; /demos/tool-rendering where
present) renders <CopilotChat/> in the attribute-bearing self-closing
form, not the children render-prop form. The self-closing form is the
CopilotChatView branch that emits data-testid="copilot-chat" +
data-copilot-running, which the probe's domDone completion gate reads;
the render-prop branch omits it, which would silently regress the cell
to the ~60s/double-send fallback path. Invisible to the probe's own
fake-injected unit tests, so guarded structurally at CI time.
Also pin the TIMING SIM test's discriminating window against the actual
FIRST_TOKEN_GRACE_MS source constant (now exported): assert
FIRST_TOKEN_GRACE_MS < firstTokenDelayMs < attempt-0 ceiling so that
raising the grace constant later fails the test loudly instead of
silently ceasing to discriminate pre-fix vs post-fix.
autoUpdates is now per-env: staging is enforced "disabled" while prod is
"unmanaged" (the drift gate skips it) so prod stays untouched until a later
migration. The gate enforces managed envs and skips unmanaged ones; the
zero-checked floor applies only to managed envs. Regenerates the SSOT JSON.
## What & why
Three independent documentation-accuracy fixes, batched into one PR.
### 1. Dead spec links + gen-ui page gaps (Closes#3975)
`generative-ui-specs-overview.mdx` (rendered at
`/whats-new/generative-ui-spec-support`) linked to the retired
`/generative-ui/specs/<spec>` subgroup. Repointed to canonical
destinations and added the frameworks list the issue asked for.
**Verified live (HTTP status against docs.copilotkit.ai):**
| Link | Before | After |
| --- | --- | --- |
| A2UI | `/generative-ui/specs/a2ui` (301 hop) | `/generative-ui/a2ui` →
**200** |
| MCP Apps | `/generative-ui/specs/mcp-apps` (301 hop) |
`/generative-ui/mcp-apps` → **200** |
| Open Generative UI (new) | — | `/generative-ui/open-generative-ui` →
**200** |
- **Supported Frameworks** list added — all 12 `/<slug>/quickstart`
targets return **200** live (LangGraph Py/TS, Google ADK, MS Agent, AWS
Strands, Mastra, PydanticAI, CrewAI, Agno, AG2, LlamaIndex, Claude Agent
SDK, Deep Agents).
- **Open-JSON-UI is intentionally NOT linked:**
`/generative-ui/open-json-ui` is a placeholder pulled from the nav and
redirected to `/generative-ui` on purpose (`next.config.ts` — `//
AI-slop placeholder pulled from nav until properly authored`). Linking
only the two specs that have live detail pages avoids sending readers to
a redirect. Open-JSON-UI is still described in the comparison table on
the page.
- Open Generative UI is a CopilotKit capability (not an external spec),
so it sits under "related capabilities."
> Note: the issue/support-bot suggested a `/learn/...` path — there is
no `/learn/` tree in shell-docs; the canonical homes are the flat
`/generative-ui/<spec>` pages.
### 2. CLI `init` vs. existing app (Closes#2525)
Maintainer resolution was "fix the docs." The original "`init`
bootstraps your existing Next.js app" claim was already removed in the
shell-docs migration (the reported `/direct-to-llm/guides/quickstart`
now resolves to the Built-in Agent quickstart, which is fully manual).
To remove the remaining ambiguity:
- **Built-in Agent quickstart:** added an "Already have an app?" callout
— existing apps skip `create-next-app`.
- **CLI guide (`cli.mdx`):** clarified that `create` (aliased `init`)
scaffolds a brand-new project in its own directory and does not
detect/bootstrap an existing app; points to the manual install in the
Quickstart.
Verified against `copilotkit@latest` (4.3.0): `init --help` →
*"Initialize a **new** CopilotKit project … before scaffolding"*, `-n,
--name` *"names the local app **and its directory**"*, and
`init`/`create` are aliases.
### 3. Codex setup for the MCP guide (Closes#2526)
Added a **Codex** section to `mcp-server-setup.mdx` using the stdio
`mcp-remote` bridge in `~/.codex/config.toml`, matching the page's
existing command-based pattern (Cursor / Windsurf / Claude Desktop),
plus the `codex mcp add` shortcut.
Verified against the installed Codex CLI: `codex mcp --help` lists
`add`/`list`/`get`/`remove`, and the `[mcp_servers.<name>]` table with
`command`/`args` matches OpenAI's Codex config reference. Per the issue
thread, the macOS `mcp-remote` port-blocking concern is **not** claimed
to be solved — only the Codex config is documented.
## Testing
- **#3975 (links):** curled every added/changed URL against the live
docs — A2UI, MCP Apps, Open Generative UI, and all 12 framework
quickstarts return **200**; confirmed `/generative-ui/open-json-ui` is a
deliberate redirect (hence unlinked). Pre-existing `/ag-ui-protocol` and
`/generative-ui` links are stable 301→200 and left as-is.
- **#2525 (CLI):** ran `npx copilotkit@latest init --help` on the
published `latest` (4.3.0) — confirmed new-directory scaffolding, no
existing-app detection; verified `[Quickstart](/quickstart)` serves the
manual-install page live (`create-next-app` + "Install CopilotKit
packages").
- **#2526 (Codex):** ran `codex mcp --help` to confirm subcommands; new
section reuses the file's existing `<Steps>`/fenced-code structure.
`<Callout>` is a registered global MDX component
(`src/lib/mdx-registry.tsx`), already used unimported on the Built-in
Agent quickstart.
- Docs-only; no code paths affected.
Closes#3975Closes#2525Closes#2526🤖 Generated with [Claude Code](https://claude.com/claude-code)
Adds a CI-owned reconcile (every 15m) that compares each staging service's
deployed digest against GHCR :latest and re-runs the staging redeploy for
lagging services, alerting Slack. The run is green only when every in-scope
service is confirmed current; any unconfirmed service (lag, digest error,
dropped redeploy, empty scope, or a thrown redeploy) alerts and exits non-zero.
Exposes per-service redeploy records from redeploy-env for accurate per-service
remediation confirmation.
New CI gate fails when a live Railway service's autoUpdates diverges from the
SSOT (every service must be disabled). Reads Environment.config (autoUpdates is
not on the typed ServiceSource output), fails closed per-env when it verifies
zero services, and skips cleanly on fork PRs that lack a Railway token.
autoUpdates was tracked nowhere and had drifted (24 services minor / 17 none).
Add an explicit disabled autoUpdates policy to every service in railway-envs.ts
and regenerate railway-envs.generated.json, making CI-explicit redeploy the
single deploy path instead of Railway's registry auto-watch.
The redeploy-staging and redeploy-staging-starters jobs guarded on
needs.build.result != 'cancelled', so a single cancelled matrix leg (e.g. the
Git-LFS shell build under runner contention) skipped the whole fleet's staging
redeploy even when the other 27 services built fine. Relax both guards to
redeploy the already-computed successful-service list. Adds a guard-evaluation
test that reads the live workflow if: strings and models GitHub's matrix rollup.
readTurnComplete() gated turn completion on `domDone || sseDone`. sseDone (the
transport-level RUN_FINISHED counter) is a synchronous raw-byte parse OUTSIDE
React and can fire at-or-before the React commit that renders the assistant
text; domDone is coalesced into that SAME commit (use-agent.tsx
batchedForceUpdate / queueMicrotask). Gating on sseDone let the fast-fail grace
window elapse with the DOM still empty under contention, redding a turn that was
about to render correctly ("empty assistant response").
Gate completion on domDone alone. observed still includes sseDone, so a turn
whose attribute never appears still gets the wider polling window. Throughput-
neutral: runAttempt returns on non-empty readAssistantText before
readTurnComplete is consulted, so the happy path is unchanged; a genuinely-empty
completed turn still reds.
Add a deterministic timing-sim regression for the D4 chat-roundtrip probe
render race. The sseLeadMs fixture decouples the transport RUN_FINISHED
counter (sseDone) from the DOM run-stop edge (domDone) in time, so a turn
whose sseDone fires early while domDone + assistant-text land later in the
same React commit is exercised on the real readTurnComplete/runAttempt path.
Two cases: a TIMING SIM (sseDone at send, text at 2500ms past the grace
window) that reds pre-fix, and a FORMULA PIN (sseDone-only, token never
arrives) asserting sends===2, catching a literal revert to domDone||sseDone
via the retry-fired discriminator rather than a wall-clock threshold.
CR: /generative-ui/open-json-ui is a deliberately-unpublished placeholder
(next.config.ts redirects it to /generative-ui, pulled from nav until
authored), so link only the two specs with live detail pages (A2UI, MCP
Apps). Open Generative UI is a CopilotKit capability rather than an
external spec, so it moves under related capabilities.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes three independent documentation-accuracy issues:
- #3975: repoint the retired /generative-ui/specs/<spec> links to the
canonical flat /generative-ui/<spec> paths, add an Open Generative UI
entry, and add a Supported Frameworks list on the Generative UI Spec
Support page.
- #2525: clarify that the CLI `create`/`init` scaffolds a brand-new
project in its own directory and does not bootstrap an existing app;
point existing-app users to manual installation.
- #2526: add a Codex setup section to the MCP server guide using the
stdio `mcp-remote` bridge, matching the page's existing pattern.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## What & why
The `showcase_promote_notify` workflow posts an init message to
`#team-showcase`, then a threaded reply summarizing the run outcome.
Operators wanted the *net* result visible at a glance on the original
init message without clicking into the thread.
This adds an emoji **reaction** to the original init message reflecting
the run outcome via Slack's `reactions.add` API.
### Outcome → emoji mapping
| `outcome` | reaction name | emoji |
|-----------|---------------|-------|
| `success` | `white_check_mark` | ✅ |
| `partial` | `warning` | ⚠️ |
| `total` | `x` | ❌ |
### Implementation notes
- **`showcase_promote_notify.yml`** — after the thread reply, maps
`outcome` → reaction name with a `case` and calls the existing
`slack_api` helper with `reactions.add` (body `{channel, timestamp,
name}` — note `timestamp`, not `ts`, for `reactions.add`). Guarded: only
attempts when both `init_ts` and `init_channel_id` are non-empty (init
post may have failed → skip with a `::warning::`). The reaction is
informational, so a failed add is **warn-only** (`|| true`), mirroring
the thread-reply idiom — NOT the fail-loud `#oss-alerts` page.
- **`showcase_promote_notify.dry-run.sh`** — makes no real Slack calls,
so it emits the reaction it *would* add as a `--- reactions.add ---`
block. The `outcome`→reaction-name `case` is byte-identical to the
`.yml`.
- **`promote-notify.bats`** — new tests assert the emitted reaction name
per fixture, plus a new anti-drift parity guard that the reaction
mapping matches between the `.yml` and the `.sh` mirror (mirroring the
existing `slack_alert_posted_ok` parity test).
## Red-Green Proof
The real failure surface is the dry-run harness + bats suite.
### RED — before the change
Dry-run on all three fixtures emits **no** `reactions.add` line (`grep
-c` = 0 each):
```
########## RED dry-run: success.json ########## -> grep reactions.add: 0
########## RED dry-run: partial.json ########## -> grep reactions.add: 0
########## RED dry-run: total-failure.json ##### -> grep reactions.add: 0
```
New bats tests fail (existing 1–11 pass):
```
ok 11 call-site: both posts ok → zero exit, no warning
not ok 12 reaction: success outcome adds white_check_mark to the init message
not ok 13 reaction: partial outcome adds warning to the init message
not ok 14 reaction: total-failure outcome adds x to the init message
not ok 15 reaction: yml and sh reaction-name mapping is identical (anti-drift)
```
### GREEN — after the change
Dry-run emits the correct reaction per outcome:
```
########## GREEN dry-run: success.json ##########
--- reactions.add ---
channel: #team-showcase (ts=<init_ts>)
name: white_check_mark
########## GREEN dry-run: partial.json ##########
--- reactions.add ---
channel: #team-showcase (ts=<init_ts>)
name: warning
########## GREEN dry-run: total-failure.json ##########
--- reactions.add ---
channel: #team-showcase (ts=<init_ts>)
name: x
```
Full bats suite passes (15/15):
```
ok 1 slack_alert_posted_ok: ok:true response returns 0 and emits no warning
...
ok 11 call-site: both posts ok → zero exit, no warning
ok 12 reaction: success outcome adds white_check_mark to the init message
ok 13 reaction: partial outcome adds warning to the init message
ok 14 reaction: total-failure outcome adds x to the init message
ok 15 reaction: yml and sh reaction-name mapping is identical (anti-drift)
```
`shellcheck .github/workflows/showcase_promote_notify.dry-run.sh` →
clean (exit 0).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01F6rwtyRSUFtgiAY2dYuyU2
Add an emoji reaction to the original promote-notify init message
reflecting the net run outcome, so operators can see success/failure at
a glance without opening the thread reply:
success -> white_check_mark (checkmark)
partial -> warning
total -> x
The live workflow calls reactions.add on the init post (guarded on a
successful init post, warn-only on failure to mirror the thread reply).
The dry-run harness emits the reaction it would add, using a
byte-identical case mapping enforced by a new anti-drift bats guard.
Adds bats coverage asserting the emitted reaction name per fixture.
Bumps the showcase's `@copilotkit/aimock` dependency from `1.26.1` to
**`1.37.4`** so the showcase's record/replay fixture stack picks up the
multi-turn fixture-matching fix.
**Why:** aimock v1.37.4 (aimock #319) fixes record/replay
`hasToolResult` symmetry — recorded multi-turn fixtures now match on
replay. The showcase records/replays LLM fixtures for its demo cells, so
this pulls the fix into the showcase.
**What changed**
- `showcase/scripts/package.json`: `@copilotkit/aimock` `1.26.1` →
`1.37.4` (exact pin, matching existing style)
- `showcase/scripts/package-lock.json`: regenerated via `npm install
--package-lock-only` (this lock is consumed by `npm ci` in the
shell/shell-docs/shell-dashboard Dockerfiles)
- `pnpm-lock.yaml`: regenerated via `pnpm install --lockfile-only`
(showcase/scripts is a pnpm workspace member)
The only version change is aimock (and its transitive tree in the npm
lock). The minor eslint-config-next peer-resolution relabeling in
`pnpm-lock.yaml` is benign normalization — no package version changes;
both jiti variants already existed in the lock.
**Validation gap:** this is a dependency bump only. Full validation
(running the showcase multi-turn recording against 1.37.4 to confirm the
recorded fixtures match on replay) requires the running showcase stack
and is not exercised here.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Supersedes #6058's two-stage approach (outer `generate_a2ui` → secondary
Python LLM call → `render_a2ui`), which severed across the prod
streaming boundary.
**Option A** (mirroring crewai-crews #6067): `injectA2UITool` defaults
to `true`, so CopilotKit's `A2UIMiddleware` injects `render_a2ui` into
`RunAgentInput.tools`. The langroid `agui_adapter.py` now merges those
injected tools into its OpenAI call, so the LLM calls `render_a2ui`
directly. The middleware intercepts the tool call stream, builds
`a2ui_operations`, and fires `RUN_FINISHED` — no secondary Python LLM
pass needed.
## Changes
- **`agent.py`**: Remove ~550 lines of two-stage A2UI infrastructure
(`generate_a2ui_via_llm`, `_a2ui_error`, `_resolve_a2ui_model`,
`_get_a2ui_llm`, `_RENDER_A2UI_FUNCTION_SPEC`, etc.). Replace
`GenerateA2UITool.handle` with a stub that logs loudly on regression
(middleware should always intercept before reaching Python).
- **`agui_adapter.py`**: Merge `run_input.tools` (AG-UI-injected) into
the OpenAI tools list so `render_a2ui` is visible to the LLM. Remove
`set_last_user_message` call (ContextVar no longer needed).
- **`route.ts`**: Remove `injectA2UITool: false`; keep
`defaultCatalogId` pin.
- **`gen-ui-declarative.json`**: Replace 9 two-stage fixtures with 4
single-stage fixtures matching `toolName: render_a2ui` + `context:
langroid`.
## Root cause of prior RED
The langroid adapter builds its OpenAI tool list from `ALL_TOOLS`
(Python-side registry) via `_get_openai_tools()`, which does NOT include
`render_a2ui`. The `A2UIMiddleware` injects `render_a2ui` into
`RunAgentInput.tools` at the AG-UI protocol level, but `agui_adapter.py`
ignored `run_input.tools` entirely — so the LLM never saw `render_a2ui`
in its tool list, never called it, and the fixture never matched.
## Red-green proof
**RED** (from main, before changes):
```
✗ d6:langroid/gen-ui-declarative red (0.0s)
state=red
0 passed, 1 failed
```
**GREEN** (after this PR's changes, rebuild from worktree):
```
✓ d6:langroid/gen-ui-declarative green (0.0s)
1 passed
✓ Tests passed for langroid:declarative-gen-ui
```
Test command: `bin/showcase test langroid:declarative-gen-ui --d6
--isolate --rebuild`
## Related
- Supersedes #6058 (two-stage approach, now reverted in this
integration)
- Mirrors #6067 (crewai-crews Option A fix, same pattern)
Benjamin Taylor