## Summary
Adds a `react-version` matrix axis (**18**, **19**) to the unit-test
workflow so react-core, react-ui, and a2ui-renderer are exercised across
the full **supported peer range** (`^18 || ^19`), not just the
repo-default React 19.
This is a **reconstruction of the durable parts of #4221**
(@tylerslaton) onto current `main`. That PR went stale (~5,000 commits
behind, conflicting) and never landed. Rather than rebase it, this
rebuilds its design fresh — and deliberately **scopes to the supported
React range**: React 17 is dropped, because it is no longer a supported
peer version and carried ~80% of the original PR's complexity
(polyfills, `use-sync-external-store` source shims, `jsx-runtime`
aliases, a legacy `renderHook` fallback).
## What surfaced
Dropping R17 and validating R18 revealed a **latent React 18
incompatibility on current `main`**: the `window = {}` test pattern
crashes React 18's concurrent renderer with `"Should not already be
working."` mid-commit, which then corrupts the scheduler for the rest of
the file — **22 failures across 5 files** under React 18. (React 19
happens to tolerate the empty-window swap, so it was invisible until
now.)
The original PR fixed this but mislabeled it R17-only; it's actually
needed for R18, a *supported* version. So the matrix earned its keep on
day one.
Replacing `window = {}` with `stubWindowLocation()` is the load-bearing
fix — it resolves the crash cascade. Separately, **two** tests differ
under R18 purely in *render scheduling*, and are handled by narrow
version gates:
| Test | React 18 behavior | Why it's not a bug |
|---|---|---|
| `renderCustomMessages` → "executes multiple renderers in order" |
`executionOrder` is `["first", "first"]` | Renderer double-invoke.
`second` still never runs, which is the actual contract. |
| `use-human-in-the-loop` → `statusHistory` | `inProgress → executing →
inProgress → complete` | Transient backwards transition from extra
effect runs. Start, end, and the set of observed statuses are all still
correct. |
**No assertion tolerates a different state value.** An earlier revision
of this PR also relaxed the three-turn state-snapshot assertion to
accept `Turn: 2` on R18; @tylerslaton correctly flagged that as an
observable-behavior difference rather than a scheduling artifact.
Re-verified against a real 18.3.1 install — the strict `Turn: 3`
assertion passes **25/25** consecutive runs — so that gate was
unnecessary and has been removed (`a227f46a8`). The two gates above were
re-tested the same way and both genuinely reproduce.
## Changes
| File | What |
|---|---|
| `.github/workflows/test_unit.yml` | `react-version: ["18","19"]` axis.
R19 installs frozen; R18 overrides the root `pnpm.overrides` React
version and installs unfrozen. Adds a guard verifying the installed
React matches the matrix leg, and suffixes `NX_CI_EXECUTION_ID` with the
React version. Layered on top of the existing nx-affected selection
logic. |
| `test-helpers/stub-window-location.ts` *(new)* | Clears
`window.location` (so the localhost auto-open-inspector heuristic skips)
while keeping the real jsdom window — the safe replacement for `window =
{}`. |
| `use-agent-error-state`, `CopilotKitProvider.onError`,
`CopilotKitProvider.test` | Swap `window = {}` for
`stubWindowLocation()`. |
| `use-human-in-the-loop.e2e`, `renderCustomMessages.e2e` | Two
React-version-gated assertions, both **render-scheduling only** (see
table above). State assertions stay strict on every leg. |
No dependency or lockfile changes. None of the R17-only machinery from
#4221.
## CI cost
Full runs go from 3 legs (node 20/22/24) to **6** (node × react). On
PRs, nx-affected still scopes what actually builds/tests; the full 6×
only hits `workflow_dispatch` or when `test_unit.yml` itself changes (so
this PR runs all 6). This is the honest price of adding R18 coverage.
## Testing
Run locally in a worktree via the exact install-override logic the
workflow uses — `react`/`react-dom` → 18.3.1,
`@types/react`/`@types/react-dom` → `^18`, `@testing-library/react` →
`^14.3.1`, `streamdown>react` → 18.3.1, then `pnpm install
--no-frozen-lockfile`. Installed versions confirmed by resolving from
`packages/react-core` (18.3.1 / 19.2.3, `@testing-library/react` 14.3.1
on the R18 leg).
| Check | Result |
|---|---|
| react-core full suite @ React 18.3.1 | **117 files, 1433/1433
passing** ✓ |
| react-core full suite @ React 19.2.3 | **117 files, 1433/1433
passing** ✓ |
| Strict `Turn: 3` state-snapshot assertion @ R18, ×25 runs | **25 pass
/ 0 fail** — gate removed as unnecessary |
| `executionOrder` gate reverted to strict @ R18 | **fails**
(`['first','first']`) — gate justified |
| HITL `statusHistory` gate reverted to strict @ R18 | **fails** (extra
`inProgress`) — gate justified |
| `oxlint` (project-aware) | **0 warnings, 0 errors** — unchanged from
`main` |
| `oxfmt --check` | clean |
| Workflow YAML parse + lefthook commit hooks (lint-fix, package tests,
commitlint) | green |
Before the `window` fix, the R18 leg was **22 failing across 5 files**;
it is now fully green.
Credit to @tylerslaton for the original design in #4221, and for
catching the over-relaxed state assertion in review.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Preserve the logical run ID when a legacy `useCopilotAction({
renderAndWaitForResponse })` frontend tool resolves and
`processAgentResult` starts its recursive follow-up.
The run handler binds each internal continuation handoff to the exact
follow-up invocation, cancels it when setup fails or no run starts, and
keeps the handoff out of the public `CopilotKitCore.runAgent` contract.
The public regression drives the legacy hook through its
`useHumanInTheLoop` and `useFrontendTool` path, renders the approval
control, resolves it, and verifies both agent calls use the same
generated ID.
Closes https://github.com/CopilotKit/CopilotKit/issues/3456
## Changes
- Preserve the originating ID across recursive frontend-tool follow-up
runs
- Keep the existing legacy HITL registration and response behavior
unchanged
- Add core follow-up coverage and a public `useCopilotAction` regression
- Retain the existing standard/legacy interrupt and StateManager
coverage from the earlier fix
## Test plan
- [x] `pnpm -C packages/react-core exec vitest run
src/hooks/__tests__/use-copilot-action.e2e.test.tsx`
- [x] `pnpm -C packages/core exec vitest run
src/__tests__/core-follow-up.test.ts`
- [x] `pnpm -C packages/react-core exec vitest run
src/v2/hooks/__tests__/use-interrupt.test.tsx`
- [x] `pnpm -C packages/core exec vitest run
src/__tests__/state-manager.test.ts`, 39 tests passed
- [x] `pnpm -C packages/react-core exec vitest run`, 123 files and 1475
tests passed
- [x] `pnpm -C packages/core exec vitest run`, 58 files and 625 tests
passed
- [x] `pnpm -C packages/core run check-types`
- [x] `pnpm -C packages/react-core run check-types`
- [x] `pnpm exec oxfmt --check` on all eight changed source/test files
- [x] `pnpm exec oxlint` on all eight changed source/test files, 5
pre-existing warnings and 0 errors
Approve/Deny buttons could reappear on a card the user had **already
answered**.
The clearest case is the teach arc: after "Open policy exception" is
approved and the chain moves on, that first card comes back showing live
buttons, while the two later cards correctly read "Response submitted."
## Root cause
`"Response submitted."` was purely local `useState`. Nothing tied it to
the tool call, so any remount lost it — and the earliest card in a
multi-step chain has its subtree replaced when the run syncs. That's why
card 1 regresses and cards 2 and 3 don't.
The visual glitch is the mild part. The buttons are live: clicking them
again fires a duplicate write against an already-settled call.
## Fix
`ApprovalButtons` takes a `resolved` prop carrying the durable signal
from the tool call itself, OR-ed with local state:
- local state collapses the buttons instantly on click, without waiting
for the round trip
- `resolved` survives remounts and thread reloads
Same result-over-status rule already applied to the PIN and charges
cards in #6259.
## Scope
Wires the 3 call sites that had no guard. The 3 that already collapse on
`result` are unchanged — there TypeScript correctly rejects `status ===
"complete"` as unreachable, which is a useful signal that the guard is
already in place.
The identical bug existed in `CopilotKit/fde` `reskinnable-demo`, where
10 of 13 call sites were unguarded. Fixed there separately (`58d400b` on
that repo's main).
## Verification
`tsc --noEmit` and `oxlint` clean.
Not yet exercised in a browser: reproducing it means running the teach
arc through to the AWS climax. The change is mechanical and
type-checked, but flagging that the runtime behaviour is reasoned rather
than observed, since on this demo green checks have not always meant the
surface is right.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Approve/Deny buttons could reappear on a card the user had already answered.
The clearest case is the teach arc: after "Open policy exception" is approved
and the chain moves on, that first card came back showing live buttons while
the later cards read "Response submitted."
"Response submitted." was purely local useState, so nothing tied it to the tool
call. Any remount lost it, and the earliest card in a multi-step chain has its
subtree replaced when the run syncs. Clicking the resurrected buttons would
fire a duplicate write against an already-settled call.
ApprovalButtons now takes a `resolved` prop carrying the durable signal from
the tool call itself, OR-ed with the local state: local collapses the buttons
immediately on click, `resolved` survives remounts and thread reloads. This is
the same result-over-status rule already applied to the PIN and charges cards.
Wires the 3 call sites that had no guard. The 3 that already collapse on
`result` are unchanged — there TypeScript correctly rejects the status
comparison as unreachable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
## Problem
The OSS/SSE runtime path — `CopilotRuntime`/`CopilotSseRuntime` built
**without** an Intelligence backend — uses `InMemoryAgentRunner`, which
kept all thread history in a process-global `Map` that nothing ever
bounded. A production Node.js deployment (`--max-old-space-size=4096`)
hit a fatal V8 OOM after ~173 threads / ~15.5k runs. Three unbounded
growth vectors: thread count, runs per thread, and per-run
`ReplaySubject(Infinity)` buffers.
## Fix
Make the in-memory runner **safe by default**, zero-config and
dependency-free.
- **`ɵBoundedThreadStore`** owns the `Map`, LRU ordering, approximate
byte accounting, and eviction. `InMemoryAgentRunner` keeps all streaming
logic and delegates storage.
- **Three bounds, whichever trips first:** `maxThreads` (**1000**, LRU),
`maxRunsPerThread` (**100**, FIFO; `Infinity`/`0` disables), `maxBytes`
(**512 MiB**, the primary guard).
- **Never evicts** a running thread, one still finalizing after
`stop()`, or the thread currently being created/appended — it accepts
temporary overage instead.
- **Thread-level `messagesSnapshot` and `createdAt`**, decoupled from
`historicRuns`, so run-cap eviction can neither drop the message history
nor drift a thread's reported creation time.
- **Subject-buffer release:** `ReplaySubject(Infinity)` is retained only
while a run is active, released on completion once events are in
`historicRuns`.
- **Both eviction paths log**, through one warn-once-per-store latch:
whole-thread eviction *and* per-thread run-cap trimming.
- **Limits validation:** `ɵnormalizeLimits` rejects negative / `NaN` /
non-integer / `-Infinity` values at construction and at `setLimits`,
clamping to defaults with a warning. Previously `maxRunsPerThread: -1`
crashed with a `TypeError` inside a fire-and-forget path.
- **Partial limit updates preserve siblings** — tuning one bound no
longer silently resets the others process-wide.
## API
Additive and non-breaking. Bounds ride on the existing options bag
introduced by `onConcurrentRun`:
```ts
export interface InMemoryAgentRunnerOptions extends InMemoryLimits {
onConcurrentRun?: "throw" | "supersede";
}
new InMemoryAgentRunner(); // unchanged, safe defaults
new InMemoryAgentRunner({ onConcurrentRun: "supersede" }); // unchanged
new InMemoryAgentRunner({ maxThreads: 200, maxBytes: 128 * 1024 ** 2 });
```
Note the differing scopes, documented on both surfaces: the limits
reconfigure the **process-global** store; `onConcurrentRun` is
**per-runner**. Passing only `onConcurrentRun` leaves limits untouched.
## Interaction with `onConcurrentRun: "supersede"`
Rebased onto `main` after supersede landed, which surfaced a bug neither
change had alone. A superseded run's teardown was not isolated from the
run that replaced it: the new run cleared the shared `stopRequested`, so
the superseded run finalized as a synthetic `RUN_ERROR`, and the
`prevSubject` bridge forwarded that terminal event — plus a replayed
`RUN_STARTED` — into the **live** run's subject. `connect()` consumers
replayed a duplicate `RUN_STARTED` followed by a dangling `RUN_ERROR`.
Fixed by capturing per-run finalize intent (`RunFinalizeControl`),
widening the concurrency gate to `isRunning || stopRequested`, removing
the leaking bridge, and identity-guarding the subject release. Both
teardown paths were extracted into one shared `finalizeRun` helper so
they cannot diverge again.
## Docs
Neither constructor option was documented before this PR.
- `backend/agent-runner.mdx` — the bounds and their defaults, the
**eviction model stated once** (whole-thread eviction removes the thread
from `GET /threads`; run-cap trimming keeps it, preserving `createdAt`
and the message snapshot; the warning is once-per-store, reset by
`clearThreads()`), the `onConcurrentRun` modes, and durability options
including the first-party `SqliteAgentRunner`.
- `troubleshooting/common-issues.mdx` — growing runtime memory and the
eviction warning, cross-linked both ways.
- Runtime skill references refreshed — they still described a removed
`globalThis`-keyed store with HMR backup — and their source citations
re-anchored on stable symbols rather than line numbers.
## Tests
**1939 passing, 0 skipped** across 133 files (`@copilotkit/runtime`);
`check-types`, `build`, `publint`, and `attw` all clean.
- `ɵBoundedThreadStore` units: thread-LRU, run-cap FIFO + opt-out,
byte-ceiling eviction, never-evict-running, never-evict-finalizing (both
count and byte paths), snapshot dedup, `createdAt` stability past the
run cap, warn-once, `clear()` reset.
- Limits validation: every field × every invalid shape, plus proof the
`0`/`Infinity` sentinels still disable the run cap.
- Supersede/stop: teardown isolation with **negative** assertions (no
phantom `RUN_ERROR`, exactly one `RUN_STARTED` for a `connect()`
subscriber on a superseded thread), plus `stop()`'s reachable rollback
branches.
- **OOM guard:** drives a sustained workload across the thread-count,
runs-per-thread, and byte bounds and asserts eviction through the
runner's public surface. It replaces an earlier heap-plateau test that
was both `--expose-gc`-gated (so it never ran in normal CI) and
non-discriminating (its workload crossed no bound, so it passed with
bounding removed). The replacement was verified to fail when
`enforceRunCap` is neutered.
## Caveats & follow-ups
1. **Non-durable by design.** Eviction is history loss, same as a
restart. The durable paths are `SqliteAgentRunner` (single instance) or
the Intelligence runner.
2. **Store stays process-global.** Pre-existing cross-endpoint bleed and
process-wide `clearThreads()` are unchanged; instance-scoping is a
follow-up needing maintainer sign-off.
3. **Multiple runners with differing limits → last-constructed wins**,
with a one-time warning. Partial updates now preserve unspecified
bounds.
4. **Byte accounting is approximate** (serialized-length estimate), not
exact heap bytes, and under-counts non-serializable payloads.
5. **Behavioral change to a default path.** History that previously
persisted until OOM now evicts past the limits — shipped as a `fix`
(crash → bounded), observable at scale. Run-cap trimming now also logs
once per store.
6. **`maxBytes` is a cross-thread ceiling.** It evicts other LRU threads
and never trims the just-appended thread, so a single hot thread is
bounded by `maxRunsPerThread` only — which is why the docs steer away
from disabling that cap.
7. **Dedup weakens past the run cap.** `historicMessageIds` is rebuilt
from `historicRuns`, so a thread beyond `maxRunsPerThread` can
re-present already-seen messages. Documented with its escape hatch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## What does this PR do?
Reworks the repo-local `reskin` skill so it teaches **what a skin's demo
has to prove**, not just how to wire the `Skin` contract — based on
David's walkthrough of the banking demo (the "beats" and the why behind
each one), plus the layout facts from #6394.
Docs only. No code changes.
## Why
The skill documented the contract thoroughly and the demo not at all. I
tested that gap before writing anything: gave a fresh agent the **old**
skill and asked it to plan a new BI skin.
It returned a technically sound plan — correct contract usage,
server-emitted a2ui, deps arrays, OGUI sandbox, dark mode — that missed
most of the beats, and substituted its own thesis (RBAC governance) for
the demo's.
| Beat | Old skill | Reworked skill |
| --- | --- | --- |
| 1. Lead with generative UI | ✅ | ✅ |
| 2. Rich thread survives reload | ❌ absent | ✅ explicit step; renders
keyed off `result` |
| 3a. Drive the app, secret withheld | ⚠️ reframed as RBAC | ✅
confidential figure never enters the transcript |
| 3b. "What's on my screen?" | ❌ **zero readables planned** | ✅ route +
per-page readables, asked on two pages |
| 3c. Navigate via real levers | ⚠️ plain `navigateTo` | ✅ HITL confirm
→ query params → highlighted controls |
| 3d. Multimodal → durable artifact | ⚠️ no durability | ✅ artifact
outlives thread deletion (demonstrated) |
| 4. Long-term memory recall | ⚠️ one line | ✅ seeded preference +
recall-first + names what it recalled |
| 5. Stored-procedure replay | ❌ absent | ✅ seeded procedure, 3 visible
writes, distractors |
| 6. Teach a new procedure | ❌ absent | ✅ symptom-only gate, decoys,
record → save → replay |
| Presenter Reset | ❌ | ✅ |
| Pills covering the flow | 6, partial | 8, one per beat |
Same prompt, same conditions, only the skill changed. The telling detail
is 3b: `useAgentContext` was already in the skill's contract table, so
the agent *knew* the field existed — it just had no reason to prioritise
it. Contract docs say what is available; only choreography says what is
required.
## What changed
**`.claude/skills/reskin/demo-beats.md`** (new) — nine beats, each
framed by *what the audience must conclude*, then what the skin needs,
then banking's implementation with file:line. Plus seeding rules, the
presentation requirements (a pill per beat so the presenter never types,
a visible affordance on every mutation, pretty markdown prose, Reset,
the chat-placement framing), the vertical shortlist with BI called out
as highest-stakes, the quality bar, and a "which skin to copy for what"
routing table.
The beats are a **strong default that an explicit instruction
overrides**, expressed as a beat map with one row per beat that records
deliberate skips. Omission failures respond to a slot you have to fill
in, not to exhortation.
**`SKILL.md`** — beat table and override rule up front; step 0 is the
beat map, before code. Two new silent-failure rules found by tracing
banking:
- Renders must key off the tool `result`, not `status`, or they go blank
on thread replay — exactly when beat 2 is being demonstrated. Only
banking does this today.
- Beat 3b needs a route readable plus per-page on-screen readables. All
four skins register readables; only banking registers those, which is
why the beat is impossible in the other three.
Verification now ends with a 10-point demo walkthrough, since a green
build proves the wiring and nothing about the demo.
**`templates.md`** — pills-per-beat template; `seed-memories.ts` against
the real `POST /api/memories` shape (long-term memory is a seeded file,
not emergent — including why beat 6's procedure must never be seeded,
and why a seeded procedure must run with no confirmation gate, which
previously left an unresolved tool call that failed the next message); a
prompt-clause table mapping clauses to beats; replay-safe render; route
and per-page readables.
**`CLAUDE.md`** — claimed four skins in the intro and then documented
two. Now covers all four across both substrates with a beat-coverage
matrix, and corrects three false claims:
- `identifyUser` was attributed to banking alone; logistics and keel
contribute one too.
- An unsupported claim that a `Panel`'s `id` becomes its emitted
`data-testid` — the cited header comment says no such thing and the
installed package has no such behaviour. Replaced with what it actually
documents.
- "two implementations" in Reference.
Also notes keel's parameterized routes (the only skin with them) and the
shell-mounted inspector that replaced the glass engine.
## Notes for reviewers
- Verified against source rather than assumed: the entire `Skin`
contract field table in both docs was already accurate and is unchanged;
so were the Commands section, all panel-size/breakpoint/radius numbers,
the `.d.cts` filename the skill cites, and `defaultSkinId`. There were
no glass-engine references left to remove.
- Two follow-ups surfaced but deliberately not included here:
`banking/components/wow/proactive-notice.tsx` is an `animate-ping` bell
toast that is mounted nowhere, and logistics/keel already ship the full
per-user identity plumbing with no memory prompts, tools or seed file —
the cheapest beat-4 upgrades available if we want existing skins pulled
up to the bar.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The skill documented how to wire the `Skin` contract and nothing about what
the resulting demo has to demonstrate. Tested by asking a fresh agent to plan
a new BI skin with only the old skill: it produced a technically sound plan
that missed most of the banking demo's beats — no rich-thread step, zero
`useAgentContext` readables, no stored-procedure or teach-mode arc — and
substituted its own thesis (RBAC governance) for the demo's. Contract
documentation says what is available; only choreography says what is required.
Adds demo-beats.md: the nine beats framed by what the audience must conclude,
each with banking's implementation cited, plus the presentation requirements
(a pill per beat so the presenter never types, a visible affordance on every
mutation, pretty prose, Reset, the chat-placement framing), the domain
shortlist and the quality bar. The beats are a strong default an explicit
instruction can override, expressed as a beat map with a row per beat that
records deliberate skips — omission failures respond to a slot you must fill,
not to exhortation. Re-running the same planning task with the reworked skill
covers all nine.
Also folds in what tracing banking turned up, all of it silent-failure class:
renders must key off the tool `result` and not `status` or they go blank on
thread replay, exactly when "reload and the chart is still there" is being
shown; "what's on my screen?" needs a route readable plus per-page on-screen
readables, which is why that beat is impossible in the three other skins; and
long-term memory is a seeded file, not emergent, so seed-memories.ts gets a
template — including why beat 6's procedure must never be seeded and why a
seeded procedure must run without a confirmation gate.
Corrects CLAUDE.md, which claimed four skins and then documented two. Now
covers all four across both substrates with a beat-coverage matrix, fixes the
`identifyUser` attribution (logistics and keel contribute one too), drops an
unsupported claim that a Panel's `id` becomes its `data-testid`, notes keel's
parameterized routes, and acknowledges the shell-mounted inspector that
replaced the glass engine.
Co-Authored-By: Claude <noreply@anthropic.com>
Owner preference for the 5.x line. Recorded honestly: this reduces the
empty-diagram failure but does not remove it.
Measured against the real Excalidraw MCP server (same system prompt, real
tool schema, via the Responses API the AI SDK actually uses):
gpt-4o-mini create_view OK 3, isError 5
gpt-5.4 create_view OK 7, isError 3
gpt-4.1 create_view OK 8, isError 0
gpt-5.5 create_view OK 10, isError 0
JSON validity of the `elements` argument:
gpt-4o-mini 7 invalid of 12
gpt-5.4 7 invalid of 28, plus two runs whose tool call came back
garbled with unrelated spam text
gpt-5.4 + a hardened prompt 2 invalid of 16 (prompting does not fix it)
gpt-4.1 0 invalid of 12
gpt-5.5 0 invalid of 16
So roughly 30% of diagrams still render as an empty iframe on gpt-5.4. Closing
that gap needs a follow-up, most likely validating or repairing the `elements`
string before the MCP call rather than relying on the model to hand-escape
nested JSON correctly.
Gold `beautiful_chat.py` uses `ChatOpenAI(model="gpt-5.4")`; align the mastra
beautifulChatAgent to the same model (was gpt-4o). Verified live: dashboard turn
stays [query_data, generate_a2ui] (no standalone chart over-call), full surface
renders (3 metrics + pie + bar).
finish is optional on the renderer type, so calling it unguarded fails tsc
even though vitest is happy. Matches the non-null assertions the rest of the
file uses for subscriber callbacks.
PNI-121's reported symptom was "the weather appears after 'Find flights from
SFO to JFK'". Two separate things produced that screenshot; the blank flight
rows were the previous commit. This is the other one.
On gpt-4o this agent opened the flights turn by restating the PREVIOUS turn's
weather ("The weather in San Francisco is currently 20C with heavy rain.")
before narrating the flights, which reads exactly like a tool result arriving
a turn late. Nothing was actually late: only one weather card exists and it
stays in turn 1, and the flights turn's card holds the flights turn's data.
The system prompt is already byte-identical to gold tool_rendering_agent.py,
so the model was the remaining divergence - gold runs gpt-5.4.
Verified live (real LLM, ticket's exact click order - "Weather in SF" then
"Find flights"): the stale weather sentence is gone and the narration now
matches gold's shape ("SFO -> JFK options: United UA231 08:15-16:45 $348; ..."
against gold's "Flights SFO -> JFK: United UA231 08:15-16:45 $348; ...").
Weather, stock and d20 pills re-checked on the same rig.
No effect on CI: aimock fixtures never match on model, and the d20 pill's
5-roll chain is fixture-scripted (7/14/3/19/20), so the e2e sequence is
unchanged. Live, gold rolls the d20 once too - so this moves the demo toward
gold rather than away from it.
recorded.json baked in a real 2026-05-15 gpt-4o session that, for the beautiful-
chat Sales Dashboard pill, called standalone pieChart+barChart at turnIndex 2
AFTER already calling generate_a2ui — so under aimock the dashboard painted loose
charts next to the A2UI surface (the same over-call the live fix removed via
parallelToolCalls:false + steering). Replace that turnIndex-2 leg's
pieChart+barChart tool calls with narration so the aimock chain is
query_data -> generate_a2ui -> narration, matching the fixed live behavior. The
now-redundant turnIndex-3 narration leg is folded in.
Issue: asking for the Sales Dashboard (esp. after a prior turn) made the model
call the standalone pieChart + barChart frontend tools AND generate_a2ui, so
loose charts painted next to the dashboard.
- beautifulChatAgent: mirror gold `beautiful_chat.py` `parallel_tool_calls=False`
(defaultOptions.providerOptions.openai.parallelToolCalls) + sharpen the
steering so a dashboard / "using A2UI" request calls generate_a2ui ONLY (it
draws the charts inside the surface), while a single-chart request still uses
the standalone pieChart/barChart tool. Verified live: dashboard turn now calls
[query_data, generate_a2ui] only, single + after-flights.
- aimock beautiful-chat flights fixture: model the fixed-schema `search_flights`
path (returns the A2UI FlightCard envelope directly) instead of the old
generate_a2ui -> render_a2ui chain, so the fixture matches the live behavior
and the e2e spec's stated intent (United $349 / Delta $289).
An agent that streams text, calls a tool, then streams more text leaves Slack's
native "is thinking…" indicator running indefinitely after the answer has
already been posted.
`postedReply` latches on the first posted reply and makes `clearStatus` a
one-shot. But `onToolCallStartEvent` and `onToolCallEndEvent` both write the
status again *after* that latch closes, and from then on nothing clears it:
`onFirstReply` early-returns, and the backstops in `finalizeTurnStream` and
`finish` are skipped precisely because a reply was posted. Slack eventually
expires the stale status, which is why it presents as a slow hang rather than
a bug.
The latch really tracks "the status is already cleared for what is on screen",
not "a reply has been posted", so `setStatus` now resets it whenever a
non-empty status is written. Both existing backstops then fire exactly when
they should, and the normal streamed-text path still skips the redundant clear.
Reproduces on any agent that narrates before a tool call — emitted stream is
text → TOOL_CALL_* → text — and is independent of `showToolStatus`: with it off
both tool events set the generic status, with it on START sets
"is using `tool`…". Either way the write lands after the latch.
Covered by a regression test driving that exact sequence, verified failing
without the change ("expected 'is thinking…' to be ''").
The MCP Apps cell intermittently rendered an empty iframe (an empty box or a
thin band) on a live endpoint while passing under aimock.
Root cause is the model, not the renderer. Excalidraw's `create_view` declares
`elements` as `type: "string"` holding a JSON array, so the model must emit
double-encoded, hand-escaped JSON. gpt-4o-mini frequently appends a stray `}`
just past the closing `]`. The MCP server then rejects the call with
"Invalid JSON in elements: Unexpected non-whitespace character after JSON",
returns isError, and there is no diagram to draw, so the iframe paints empty.
aimock never catches this because it replays a recorded, valid payload.
Measured against the real Excalidraw MCP server using this agent's exact
system prompt and the real tool schema:
gpt-4o-mini create_view OK 3, isError 5
gpt-4.1 create_view OK 8, isError 0
And on JSON validity alone (n=12 unless noted):
gpt-4o-mini 8 invalid
gpt-4o-mini + hardened prompt 7 invalid (prompting does not fix it)
gpt-4.1-mini 3 invalid, and it bloats output
gpt-4.1 1 invalid of 24, output stays compact
gpt-4.1 is already used by five other agents in this file, so this keeps the
integration consistent. The cell's aimock fixture does not key on the model, so
d6 replay is unaffected.
On a live endpoint the tool-rendering flight card rendered every row blank
("United ? -> ? --") while the model's narration below it carried the real
times and prices. The result was delivered in full; the card just never
matched it.
search_flights emitted Mastra-flavored keys (flightNumber / departureTime /
arrivalTime / price) but FlightListCard - in both this integration and gold
langgraph-python, byte-identical - reads { airline, flight, depart, arrive,
price_usd }, which is exactly what gold's tool_rendering_agent.py returns.
Only `airline` overlapped, so the rest fell back to placeholders.
Return the gold result shape directly. The legacy caller-supplied `flights`
passthrough is untouched, and the only consumers of this tool are the three
tool-rendering-style agents, all of which drive gold-shaped cards.
Verified on a live real-LLM endpoint (no aimock): reproduced the blank rows
before the change, then confirmed all three rows render
"United UA231 08:15 -> 16:45 $348" (plus Delta and JetBlue) after it, across
tool-rendering, tool-rendering-custom-catchall and
tool-rendering-reasoning-chain, and across a two-turn weather + flights
conversation.
The existing e2e only asserted origin/destination and a row count, so blank
rows passed. It now asserts the result's depart/arrive/price and rejects the
"? -> ?" placeholder.
Fixes PNI-121
The "Search Flights (A2UI Fixed Schema)" pill narrated flight results as plain
text instead of rendering FlightCards. Root cause: mastra's beautiful-chat
reused the shared `searchFlightsTool` (returns plain `{ flights }`, which the
tool-rendering cells render via their own frontend FlightListCard), so nothing
produced an A2UI surface and the model just described the data.
langgraph-python's beautiful_chat.py wires a DEDICATED fixed-schema
`search_flights` whose tool RESULT is a complete `a2ui_operations` envelope (a
flat Row of literal FlightCards on `app-dashboard-catalog`, surface
`flight-search-results`). Mirror it:
- Add `searchFlightsA2uiTool` returning that envelope (buildFlightCardComponents
mirrors `_build_flight_components` — inline values, no structural-template
children).
- Add a dedicated `beautifulChatAgent` (query_data, todos, generate_a2ui, the
fixed search_flights, + the flight/dashboard steering prompt) and point the
beautiful-chat route at it, so the fixed-schema flights and steering don't
leak into the shared weatherAgent / tool-rendering cells.
Verified live (real LLM, dedicated agent): the flights pill paints two
FlightCard surfaces (catalogId app-dashboard-catalog, surface
flight-search-results), and the Sales Dashboard dynamic surface still renders in
full (metrics + pie + bar) via the grounded generate_a2ui.
## Problem (PNI-119)
`open-gen-ui-advanced` (and its minimal sibling) did **not** render
working generative UI on a **live** endpoint, though both passed under
aimock.
Both cells reused the shared `weatherAgent` (`instructions: "You are a
helpful assistant."`, gpt-4o). On a live LLM that emitted **static HTML
with no JS wiring** — the calculator's buttons rendered but did nothing
(staging repro: `7 + 8 =` left the display on `0`; captured SSE
`generateSandboxedUi` args contained only
`initialHeight/placeholderMessages/css/html`, no
`<script>`/`jsFunctions`/`jsExpressions`).
aimock hid the divergence: its fixtures match on
`userMessage`/`hasToolResult` and replay a fully-wired UI regardless of
the agent.
## Root cause
Two factors, one dominant:
1. **No dedicated agent.** Gold `langgraph-python` uses purpose-built
`open_gen_ui_agent.py` / `open_gen_ui_advanced_agent.py` whose system
prompt mandates a single interactive `generateSandboxedUi` call (iframe
restrictions, wire host functions via `Websandbox.connection.remote.*`,
visible result element). Mastra reused a generic agent.
2. **Context is a read-channel on Mastra.** The provider delivers the
design-skill + sandbox-function descriptors via `copilotkit.addContext`
→ `RunAgentInput.context`. The `@ag-ui/mastra` bridge maps that to
`requestContext.get("ag-ui").context` (a tool read-channel) and does
**not** inject it into the LLM prompt (unlike langgraph's
`CopilotKitMiddleware`). So even the tool description alone was not
enough for gpt-4o.
## Fix (surgical, showcase-only)
- `src/mastra/agents/index.ts` — add dedicated `openGenUiAgent` /
`openGenUiAdvancedAgent` porting gold's system prompts, with **dynamic
instructions** that read `requestContext.get("ag-ui").context` and fold
the design-skill + sandbox descriptors into the prompt (surfacing the
read-channel the Mastra-idiomatic way, matching what gold's agent sees).
- `src/mastra/index.ts` — register both.
- `src/app/api/copilotkit-ogui/route.ts` — point `getLocalAgent` at the
new agents.
No published-package or bridge change.
## Verification (live)
Local `next dev` against **real OpenAI** (aimock bypassed). All three
advanced pills render interactive, host-wired UI; host round-trips
confirmed in console:
- **Calculator** → `evaluateExpression 7*8 = 56`
- **Ping the host** → `notifyHost: Ping!` (+ "Host notified
successfully." in-iframe)
- **Inline expression evaluator** → `evaluateExpression 5 + 3 * 2 = 11`
**aimock unaffected** — matcher keys
(`userMessage`/`hasToolResult`/`context`) don't touch the prompt or
agent, so D6 replay still passes. Changed files typecheck clean.
## Follow-up (out of scope)
Live surfaced a benign `@copilotkit/runtime` OGUI-middleware warning: an
empty/null `jsFunctions` yields a JSON-patch `add /jsFunctions` with no
`value` (`OPERATION_VALUE_REQUIRED`, patch dropped). Tracked separately
(needs a runtime release).
## What does this PR do?
Reworks the reskinnable demo's layout from four flush, edge-to-edge
columns into a
shadcn-inset-style frame: the assistant column and the skin's app each
float as a
rounded card, separated by a resizable gutter.
**Layout**
- Two panels with simple bounds — assistant `min 250px / default 600px /
max 50%`,
app takes the remainder. Capping the assistant as a *share* means the
app needs no
floor of its own and the mobile breakpoint stays a genuine 768px.
- Drag to resize, swap which side the assistant docks on, or hide it
entirely. Side
and open state persist, shell-global, so switching skins never
rearranges the
workspace.
- Card radius is a fixed 12px in px, deliberately *not* `--radius` — the
frame is
shell chrome and must read identically in every skin. Card colours stay
themed, so
a reskin still restyles it.
**Chat**
- `CopilotSidebar` → inline `CopilotChat`, which is what lets the
cluster live in a
panel. That deletes a whole geometry contract (`--nw-chat-width`,
`--nw-rail-offset`, `data-nw-chat-open`) plus ~110 lines of CSS that
existed only
to keep a floating selector off a fixed panel.
- The thread rail is a fixed-width element, not a nested panel — it sits
beside the
conversation and hides via a container query when the card is dragged
narrow.
**Skin selector**
- Now a dropdown, so its footprint stays flat as skins are added (the
pill row took
three rows at four skins). Shell controls — switcher, swap sides, hide —
are
grouped in the card; the chat header keeps only conversation actions.
**Type**
- Base scale set to 15px at the root, so it reaches all four skins with
no per-skin
edits.
## Notes for reviewers
- `react-resizable-panels` is pinned to **4.x**, whose API is renamed
from the 2.x/3.x
used by six sibling examples
(`Group`/`orientation`/`Separator`/`useDefaultLayout`).
Only 4.x accepts pixel size constraints. See the header comment in
`src/components/ui/resizable.tsx` before touching it.
- The chat CSS re-key landed as a deliberate bracket — duplicate onto
`.nw-chat`,
make the structural change, delete the originals — because its failure
mode is
silent: a lint-clean, test-clean build with subtly wrong chat
typography. A guard
test asserts first parity, then absence.
- Adds `e2e/inset-layout.spec.ts` (9 tests). Weighted toward what
actually broke
during development — every bug here was a sizing behaviour invisible to
jsdom.
- The reskin skill taught two now-retired patterns (publishing
`--nw-nav-inset-*`,
rooting layouts at `h-screen`); both are corrected, so authoring a new
skin no
longer produces dead code and a broken root height.
**Known limitation:** the sub-768px overlay branch is not verified.
Mobile was
already broken before this work and is explicitly out of scope; the e2e
suite runs at
1280px.
## Checklist
- [x] I have read the Contribution Guide
- [x] If the PR changes or adds functionality, I have updated the
relevant documentation
- [x] "Allow edits by maintainers" is checked
When the LLM emits jsFunctions/css as null (frequent on open-gen-ui-advanced),
setParam coerces the value to undefined and emitParamDelta produced a JSON
Patch `{op:"add", path}` with no `value` property. fast-json-patch rejects
that client-side with OPERATION_VALUE_REQUIRED and drops the whole activity
patch (console warning, benign but noisy and fragile).
Guard emitParamDelta to skip emitting when the value is undefined. Empty
arrays and completion markers still emit. Adds a regression test feeding a
null jsFunctions and asserting no value-less patch is produced.
The reskin skill was the urgent part: it is the sanctioned path for adding a skin,
and it still taught two patterns the frame retired. It told authors to publish
--nw-nav-inset-left/right from a useEffect so the floating selector could dodge their
nav -- both the variables and that selector are gone -- and to root the layout at
h-screen overflow-hidden, which overflows the app card by the frame's padding.
Following it produced a skin with dead code and a broken root height. The nav-inset
step is removed with a short "Retired" note so it is not reintroduced by copying an
older skin, and the root guidance now explains that the CARD is the bound.
Adds a "The inset frame" section to CLAUDE.md covering what a reader needs before
touching src/shell/layout: the two-panel model and its bounds, why the thread rail is
a fixed element rather than a nested panel, the v4 API rename and the
id-becomes-data-testid behaviour, where the shell controls live, the fixed 12px card
radius, and the h-full requirement for skin layouts.
Also corrects the composition chain, the shell's ownership list, and the
four-skins-not-two count across CLAUDE.md, README.md and DESIGN.md, plus the skill's
verification step, which told authors to look for their skin in a bottom-left
floating pill.
Nine new tests over what only a real browser can see: panel order per docked side,
resize bounds in both directions, persistence across a reload and across a skin
switch, the rail collapse/reopen cycle, the hide/launcher cycle, that the selector
renders options only while open, that every skin's chrome fits its card without
scrolling the document, and that the chat header shows the active skin's assistant
name.
Weighted toward what actually broke. Every bug this layout shipped was a sizing or
resize behaviour invisible to jsdom, which has no layout engine: a rail collapsed to
zero width, a rail that could not be reopened, an assistant column that could be
widened but never narrowed, and a header showing the framework default. All four
passed a green unit suite. Each assertion was mutation-checked rather than assumed --
widening the assistant cap to 95% fails only the resize test, hardcoding the header
title fails only the header test.
Navigation goes through a gotoSkin helper that waits for the frame; the shell is
client-rendered, so reading geometry straight after goto returns an empty panel set.
Updates smoke.spec.ts for the dropdown: switching opens the menu first, and the
"no other skin's chrome leaked" guards get STRICTER -- from exactly-one selector pill
to zero occurrences, since a closed menu renders no other brand.
Retires the CopilotSidebar launcher from the fixme'd specs. a2ui-canvas clicked
copilot-chat-toggle, which no longer exists -- inert only because the spec never
runs, so it would have failed the moment anyone re-enabled it. memory-learning keeps
its guarded open-chat block deliberately: the count check makes it a no-op today, and
that spec needs the docker memory stack so it cannot be run here to verify a
behavioural edit.
Composes ShellFrame in the per-skin layout, passing the skin's Layout as its app
slot and the shared ChatPanel as its chat slot, replacing the sibling
Layout + ChatPanel + FloatingSelector trio.
Each skin's chrome drops its viewport height for h-full: it now fills the app card,
which the frame has already inset by its own padding, so a viewport-height root
overflowed the card by exactly that much. Logistics keeps overflow-hidden -- its nav
stays pinned only while the container is bounded, which is now the card. Banking
keeps its second, unrelated canvas-clearing effect and therefore its useEffect
import.
The four --nw-nav-inset-* publishing effects go with the floating selector they fed.
Each skin published the width of its own edge-nav so a floating pill could compute a
safe band and dodge it; the selector now occupies a slot in the assistant column and
overlaps nothing, so the whole mechanism is unnecessary and its component is deleted.
This is one commit because the halves are not independently correct: without the skin
edits the cards overflow, and deleting the selector before its import is removed
would not build.
Replaces CopilotSidebar with an inline CopilotChat so the cluster can live in a
panel. CopilotSidebar was a fixed <aside> that pushed document.body's margin and
faked two columns by being handed the width of both, insetting its own contents past
the rail while the rail painted into the freed strip as a separately-fixed sibling.
That deletes an entire geometry contract: --nw-chat-width, --nw-rail-offset and
data-nw-chat-open existed only to describe a fixed panel's footprint to the rest of
the page. Also gone: the force-open-on-mount ref dance, the header slot cast
(CopilotChat has no header slot -- that is a modal concern, so the header is now an
ordinary sibling), and the rail's own fixed positioning and translate animation.
The thread rail is a FIXED-WIDTH element, not a panel. v4's collapse API fought that
three ways: a collapsed 0 written to storage was restored forever, expand() restores
the "most recent size" which after collapsing to 0 is 0, and resize() is ignored
while collapsed. Conditionally rendering it makes isInboxOpen the single source of
truth with no imperative sync.
The header reads the SKIN for its title, not the chat configuration. As
CopilotSidebar's header slot it rendered inside the chat's own provider so our labels
reached it; as a sibling it reads the wrapper's, whose default "CopilotKit Chat" is
non-null and won the ?? chain -- every skin's header showed that instead of its
assistant name. The header is skin chrome now, and holds only conversation actions.
src/shell/layout/ owns the app's outer geometry: a padded region holding the
assistant column (selector card above chat card) and the skin's app card, separated
by a resizable gutter.
The model is deliberately just "one bounded panel, one that takes the remainder" --
assistant min 250px / default 600px / max 50%, app gets what is left. An earlier
version nested the thread rail as a resizable panel INSIDE the assistant column,
which made its floor a compound of rail + conversation and forced a derived
breakpoint, a switching collapsed floor and an app floor to compensate. Capping the
assistant as a SHARE rather than a pixel count is what removes the need for an app
floor and lets the mobile breakpoint stay a genuine 768px instead of being derived
from panel arithmetic.
- selector-card: the skin switcher as a dropdown, so its footprint stays flat as
skins are added, plus the shell controls -- swap sides and hide. All three are
shell concerns, which is why they are here and not in the chat header.
useSkinThemeReconcile stays on the card root: it reads the computed
--nw-dark-capable from inside the skin's theme root, and losing it would let a
light-only skin render dark chat chrome.
- layout-preferences: side and open state, shell-global so switching skins never
rearranges the workspace. Read through useSyncExternalStore because this repo
treats react-hooks/set-state-in-effect as an error, with write-through
persistence so only a deliberate choice is stored. The hook returns inert
defaults outside its provider rather than throwing.
- use-is-desktop: matchMedia through the same external-store pattern.
- panel-sizes: the three numbers, in one place.
Three changes to the shell stylesheet.
Re-scopes chat typography from the SDK's [data-copilot-sidebar] attribute to a
.nw-chat wrapper. That attribute came from CopilotSidebar, which the inset frame
replaces with an inline chat, so ~23 rule blocks styling assistant markdown would
have silently stopped matching -- a lint-clean, test-clean build with subtly wrong
chat text. The migration ran as a bracket: duplicate onto .nw-chat, land the
structural change, then delete the originals, with a guard test asserting first the
parity and then the absence. Typography is provably unchanged -- a probe mounting
markdown-shaped content with the library's prose class reported 140 identical
computed values across 14 selectors before and after.
Adds the frame's own rules: .nw-panel-card (a FIXED 12px radius, deliberately not
reading --radius, because the frame is shell chrome and must read identically in
every skin) and the chat cluster's fixed-width rail with the container query that
hides it when the card is dragged narrow.
Sets the root font size to 15px. The root, not body: every rem in the shell and all
four skins resolves against it, so this is the one lever that rescales type
everywhere without per-skin edits. Spacing scales with it by design; anything that
must not move is written in px. The chat prose moves 0.9375rem -> 1rem, since that
value existed to hit 15px under a 16px root and would otherwise sit below the new
default.
Deletes the ~120-line floating-selector dock and the unused .brand-text-gradient.
Pins ^4.12.2 and wraps its Group/Panel/Separator API behind two styled handles: an
8px gutter for the gap between cards and a 1px hairline. 4.x rather than the 2.x/3.x
used by six sibling examples because only 4.x accepts PIXEL size constraints, and
the layout's bounds are pixels.
Three v4 behaviours the consumers depend on, each with a test:
- The API is RENAMED from 2.x/3.x: PanelGroup->Group, direction->orientation,
PanelResizeHandle->Separator, autoSaveId->useDefaultLayout. Most material online,
including shadcn's Resizable block, targets the old names and will not compile.
- data-testid is DERIVED from a Panel's id and overwrites any passed in, so a
panel's id is its query handle. Getting this wrong makes every
[data-testid$='-panel'] query silently match nothing.
- Separator refuses flex-grow/flex-shrink overrides, so shrink-0 would be dead.
Adds a jsdom ResizeObserver stub, without which every test rendering a Group throws.
Note for anyone adding a dependency here: declare it in package.json and run
'pnpm install --lockfile-only' from the REPO ROOT. This example pins pnpm 10.10.0
while the root pins 10.33.4, so 'pnpm add' from this directory makes corepack switch
versions and the older pnpm re-resolves the entire workspace lockfile.
Root package.json pins patchedDependencies (eventsource@3.0.7 ->
patches/eventsource@3.0.7.patch, added in #6334). The harness Dockerfile
selectively copies root manifests but omitted patches/, so the frozen
install hashes the patch file, ENOENTs, and exits 254. Broke the
showcase-harness build-check for any PR rebased onto post-#6334 main.
Copy patches/ before the install. Verified: image builds clean locally.
The dynamic `generate_a2ui` tool grounded its inner `render_a2ui` subagent from
the tool's `contextEntries` arg, which the outer model always sends empty
(captured live: `{"messages":[…],"contextEntries":[]}`). On a live LLM the inner
render then ran with an EMPTY system prompt: ungrounded, it emitted
invalid/misnamed components (or none), so the surface never resolved against the
catalog. Result: the Beautiful Chat A2UI dynamic surface (Sales Dashboard,
flights) rendered no UI, with a render error that varied run to run. aimock hid
it: the recorded fixture returns a valid envelope regardless of the empty
context.
Read the catalog schema + A2UI generation guidelines the `@ag-ui/mastra` bridge
already forwards onto Mastra's request context
(`requestContext.get("ag-ui").context`) and ground the render there instead of
trusting the model-supplied arg. Mirrors `readAgUiContext` in `@ag-ui/mastra`'s
`getA2UITools`. Preserves per-demo catalogId (the grounded model emits it, and
`buildA2uiOperationsFromToolCall` uses `args.catalogId`), so the same shared
weatherAgent serves both beautiful-chat and declarative-gen-ui. Falls back to
the arg when no request context is present.
multimodal was wrongly quarantined based on a local-only failure: the
sample.png/sample.pdf demo assets are Git LFS pointers, and without
git-lfs on PATH the attachment send fails before the run starts
(runStartCount=0). langgraph-python multimodal fails locally for the
exact same reason, yet declares the feature supported.
Verified the MAF agent actually works: with the real assets in place
the D6 cell is green (2 turns, assertions passed). Both production
deploys serve the real 10KB PNG (LFS pulled), so multimodal is green
in prod for MAF just like langgraph. not_supported_features now matches
langgraph exactly: [gen-ui-interrupt, interrupt-headless].
- Remove multimodal from features (already in not_supported_features);
the duplicate made generate-registry reject the manifest, blocking D6
from starting.
- Pin agent-framework-{ag-ui,openai,core} to exact published latest
(1.0.1 / 1.12.0 / 1.13.0) so validate-pins classifies them as exact
framework deps; drop unused langchain-openai/langchain-core.
- Ratchet validate-pins baseline down 31 -> 27 (the 4 non-exact framework
fails are now exact); ms-agent-python's only remaining FAIL is the
shared-frontend openai ^5.9.0, identical to every other integration.
Tyler Slaton