- Move `package` from properties string to top-level `{ name }` object per
Ben's confirmed IngestPayload schema (telemetry-sink-ingest/index.ts:127-134)
- Add typed per-event helpers trackBannerViewed/trackBannerClicked/trackThreadsTabClicked
to enforce property shapes at call sites and prevent PII leakage under wrong keys
- Add trackBannerClickedOnce guard in index.ts (per-mount Set keyed by
banner_id + cta) to prevent banner_clicked inflation on repeated clicks
- Fix handleTelemetryOptOutToggle: replace ?? true fallback with
instanceof HTMLInputElement guard (wrong fallback was a privacy bug)
- Add threadsTabClicked re-selection guard (skip if already on threads tab)
- Replace getTelemetryDistinctIdForUrl() call on mount with ensureTelemetryDistinctId()
- Add inMemoryFallbackId in persistence.ts for funnel coherence when
localStorage is unavailable (same UUID returned per page load)
- Add _resetTelemetryPersistenceForTesting() for test isolation
- Remove @copilotkit/shared dep from telemetry-disclosure.ts (inline
env-var check; keeps module self-contained and testable in isolation)
- Add clearMocks: true to web-inspector vitest config (fixes spy call
history accumulating across tests)
- Expand telemetry.test.ts to 22 tests covering wire body shape, opt-out,
5 error-resilience paths, typed helpers, distinct ID lifecycle (SSR +
localStorage-throws + funnel coherence), maybeShowDisclosure, and
getTelemetryDistinctIdForUrl
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three V1 funnel events from the inspector — oss.inspector.banner_viewed,
oss.inspector.banner_clicked, oss.inspector.threads_tab_clicked — plus a
privacy panel for opt-out, a first-run console disclosure on inspector
mount and runtime startup, and inspector content added to the canonical
/telemetry docs page on main.
Inspector POSTs directly from the browser to telemetry.copilotkit.ai/ingest
(per ticket: URL is intentionally clearly named for transparency in DevTools).
Inline fetch POST in lib/telemetry.ts — no @copilotkit/shared dep on the
inspector, no dependency on any non-main branch.
Wire body shape (conservative; needs Ben confirmation):
POST https://telemetry.copilotkit.ai/ingest
{ event, properties: { ...caller, distinct_id, package }, ts }
If the lambda expects a richer envelope, update the single JSON.stringify
in lib/telemetry.ts.
Privacy invariants:
- Opt-out toggle short-circuits before any network call (verified by test).
- Properties are scoped to event metadata only — no message content, agent
state, prompts, completions, banner markdown. Negative test pins the wire.
- Anonymous distinct ID (UUID v4 in localStorage) set on inspector load and
propagated onto banner CTA links as ?posthog_distinct_id=<uuid> so the
destination site can posthog.alias() and close the
banner_viewed → banner_clicked → signup_attributed funnel. URL param
suppressed when opted out.
- Console disclosure on first inspector mount and runtime startup. Both
link to https://docs.copilotkit.ai/telemetry.
Plan gaps addressed:
- CTA name on banner_clicked: cta:'body'|'dismiss' (click location) plus
optional cta_label read defensively. Sam: confirm dismiss treatment.
- De-anon opt-out folded into the single toggle. Docs say so explicitly.
- banner_viewed dedup: per-instance Set<string> keyed by timestamp.
- EPIC consent / pixel review: out of scope for this PR; flagged at merge.
Deferred for V1.1:
- Wire body shape (Ben).
- Event-type allowlist for oss.inspector.* (Ben — oss-path-to-production).
- posthog_distinct_id URL-param key name (Ben/Tyler/website team).
Refs https://linear.app/copilotkit/issue/OSS-96
ContextVar-based ambient state + httpx event hook. Incoming
x-aimock-* headers from AG-UI requests are forwarded to outgoing
LLM API calls. Keys normalized to lowercase. Warning emitted when
install_httpx_hook receives an unrecognized client type.
Probes now send X-AIMock-Strict: true on all requests via
extraHTTPHeaders, triggering strict fixture matching in aimock.
Live demo traffic (no header) continues to proxy to real LLMs.
Mirror of #4430 by @MalaikaAbb.
Adds an agentId note for non-default agents, replaces `agent.id` with
`agent.agentId`, corrects the `@copilotkit/shared/v2` import to
`@copilotkit/shared`, and adds the missing `[agent]` dependency to a
useEffect.
## Summary
Multi-pill sessions in the showcase demos were silently breaking because
aimock fixtures gated their tool-emit vs. follow-up branches on global
thread state (`hasToolResult`, `turnIndex`). Once the user clicked a
tool-using pill, the thread already had tool/assistant messages so the
next pill's first-leg fixture got skipped and the matcher fell through
to the wrong branch.
Concrete failures Alem reported:
- **tool-rendering**: Find flights then Roll a d20 rendered only 3 d20
cards instead of 5; Chain tools emitted zero cards and went straight to
"Done — Tokyo is sunny…".
- **frontend-tools-async**: First pill rendered the NOTES DB card;
second and third pills only rendered narration with no card.
- **hitl-in-app**: After approving the first pill the second pill
skipped its approval dialog entirely.
## Fix
- Re-key every follow-up fixture on the prior step's `toolCallId` (the
matcher checks `messages[last].tool_call_id`).
- Drop `hasToolResult: false` from the tool-emitting fixtures so they
fire on any iteration whose last message is the user prompt.
- Reorder so the `toolCallId`-specific fixtures come first under
first-match-wins.
- `userMessage` matchers are unchanged — fixtures still match the same
prompt text they always did.
D20 becomes a linear `toolCallId` graph (`seq_001 → _002 → … → _005`);
Chain tools gets disambiguator fixtures for each of its three parallel
`tool_call_id`s; Weather / AAPL / query_notes / HITL approve+reject
branches all gate on the specific `request_user_approval` /
`get_weather` / `query_notes` / `get_stock_price` id that landed last.
## Regression coverage
Adds a multi-pill Playwright test to each affected spec — clicks every
pill sequentially in one thread and asserts the full card count for
each:
- `tool-rendering-default-catchall`: Find flights → 5 d20 rolls (with 20
last)
- `tool-rendering-custom-catchall`: 1 flights + 5 d20 + 3 chain = 9
wildcard cards
- `frontend-tools-async`: 3 NOTES DB cards with the right keyword per
pill
- `hitl-in-app`: refund approve then escalate, each with its own
approval dialog
## Test plan
- [x] Verified each fix end-to-end via Playwright MCP against local dev
server
- [ ] CI runs the new regression specs against the canonical Docker
stack
- [ ] Existing aimock fixture validation suites still pass (verified
locally: 21/21)
## Summary
In LangGraph API, persistence is handled automatically by the platform,
so providing a custom checkpointer throws an Application Failure error.
Therefore, it had to be removed
## What does this PR do?
- Removed ```checkpointer=MemorySaver()``` from python code snippet in
section (Initialize Your Agent Project, and also removed its relevant
import
## Checklist
- [x] I have read the [Contribution
Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md)
- [x] If the PR changes or adds functionality, I have updated the
relevant documentation
- [x] "Allow edits by maintainers" is checked (lets us help iterate on
your PR directly — faster turnaround for everyone)
The tool-rendering, frontend-tools-async, and hitl-in-app fixtures all gated
their first-leg (tool-emitting) vs. follow-up (narration) responses on
`hasToolResult: false/true` and/or `turnIndex`. Those constraints count the
*entire* thread, so once a user clicked a tool-using pill the thread already
contained tool messages and assistant turns and subsequent pill clicks fell
through to the wrong branch — d20 dropped from 5 rolls to 3, Chain tools
emitted no cards, query_notes returned narration without the Notes DB card,
and the second HITL pill never raised an approval dialog.
Re-key every follow-up fixture on the prior step's `toolCallId` (the matcher
checks `messages[last].tool_call_id`), drop the global `hasToolResult` gates
from the tool-emitting fixtures, and reorder so the toolCallId-specific
fixtures come first under first-match-wins. The d20 chain becomes a linear
toolCallId graph (`call_tr_d20_seq_001` → `_002` → … → `_005`), Chain tools
gets disambiguators for each of its three parallel tool_call_ids, and
Weather/AAPL/query_notes/HITL approve+reject branches all gate on the
specific request_user_approval / get_weather / query_notes / get_stock_price
id that landed last. userMessage matchers are unchanged.
Adds Playwright multi-pill regression tests to the four affected demos that
click every pill sequentially in one thread and assert the full card counts:
- tool-rendering-default-catchall: Find flights → 5 d20 rolls (with 20 last)
- tool-rendering-custom-catchall: 1 flights + 5 d20 + 3 chain = 9 cards
- frontend-tools-async: 3 NOTES DB cards with the right keyword per pill
- hitl-in-app: refund approve then escalate, each with its own dialog
## Summary
- **Fix**: the three interactive Open-Generative-UI (Advanced) pills
(Calculator, Ping the host, Inline expression evaluator) shipped HTML +
CSS only in `showcase/aimock/d5-all.json`. The buttons rendered but
every click was a silent no-op because the fixtures never included
`jsFunctions` to wire up the host-bridge calls
(`Websandbox.connection.remote.evaluateExpression` / `notifyHost`).
Restores the missing JS.
- **Two-layer regression guard** so this can't land silently again:
1. `showcase/scripts/__tests__/open-gen-ui-advanced-fixtures.test.ts`
(vitest, runs on every PR in `showcase_validate`) — asserts each
interactive fixture entry references the right host bridge in its
`jsFunctions`.
2.
`showcase/integrations/langgraph-python/tests/e2e/open-gen-ui-advanced.spec.ts`
(playwright, runs in `test_e2e-showcase-on-demand`) — drives the
in-iframe controls, captures the host handler's `console.log`, and
asserts the iframe output reflects the host response.
The existing e2e smoke tests are kept (still useful) but switched from
pill-click to textarea fill+Enter, following the precedent set by commit
15db0bbf3 (`gen-ui-headless-complete`). Combined with
`[data-testid="copilot-chat-textarea"]`, an explicit click, and
`waitForLoadState("networkidle")` in `beforeEach`, the suite is no
longer flaky against React hydration.
## Why fixture-only fix is sufficient
The agent system prompt (`open_gen_ui_advanced_agent.py:71`) tells a
real LLM to emit `jsFunctions`. A real model does. The fixtures simulate
that contract — they just hadn't been updated to. No agent code or
framework code needed to change; the runtime already injects
`jsFunctions` into the Websandbox correctly
(`packages/react-core/src/v2/components/OpenGenerativeUIRenderer.tsx`
Effect 2).
## Test plan
- [x] `pnpm --filter @copilotkit/showcase-scripts test
open-gen-ui-advanced-fixtures` — 3/3 passing
- [x] Full `open-gen-ui-advanced.spec.ts` playwright suite,
aimock-driven, locally — 7/7 passing (4 smoke + 3 round-trip) in ~28s
- [x] Manual: Calculator builds + evaluates, Ping the host updates
output with `host replied at ...`, Inline evaluator returns `= <value>`
- [ ] `/test-aimock langgraph-python` once this PR is open (validates
against the CI aimock stack)
The previous fixture regression (HTML+CSS only, no jsFunctions) slipped
past CI because the e2e suite only asserted "iframe mounts with non-empty
srcdoc" — which passes whether or not the iframe is interactive. Adds
two layers of guard so the same regression cannot land silently:
1. showcase/scripts/__tests__/open-gen-ui-advanced-fixtures.test.ts
(vitest, runs in showcase_validate on every PR): asserts each of the
three interactive fixture entries in d5-all.json ships jsFunctions
referencing the matching host bridge (evaluateExpression / notifyHost).
Catches "someone removed jsFunctions" at PR-time with no
infrastructure dependencies.
2. showcase/integrations/langgraph-python/tests/e2e/open-gen-ui-advanced.spec.ts
(playwright, runs in test_e2e-showcase-on-demand): adds three
round-trip tests that drive the in-iframe controls and assert the
host-side handler ran by capturing its console.log + verifying the
iframe output element reflects the host response. Catches "the
renderer fails to inject jsFunctions into the sandbox" too.
The e2e tests also switch the existing smoke tests off pill-click and
onto a textarea-driven fill+Enter path, following the same precedent as
commit 15db0bbf3 (gen-ui-headless-complete) — chip mounts diverge
between EmptyState and SuggestionBar surfaces, and Playwright's pill
click races React hydration. Using [data-testid="copilot-chat-textarea"]
with an explicit click + waitForLoadState("networkidle") makes the
suite reliable end-to-end (7/7 passing locally against the aimock-driven
stack).
The three interactive Open-Generative-UI (Advanced) pills — Calculator,
Ping the host, and Inline expression evaluator — shipped HTML + CSS
only in d5-all.json. The agent system prompt instructs the LLM to also
emit jsFunctions, but the fixtures didn't, so every in-iframe button
became a silent no-op: the iframe rendered, clicks dispatched no events,
and the sandbox-function bridges to evaluateExpression / notifyHost
were never exercised.
Adds the missing jsFunctions to all three fixtures, using single-quoted
JS so no escaping is needed inside the JSON-stringified tool arguments.
Each handler respects the sandbox="allow-scripts"-only iframe constraints
(no <form>, plain addEventListener) and calls back into the host via
Websandbox.connection.remote.<name>, with the expected return-shape
contract (res.ok + res.value / res.receivedAt).
## Summary
- The chat-slots cell is wired to the neutral `sample_agent` graph
(plain `ChatOpenAI`, no Responses API, no reasoning config), so it never
emits AG-UI `REASONING_MESSAGE_*` events. The "Show reasoning"
suggestion pill therefore had no way to light up the wrapped
`messageView.reasoningMessage` slot.
- Its prompt — `"Think out loud step by step about whether 17 is prime,
then answer."` — also wasn't a substring of any `userMessage` in
`showcase/aimock/d5-all.json`, so aimock-backed runs returned `404 "No
fixture matched"` rather than the intended reasoning fixture.
- `qa/chat-slots.md` and `tests/e2e/chat-slots.spec.ts` already only
assert the two remaining pills ("Write a sonnet" / "Tell me a joke"), so
dropping the third pill aligns the demo with what the QA + e2e already
expect.
The `CustomReasoningMessage` slot wrapping is intentionally left in
place — the chat-slots page is a "Slot Atlas" where every overrideable
slot is wrapped, even if dormant for this cell. Live reasoning rendering
already has its own dedicated demos at `/demos/reasoning-default` and
`/demos/reasoning-custom`, which use the `reasoning_agent` graph
(Responses API + `reasoning={"effort":"medium","summary":"detailed"}`).
## Test plan
- [ ] `/demos/chat-slots` renders exactly two suggestion pills: "Write a
sonnet" and "Tell me a joke"
- [ ] Both pills produce an assistant reply wrapped in the
`CustomAssistantMessage` slot marker
- [ ] No console errors or missing-fixture 404s appear during the flow
- [ ] `/demos/reasoning-default` and `/demos/reasoning-custom` still
light up the reasoning slot (unchanged here, but worth a sanity check)
The chat-slots cell is wired to the neutral sample_agent graph (plain
ChatOpenAI, no Responses API, no reasoning config), so it never emits
AG-UI REASONING_MESSAGE_* events. The pill could never light up the
wrapped messageView.reasoningMessage slot, and its prompt didn't match
any fixture in showcase/aimock/d5-all.json — aimock-backed runs hit
"No fixture matched". Drop the pill (the QA doc and the e2e spec
already only expect "Write a sonnet" and "Tell me a joke") and leave a
note pointing reasoning demos at /demos/reasoning-default and
/demos/reasoning-custom where the dedicated reasoning_agent graph lives.
## Summary
- Widen the `declarative-gen-ui` demo so generated A2UI cards (KPI
dashboards, charts, status reports) actually have room to breathe. Outer
page wrapper goes `max-w-4xl` → `max-w-6xl`, and a scoped attribute
selector lifts the chat's internal `cpk:max-w-3xl` to 64rem **for this
demo only** — no impact on other demos.
- Fix the `InfoRow` trailing-separator artifact. Each row now draws its
own `border-b` with `last:border-b-0 last:pb-0 first:pt-0`, so the final
row in a Card (most visible on the **Status Report** suggestion pill) no
longer leaves a dangling line. Works regardless of whether the agent
wraps rows in a Column or drops them directly into the Card child slot.
- `Card` gains `w-full overflow-hidden` so it stretches into the wider
column; `InfoRow` value gets `text-right tabular-nums` for cleaner
alignment in stacks.
Scoped to the north-star (`showcase/integrations/langgraph-python`) only
— will propagate to the other integrations via `pnpm parity:sync` in a
follow-up.
## Why
Clicking the suggestion pills (KPI dashboard, pie chart, bar chart,
status report) rendered cards inside a 768px column even though the page
wrapped at 1024px. The bottleneck was the chat's internal
`cpk:max-w-3xl`. Status Report also looked busy because every `InfoRow`
rendered a full-width divider after itself, including the last row.
## Test plan
- [ ] `pnpm --filter @copilotkit/showcase-langgraph-python dev`, open
`/demos/declarative-gen-ui`
- [ ] Click each suggestion pill (KPI dashboard, pie chart, bar chart,
status report); confirm cards have noticeably more horizontal room
- [ ] Confirm the last row in **Status Report** has no trailing divider
- [ ] Confirm other demos in the same integration are visually unchanged
(the override is scoped to `.declarative-gen-ui-wide`)
## Summary
- Delete the legacy `threadId`/`setThreadId` docs under
`docs/content/docs/integrations/langgraph/advanced/persistence/` and
`docs/content/docs/integrations/crewai-flows/persistence/` — superseded
by the `useThreads`-based `/langgraph/threads` and
`/crewai-flows/threads` pages that share
`docs/snippets/shared/threads/threads.mdx`.
- Drop `"persistence"` from the CrewAI Flows sidebar
(`docs/content/docs/integrations/crewai-flows/meta.json`); LangGraph's
`advanced/meta.json` auto-lists, so no sidebar edit needed there.
- Redirect every deleted path — plus the legacy
`/langgraph/persistence/*` aliases and the `/coagents/persistence`
middleware entry — to the corresponding `/{framework}/threads` page so
existing inbound links don't 404. The pre-existing `/coagents/:path*`
and `/crewai-crews/:path*` wildcards chain through to these.
No content in `docs/content`, `docs/snippets`, or `docs/src` still
references the removed paths.
## Test plan
- [ ] `pnpm --filter docs build` succeeds
- [ ] `/langgraph/threads` and `/crewai-flows/threads` still render
- [ ] Hitting `/langgraph/advanced/persistence/loading-message-history`,
`/langgraph/persistence/message-persistence`,
`/crewai-flows/persistence/loading-agent-state`, and
`/coagents/persistence` lands on the appropriate `/{framework}/threads`
page
- [ ] CrewAI Flows sidebar no longer shows "Persistence"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The `[&_div[style*='flex-direction:_row']]:gap-4` arbitrary variant
(quotes inside doubly-nested brackets) is the most exotic Tailwind
syntax in this PR and lines up exactly with when the Vercel
form-filling deploy started failing. Tailwind v4's content scanner is
likely choking on the apostrophes in the nested attribute selector.
The Metric `flex-1 min-w-[120px]` and the chart `flex-1 min-w-0`
already give us even distribution inside the basic catalog's gap-less
Row; the auto-injected nested gap was nice-to-have, not load-bearing.
The /langgraph/advanced/persistence/* and /crewai-flows/persistence/* pages
documented the older threadId/setThreadId mechanism. They're replaced by the
useThreads-based /langgraph/threads and /crewai-flows/threads pages backed by
the shared threads snippet.
Adds redirects from the deleted paths (and their legacy /langgraph/persistence
aliases) to the new framework-scoped /threads pages, and drops "persistence"
from the crewai-flows sidebar.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the inline <style> override that widened the chat's `cpk:max-w-3xl`
column and the outer max-w-6xl bump. The chat keeps its normal width;
the real bug was that the basic catalog's Row/Column primitives are
bare `display: flex` divs with no gap and no min-width control on
children, so when the agent dropped multiple Metrics or charts into a
Row they collapsed to content width and looked glued together.
Four targeted fixes inside our renderers:
- Metric gains `flex-1 min-w-[120px]` so a row of KPI tiles distributes
the available width evenly inside the gap-less basic Row, instead of
shrinking to content.
- PieChart and BarChart switch from a hardcoded max-w to `flex-1
min-w-0` so two charts side-by-side each take half the card column
with Recharts' ResponsiveContainer doing the rest, instead of one
chart insisting on 640px and overflowing.
- Card's CardContent picks up a Tailwind arbitrary variant
`[&_div[style*='flex-direction:_row']]:gap-4` (plus the column
equivalent) that injects a gap into any nested basic Row/Column the
agent drops in. Underscores in the arbitrary value compile to literal
spaces, matching React's serialized inline `flex-direction: row`.
- Card itself drops the old `min-w-[260px]` floor in favour of
`min-w-0` so it cooperates if the agent ever stacks Cards horizontally
inside a Row.
The chat shell caps its scroll column at cpk:max-w-3xl (~768px), which
left A2UI-generated cards (KPI dashboards, charts, status reports)
feeling pinched on the declarative-gen-ui demo. Locally widen that
wrapper to 64rem via a scoped attribute selector on the demo and bump
the outer page wrapper from max-w-4xl to max-w-6xl so the card column
actually has room to grow.
While here, fix the InfoRow trailing-separator artifact: each row now
draws its own border-bottom with last:border-b-0 so the final row in a
Card (e.g. the Status Report demo) no longer leaves a dangling line,
regardless of whether the agent wraps the rows in a Column or drops
them directly into the Card child slot. Right-align the value with
tabular-nums for cleaner stacks. Card itself gains w-full
overflow-hidden so it stretches into the now-wider column instead of
sitting at its min-width.
## Summary
- The mcp-apps demo's "Draw a flowchart" suggestion pill silently broke:
clicking it produced a generic "Here is my plan..." response with no MCP
tool call, so the sandboxed iframe never mounted.
- Root cause: the verbatim pill prompt `"Use Excalidraw to draw a simple
flowchart with three steps."` had no matching `create_view` fixture in
`d5-all.json`. aimock's first-match-wins substring matcher fell through
to `feature-parity.json`'s `{userMessage: "steps"}` content-only
catch-all (matches "with three **steps**.").
- Fix: add a fixture pair keyed on the distinctive substring `"draw a
simple flowchart"` to `d5-all.json` and its mirror in
`harness/fixtures/d5/mcp-apps.json`. Turn 1 emits `create_view` with a
Start → Process → End flowchart; turn 2 (post tool result) emits the
narration.
- Regression guard: a fast vitest under `showcase/scripts/__tests__/`
loads fixtures in the same order as `docker-compose.local.yml` and uses
aimock's `matchFixture` to assert each mcp-apps pill routes to its
`create_view` fixture on turn 1 and its narration fixture on turn 2. The
test fails loudly with a diagnostic naming the absorbing fixture if the
bug returns (verified by reverting the fix locally).
## Test plan
- [x] `pnpm --filter @copilotkit/showcase-scripts test
__tests__/mcp-apps-suggestion-routing.test.ts` — 3 tests, ~260 ms, all
green.
- [x] `pnpm --filter @copilotkit/showcase-scripts test
__tests__/aimock-fixtures.test.ts` — fixture files still validate
cleanly with the new entries.
- [x] Reverted just the two new fixtures, re-ran the regression test,
confirmed it fails with a diagnostic that explicitly names the wrong
fixture (`feature-parity.json`'s `{userMessage: "steps"}` content
blurb).
- [x] Local docker stack (`bin/showcase up langgraph-python`) + browser:
clicked "Draw a flowchart" at http://localhost:3100/demos/mcp-apps —
agent emits `create_view`, MCP middleware fetches the Excalidraw UI
resource, sandboxed iframe mounts inline, narration arrives.
- [x] aimock direct POST to `/v1/chat/completions` confirms both legs:
- Turn 1 (pill + `create_view` in tools[]) → `tool_calls: [{ name:
"create_view", id: "call_d5_mcp_apps_create_view_flowchart_001", ...
}]`.
- Turn 2 (with tool result appended) → `content: "Drew a three-step
flowchart (Start → Process → End)..."`.
The "Draw a flowchart" suggestion pill in the mcp-apps demo sent
"Use Excalidraw to draw a simple flowchart with three steps." which
had no matching create_view fixture in d5-all.json. aimock walked
through to feature-parity.json's `{userMessage: "steps"}` substring
fixture and returned a generic "Here is my plan..." content blurb
with no MCP tool call, so the runtime never invoked create_view, the
MCP middleware never fetched the UI resource, and the sandboxed
iframe never mounted.
Add a fixture pair in d5-all.json (and its harness mirror) keyed on
"draw a simple flowchart": turn 1 emits create_view with a three-
step Start -> Process -> End flowchart, turn 2 emits the narration
after the tool result. The distinctive substring beats the generic
feature-parity catch-alls under first-match-wins.
Adds a regression test that loads the same fixture files in the same
order as docker-compose.local.yml and asserts via aimock's matchFixture
that each mcp-apps pill routes to its create_view fixture on turn 1 and
its narration fixture on turn 2.
## Summary
Re-lands the multimodal-attachments fix from #4584 (May 1, never merged)
onto current `main`, ported to the post-refactor file layout where
`page.tsx` was split into `legacy-converter-shim.tsx`,
`multimodal-chat.tsx`, and `file-to-data-attachment.ts`.
Auto-send was the visible regression: clicking **Try with sample image /
Try with sample PDF** only queued the attachment chip instead of sending
the canned prompt. This PR restores the full end-to-end behavior plus
five regression tests so it can't silently break again.
## What was broken and what changed
1. **Random uploads crashed with `Failed to fetch`.** aimock returned
HTTP 404 on no-match, the LangGraph SDK surfaced `NotFoundError`, the
AG-UI stream surfaced a `RUN_ERROR`, the demo crashed. → Added
`--proxy-only` + `--provider-openai https://api.openai.com` to the local
aimock command so unmatched user prompts fall through to real OpenAI
(mirrors Railway).
2. **Bundled-sample fixtures keyed on user-visible canned prompts.**
Auto-prompts are now natural and specific ("can you tell me what is in
this demo image/pdf I just attached") so they render cleanly as the user
message bubble AND can't collide with arbitrary user prompts — random
uploads phrase questions differently and fall through to the proxy.
3. **Sample buttons now auto-send via `useAgent`.** The previous
DataTransfer path queued the attachment via the chat's hidden file input
but required clicking send while the attachment was still uploading —
`CopilotChat.onSubmitInput` rejects submits during upload AND clears the
input regardless, so the canned prompt was eaten. Rewrite calls
`agent.addMessage(...)` + `copilotkit.runAgent({ agent })` directly with
the base64'd content part.
4. **PDF flattened text bled into the rendered user message.**
`_PdfFlattenMiddleware` ran in `before_model` and persisted the rewrite
to agent state. Switched to `wrap_model_call` so the PDF→text rewrite is
scoped to the model request only.
5. **Attachments doubled (and PDFs rendered as broken `<img>`).** The
`@ag-ui/langgraph` round-trip mis-tags PDFs as `image` and re-injects
the user's original modern part, doubling chips. Added
`dedupeUserMessageMedia` subscriber on `onMessagesSnapshotEvent` +
`onRunFinalized` to dedupe by `source.value` and re-key type from
mimeType. Also flipped `onRunInitialized` from REPLACE to APPEND so the
modern part stays for the UI alongside a legacy `binary` sibling for the
converter.
6. **Regression suite (`tests/e2e/multimodal.spec.ts`).** Five focused
tests, all pass against live local stack (15.4s):
- page loads with all expected affordances
- sample image: auto-sends, EXACTLY ONE `<img>`, assistant references
the logo
- sample PDF: auto-sends, EXACTLY ONE `DocumentAttachment` chip ("PDF"
label), NO `<img>`, no `[Attached document]` text bleed
- image then PDF in the same session: each message keeps its own single
chip
- PDF then image in the same session: symmetric
## Test plan
- [x] `showcase up langgraph-python` — both sample buttons auto-send;
image renders as `<img>`, PDF renders as PDF chip; random paperclip
uploads go through proxy
- [x] `BASE_URL=http://localhost:3100 CI=1 npx playwright test
multimodal.spec.ts` — **5 / 5 passing**
- [ ] Post-merge: e2e-deep cycle for langgraph-python multimodal cell
stays green
## Closes
Closes#4584.
The langgraph-python multimodal-attachments demo had a stack of bugs
that compounded each other. Fixing them required touching the local
docker-compose, the aimock fixtures, the LangChain middleware, the
client-side AG-UI shim, and the sample-attachment buttons. This
commit lands the full set together because they only make sense as
a unit — verified end-to-end against `showcase up langgraph-python`
in a headed browser. New e2e suite pins each regression.
Supersedes #4584 (the original fix from May 1 that never landed —
this is a fresh port onto the post-refactor file layout where
page.tsx is split into legacy-converter-shim.tsx, multimodal-chat.tsx,
file-to-data-attachment.ts).
What was broken and what changed:
1. Random uploads crashed with `Failed to fetch`. aimock returned
HTTP 404 on no-match, the LangGraph SDK surfaced `NotFoundError`,
the AG-UI stream surfaced a `RUN_ERROR`, the demo crashed.
Added `--proxy-only` + `--provider-openai https://api.openai.com`
to the local aimock command so unmatched user prompts fall through
to real OpenAI (mirrors the Railway aimock setup).
2. Bundled-sample fixtures keyed on user-visible canned prompts.
The auto-prompts are deliberately long, specific, and natural-
reading ("can you tell me what is in this demo image/pdf I just
attached") so they (a) render cleanly as the user message bubble,
and (b) can't collide with arbitrary user prompts — random
uploads phrase questions differently and fall through to the
proxy.
3. Sample buttons now auto-send via `useAgent`. The previous
DataTransfer-based path queued the attachment via the chat's
hidden file input, then required clicking send while the
attachment was still uploading — `CopilotChat.onSubmitInput`
rejects submits during upload AND clears the input regardless,
so the canned prompt was eaten. Rewrite to call
`agent.addMessage(...)` + `copilotkit.runAgent({ agent })`
directly with the base64'd content part, sidestepping the
upload race entirely.
4. PDF flattened text bled into the rendered user message.
`_PdfFlattenMiddleware` ran in `before_model` and returned
`{"messages": rewritten}`, which persisted to agent state. The
chat UI then rendered the `[Attached document]\n<pdf body>` text
part inline with the user prompt. Switched to `wrap_model_call`
so the PDF→text rewrite is scoped to the outgoing model request
only and never pollutes state.
5. Attachments doubled (and PDFs rendered as broken `<img>`). The
`@ag-ui/langgraph` round-trip translates outgoing `binary` parts
to LangChain `image_url` and incoming `image_url` back to `image`
AG-UI parts — regardless of mimeType, so PDFs came back as
`type: "image"` with `mimeType: "application/pdf"` and were
forced into `ImageAttachment`, where the load failed and the
chat showed two "Failed to load image" boxes. Plus the user's
original modern part survived alongside the round-tripped one,
doubling visible chips.
Added a `dedupeUserMessageMedia` subscriber on both
`onMessagesSnapshotEvent` and `onRunFinalized` to:
- dedupe media parts by `source.value` so the local + round-
tripped copy collapse to one chip
- re-key part `type` from `mimeType` so PDFs route to
`DocumentAttachment` (icon + filename) and images to
`ImageAttachment`.
Also flipped the `onRunInitialized` shim from REPLACE to APPEND
— keep the modern part for the UI AND emit a legacy `binary`
sibling for the converter.
6. Regression suite (`tests/e2e/multimodal.spec.ts`). Replaces the
pre-rewrite suite with five focused tests:
- page loads with all expected affordances
- sample image: auto-sends, EXACTLY ONE `<img>`, assistant
references the logo
- sample PDF: auto-sends, EXACTLY ONE `DocumentAttachment` chip
("PDF" label), NO `<img>`, no `[Attached document]` text bleed
- image then PDF in the same session: each message keeps its own
single chip, no cross-contamination
- PDF then image in the same session: symmetric
All 5 pass against the live local stack (15.4s).
## Summary
- The voice route's OpenAI client used to fall through to
`OPENAI_BASE_URL`, which `docker-compose.local.yml` sets to
`http://aimock:4010/v1`. Aimock has a catchall `endpoint:
"transcription"` fixture that returns `"What is the weather in Tokyo?"`
for every audio file, so the mic button always produced that phrase
regardless of what the user actually said into the microphone. Pin
`baseURL` to real OpenAI Whisper (overridable via
`OPENAI_TRANSCRIPTION_BASE_URL`). The sample-audio button stays as
synchronous text injection — that's the documented design and what the
e2e + d5 probe rely on.
- Tidy the sample-audio button label so the affordance matches what it
does: "Try a sample question" → "Try a sample audio".
- Realign `tests/e2e/voice.spec.ts` with the shipped component (the spec
was asserting on a `voice-sample-audio` container testid and a `Sample:
"..."` caption that never existed in the current
`sample-audio-button.tsx`), and add cold-start timeout headroom for the
mic button render (waits on `/info` round trip) and the agent-flow test.
- Add `"env": ".env"` to `langgraph.json` so `langgraph_cli dev` picks
up `OPENAI_API_KEY` for local dev. Docker/Railway paths inject env vars
directly via `env_file`, so this is a no-op there.
Scope was deliberately limited to `langgraph-python` (north-star) per
the user's request — the other 17 integrations still route mic
transcription through aimock. Demo parity sync can propagate this when
desired.
## Test plan
- [x] `pnpm exec playwright test voice.spec.ts` — all 3 tests pass
(`page loads…`, `sample audio button injects the canned phrase…`,
`sending the transcribed text produces a weather tool render`)
- [x] Manual mic verification on `http://localhost:3000/demos/voice`
with a real `OPENAI_API_KEY` — speaking "hello" now returns "Hello."
instead of the Tokyo fixture
- [ ] Visual check the new "Try a sample audio" label in the deployed
preview
- [ ] Confirm Railway/Docker deploys still transcribe through real
Whisper (this path was unaffected — `OPENAI_TRANSCRIPTION_BASE_URL`
stays unset, defaulting to api.openai.com)
The voice route's OpenAI client previously fell through to OPENAI_BASE_URL,
which docker-compose.local.yml sets to http://aimock:4010/v1. Aimock has a
catchall transcription fixture that returns "What is the weather in Tokyo?"
for every audio file, so the mic button always produced that phrase no
matter what the user actually said.
Pin baseURL to real OpenAI (overridable via OPENAI_TRANSCRIPTION_BASE_URL).
The sample-audio button stays as synchronous text injection — that's the
documented design, and what the e2e + d5 probe rely on.
Also:
- Tidy the sample button label ("Try a sample question" -> "Try a sample
audio") so the affordance matches what it does.
- Realign tests/e2e/voice.spec.ts with the shipped component (the
voice-sample-audio container testid and Sample: "..." caption it asserted
on never existed on HEAD) and add cold-start timeout headroom for the
mic-button render and the agent-flow test.
- Add "env": ".env" to langgraph.json so langgraph_cli dev picks up
OPENAI_API_KEY locally. Docker/Railway paths inject env vars directly so
this is a no-op there.
## Summary
Follow-up to #4723. The post-merge cycle showed
`gen-ui-headless-complete` still red with `page.click: Timeout` waiting
for the aria-label selector I added in #4723.
Root cause: the demo has TWO chip surfaces with diverging aria-label
shapes:
- **EmptyState** (first paint, before any messages): `aria-label="Try
suggestion: <message>"`, visible text = the message string.
- **SuggestionBar** (after first message lands):
`aria-label="Suggestion: <title>"`, visible text = the short title.
Which surface is mounted depends on `messages.length` —
timing-dependent. Both my prior selectors (`button >> text="Weather"`
and `button[aria-label="Suggestion: Weather"]`) matched only one surface
and timed out on the other.
Fix: drop the chip-click preFill entirely. The runner's normal
fill+press hits the textarea with the verbatim prompt; chip clicks and
textarea-Enter dispatch the same `runAgent`, so the fixture matcher
catches either route — and the textarea is always present.
Also includes a small refresh of `e2e-deep.test.ts` for a stale
assertion (Phase-2A REGISTRY_TO_D5 split renamed `tool-rendering` →
`tool-rendering-default-catchall`).
## Test plan
- [ ] Land + watch the next e2e-deep cycle
- [ ] Confirm `gen-ui-headless-complete` flips green
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
The LGP **tool-rendering-reasoning-chain** D5 probe hung on Turn 2 (Find
flights from SFO to JFK) after Turn 1 (weather Tokyo) succeeded.
**Root cause:** the SFO/JFK first-leg fixture matched on `hasToolResult:
false`, but Turn 1's tool result remains in the conversation by Turn 2.
So aimock's matcher skipped the first-leg fixture and fell through to
the second-leg (`hasToolResult: true`), returning narration text without
the `search_flights` tool call. The probe then timed out waiting for the
FlightListCard testid that never mounted.
**Fix:** switch the SFO/JFK second-leg fixture to `toolCallId` matching
(last message must be a tool result with that specific `call_id`) and
drop `hasToolResult` from the first-leg. The second-leg fixture must
come BEFORE the first-leg in file order — the matcher is
first-match-wins, but with `toolCallId` it cannot accidentally swallow
first-leg requests (whose last message is the user prompt, not a tool
result).
Also adds a `content` field to the Tokyo first-leg fixtures
(`get_weather` underscore + `get-weather` Mastra-hyphen variants) so
they route through aimock's `buildContentWithToolCallsStreamEvents`
instead of `buildToolCallStreamEvents`. The latter codepath silently
drops the `reasoning` field (4-arg signature, no reasoning param), which
is why the reasoning-block testid never mounted on Turn 1 either before
this change.
Tokyo (Turn 1) keeps the simpler `hasToolResult` pattern because it has
no prior turns; only the SFO/JFK fixture pair needs the `toolCallId`
shape to survive multi-turn use.
## Test plan
- [x] `pnpm --filter @copilotkit/showcase-scripts test aimock-fixtures`
— 18 passed (load-time schema validation)
- [x] `pnpm --filter @copilotkit/showcase-scripts test` — 574/574 passed
- [x] `pnpm --filter @copilotkit/showcase-harness test` — 1588/1588
passed
- [ ] Railway deploys aimock with new fixtures, then
`tool-rendering-reasoning-chain` cell on the dashboard flips D4 → D5 for
langgraph-python
## Summary
Phase 4 validation (run today against `docs.showcase.copilotkit.ai`)
surfaced four cutover-blocking issues. This PR fixes all of them in four
focused commits.
## Commits
1. **chore(showcase): close last 2 yellow Missing snippet boxes for
cutover** — adds in-place region markers to
`langgraph-python::frontend-tools` and a sibling
`slot-overrides.snippet.tsx` teaching file for
`langgraph-python::chat-slots`. Both pages now render zero `Missing
snippet` warnings.
2. **fix(shell-docs): correct feature-viewer slug + demo-id translation
for code tab** — adds `getFeatureViewerSlug()` and
`getFeatureViewerDemoId()` helpers in `registry.ts`, with explicit
override maps for the 8 framework-name mismatches (built-in-agent,
google-adk, claude-sdk-{python,typescript}, ms-agent-{python,dotnet},
crewai-crews, llamaindex) and 5 demo-id mismatches (gen_ui_tool_based,
gen_ui_agent, shared_state_streaming, shared_state_read_write,
hitl_in_chat). When the framework or demo has no feature-viewer
equivalent, the helper returns null and the Code tab is hidden on that
page (graceful degradation; Demo tab still renders).
3. **fix(shell-docs): redirect catalog hygiene (self-loops +
framework-scoped gap)** — removes 31 self-redirect entries from
`seo-redirects.ts` (where source === destination caused infinite 301
loops on canonical URLs like /frontend-tools, /faq, /human-in-the-loop).
Reorders middleware logic so the redirect catalog is consulted before
the framework-scoped short-circuit fires, fixing 97 framework-scoped
slug-rename URLs that were soft-404ing instead of redirecting. Adds
defense-in-depth skip-when-equal guard in middleware.
4. **fix(shell-docs): resolve 53 sitemap 500s before cutover** — three
independent root causes in MDX rendering:
- 4 missing \`<Component />\` registrations in \`mdx-registry.tsx\`
(CopilotCloudConfigureCopilotKit,
SelfHostingCopilotRuntimeConfigureCopilotKit, CloudCopilotKit, Content)
— affected ~37 pages.
- 3 langgraph tutorial pages had markdown lists immediately preceding
JSX closing tags, causing remark to bail. Fix = blank line between
bullet and close tag (~9 pages).
- 5 MDX files used escaped JSX comments that Acorn cannot parse —
replaced with proper unescaped form (~7 pages).
## Verification
- Production build clean: 27 static pages generated, 0 errors
- Local probes: all 53 sitemap-500 URLs return 200; all 31 former
self-loop URLs return 200; all 85 framework-scoped non-loop catalog
entries 301 to expected destinations
- Code tab: 47/60 (framework × demo) URLs resolve to real code panels
post-fix; remaining 13 are feature-viewer per-framework deploy-coverage
gaps in the \`ag-ui-protocol/ag-ui\` repo, not this one
- Both langgraph-python pages render 0 \`Missing snippet\` boxes after
re-bundling demo-content
## Test plan
- [ ] CI passes
- [ ] Sitemap probe: every URL in \`/sitemap.xml\` returns 200
- [ ] Redirect probe: spot-check 5 framework-scoped slug renames (e.g.
\`/agno/frontend-actions\`, \`/pydantic-ai/use-agent-hook\`)
- [ ] Self-loop probe: \`/frontend-tools\` returns 200, not infinite 301
- [ ] Code tab: visit \`/built-in-agent/frontend-tools\` (tab should be
hidden), \`/langgraph-python/agentic-chat\` (tab should render code),
\`/llamaindex/agentic-chat\` (tab should render after llama-index slug
rename)
- [ ] Yellow boxes: visit \`/langgraph-python/frontend-tools\` and
\`/langgraph-python/custom-look-and-feel/slots\` — zero Missing snippet
warnings
## Summary
Pins the three classes of regression that #4733 + #4734 just fixed, so a
future revert/refactor trips a test instead of breaking the deployed
showcase silently.
### Python unit tests — `src/agents/test_a2ui_internal_tools.py`
7 tests, ~2s, no LLM calls (secondary LLM is stubbed):
1. **Tool name doesn't collide with the A2UI middleware's intercept
list.** Parametrised over `beautiful_chat._design_a2ui_surface`,
`a2ui_dynamic._design_a2ui_surface`, and `a2ui_fixed.display_flight`.
Asserts none of them appear in `a2uiToolNames` (default
`{"render_a2ui"}`). Catches an accidental rename revert that would
re-enable the streaming-args bypass.
2. **`generate_a2ui` force-pins canonical `catalog_id`.** Stubs
`ChatOpenAI` with a fake response carrying a hallucinated `catalogId`,
calls the tool, parses the returned ops, asserts the surface op carries
the module's `CUSTOM_CATALOG_ID`. Covers both `beautiful_chat` and
`a2ui_dynamic`.
3. **Malformed root drops cleanly.** Stubs the secondary LLM emitting a
root component without a `component` field. Asserts `generate_a2ui`
returns a clean error string instead of feeding the renderer a partial
tree (which surfaces as the deploy-only "Cannot create component root
without a type" infinite-loop).
### E2E regression assertions — added to existing specs
- `tests/e2e/beautiful-chat.spec.ts` (Sales Dashboard test): no `Catalog
not found` / `Cannot create component … without a type` banner; at most
2 `recharts-responsive-container` (one pie + one bar = single
dashboard).
- `tests/e2e/declarative-gen-ui.spec.ts` (BarChart test): no banners; at
most 1 `recharts-responsive-container`.
- `tests/e2e/a2ui-fixed-schema.spec.ts` (Find SFO → JFK test): exactly
one `Flight Details` and one `Book flight` button (was 6+ on deploy
pre-fix from the `display_flight` loop); no banners.
## Test plan
- [x] `pytest src/agents/test_a2ui_internal_tools.py` — 7/7 green
locally.
- [ ] Existing Playwright suites still pass (assertions are additive —
only fail if a regression actually surfaces).
## What does this PR do?
Marks the legacy `byoc-hashbrown` and `byoc-json-render` registry
entries as `"deprecated": true` so the dashboard's "Show deprecated"
toggle hides them by default.
[#4756](https://github.com/CopilotKit/CopilotKit/pull/4756) renamed
langgraph-python's `byoc-*` slugs to `declarative-*` but kept the legacy
IDs in the registry so the other 17 integrations (whose manifests still
declare `byoc-*`) wouldn't drop out of the catalog. That left the BYOC
(Bring Your Own Components) section showing **both** rows for each of
the two demos — legacy IDs red-X'd for langgraph-python (no manifest
declaration there anymore) and the renamed IDs green.
This PR keeps both ID forms live (the other 17 integrations still need
the legacy entries) but hides the legacy duplicates behind the standard
`deprecated` flag — same pattern already used for
`agentic-chat-reasoning`, `hitl`, `hitl-in-chat-booking`,
`reasoning-default-render`. The dashboard's "Show deprecated" toggle in
`feature-grid.tsx` decides visibility from there.
## Why
The dashboard view post-#4756 was confusing — two identical-looking
"Declarative UI: Hashbrown" / "Declarative UI: json-render" rows where
one is green and one is red. Deprecating the legacy entries gives
readers one canonical row per demo by default while preserving the data
path the other 17 integrations rely on.
## Test plan
- [x] `showcase/scripts` vitest suite green (574 tests)
- [x] `npx tsx generate-registry.ts` regenerates `catalog.json` for all
four shells without errors
- [x] No catalog count changes (deprecated cells stay in the catalog,
only dashboard visibility changes)
## Related
- [#4756](https://github.com/CopilotKit/CopilotKit/pull/4756) — the slug
rename that introduced the duplicate rows
🤖 Generated with [Claude Code](https://claude.com/claude-code)
PR #4756 renamed langgraph-python's `byoc-hashbrown` / `byoc-json-render`
slugs to `declarative-*` but kept the legacy IDs in the registry so the
other 17 integrations (which still declare `byoc-*` in their manifests)
don't drop out of the catalog. The dashboard then showed both rows side
by side under "BYOC (Bring Your Own Components)": the legacy IDs red
for langgraph-python (no manifest declaration) and the renamed IDs
green.
Mark the two legacy IDs `"deprecated": true` so the dashboard's "Show
deprecated" toggle hides them by default. The canonical declarative-*
rows stay visible. Data stays intact for the other 17 integrations
until they migrate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## What does this PR do?
Multi-phase correctness + cleanliness pass on
`showcase/integrations/langgraph-python` plus the shared
harness/aimock/dashboard surfaces it depends on. The langgraph-python
integration is the reference implementation other showcase integrations
are graded against, so the diff cleans up real demo-time bugs, brings
the e2e + QA contract into line with each demo's actual source, and
updates the D5 harness so probe coverage works against the renamed
slugs.
The 18 commits split cleanly into two parts:
**Phase commits (5 — original audit work, already on the branch when
cr-loop started):**
- **`7a66040c2` Phase A correctness fixes** — `interrupt-headless` now
reads `payload.slots` from the backend instead of rendering stale
hardcoded dates; `package.json#dev` runs `langgraph_cli dev --port 8123`
(matching the API route and the production entrypoint) instead of
pointing at a 3-line `agent_server.py` stub on port 8000; the legacy
stub is deleted; the entrypoint smoke-check for `src/agents/tools.py` (a
file that never existed) is removed; the two reasoning-only agents get a
one-line comment explaining why `CopilotKitMiddleware` is intentionally
absent.
- **`ee429d1b2` Phase B demo cleanup** — `shared-state-read` flattens to
a single source of truth (drops dual-state sync that had the read-only
demo locally mutating recipe state); extracts `useAutoScroll` and
`useTypingIndicator` out of `headless-complete`; extracts
`parseJsonResult` into a shared helper; extracts `NOTES_DB` out of
`frontend-tools-async/page.tsx`; introduces a `makeSlotOverride<T>`
helper that replaces 11 `as unknown as` casts in `chat-slots/page.tsx`.
- **`fc8c0f5e4` Phase C Python agent cleanup** — extracts
`_a2ui_utils.py` (`sanitize_a2ui_components` + `has_root_component`)
consumed by `beautiful_chat.py` and `a2ui_dynamic.py`; collapses three
try/except blocks in `multimodal_agent._extract_pdf_text`; replaces
hardcoded PDT offset with `zoneinfo.ZoneInfo(\"America/Los_Angeles\")`
so DST works.
- **`8b6eca973` Phase D rename** — `byoc-{hashbrown,json-render}`
slugs/folders/routes/specs/QAs become `declarative-*` (the internal
Python graph IDs intentionally stay legacy for now); Tool Rendering trio
renames to Default/Catch-all/Custom progression; showcase index now
sorts demos within each tag by `manifest.features` order.
- **`99ad42e9b` Phase E test contract alignment** — agentic-chat spec
rewrites to match the now-simplified vanilla-chat demo; deletes specs
for demos that no longer exist (`shared-state-write.spec.ts`,
`renderer-selector.spec.ts`); aligns QAs with the demos they describe.
**Cr-loop commits (13 — convergence-driven fixes from a 7-round
multi-agent review):**
The cr-loop ran 7 confirmation rounds with 7 specialized review agents
per round (code-reviewer, silent-failure-hunter, pr-test-analyzer,
comment-analyzer, type-design-analyzer, infra code-reviewer, Python
code-reviewer). Findings each round were partitioned into bucket (a)
subject-scope + load-bearing / (b) subject-scope + trivial / (c)
out-of-subject pre-existing / (d) out-of-this-PR subject. The 13 fix
commits address every confirmed bucket (a):
- **Round 1 (`493a6002e`)** — `agentic-chat.spec.ts` was asserting
`[data-role=\"assistant\"]` which the v2 react-core bundle never emits
(it ships `data-testid=\"copilot-assistant-message\"`); fixture comment
that said \"Empty content is load-bearing\" sat next to a non-empty
content field; harness `d5-feature-mapping.ts` / `live-status.ts` /
`d5-byoc.ts` got `declarative-*` entries so D5 probes still target
langgraph-python after the slug rename; bucket-b polish.
- **Round 2 (`c61dc65f3`)** — `subagents._invoke_sub_agent` had a
regression where the docstring cleanup removed an `isinstance(text,
str)` guard, so `{\"type\":\"text\",\"text\":null}` payloads (a known
provider quirk) would crash `\"\".join(parts)` with `TypeError`;
restored the guard. `shared-state-read.spec.ts` was a complete
false-green (asserted on \"Sales Pipeline\" but the demo is now a Recipe
Editor); rewrote against the recipe-card testids. 11 sibling specs were
sweep-fixed for the same stale `data-role` selector. Restored one log
line at `multimodal_agent._extract_pdf_text` (the three-stage prints
were dropped in cleanup; one outer log keeps Railway triageable). Fixed
false `kind: \"testing\"` claim in `qa/reasoning-default.md`.
- **Round 3 (`328282297`)** — `shared-state-read/page.tsx` had `void
copilotkit.runAgent({agent}).catch(() => {})` which explicitly silenced
every error from \"Improve with AI\"; replaced the empty catch with
`console.error`. Demo's visible h1 still read \"BYOC: Hashbrown\" after
the slug rename; updated page heading, spec, and QA in lockstep.
Manifest description for `agentic-chat` claimed \"Natural conversation
with frontend tool execution\" but Phase E rewrote the demo to vanilla
chat with no tools. `reasoning-default.spec.ts` asserted on
`[data-testid=\"copilot-reasoning-message\"]` which
`CopilotChatReasoningMessage` never emits; switched to the text-based
\"Thinking…/Thought for…\" label the component actually renders.
- **Round 4 (`b905869e9` + `21fb20cf0` + `cff562c0d` + `ae24603bb`)** —
fixture-mirror fixes (the new reasoning-default fixture was added to the
bundled `d5-all.json` but not to the per-demo source-of-truth file);
added 3 declarative-* test cases to `d5-byoc.test.ts`; fixed
`qa/chat-slots.md` to match the slot-wrappers refactor; tightened
comment polish (typo, count drift, ported-from path).
- **Round 5 (`39fadec3f` + `64bee6453`)** — Sunday edge case in
`interrupt_agent._candidate_slots` where `tomorrow` and `next_monday`
collapsed to the same date; fixed both Python and TS sides. Five spec/QA
false-fails: `gen-ui-tool-based.spec.ts` asserted on UI text (\"Use the
sidebar...\", \"Chart Generator\") that doesn't exist;
`agent-config.spec.ts` asserted heading \"Agent Config Object\" but the
demo has \"Agent Config\"; `multimodal.spec.ts` asserted a non-existent
heading; `qa/chat-slots.md` said max-w-4xl but the demo is max-w-5xl;
`qa/shared-state-read.md` had a stale default-instruction string.
- **Round 6 (`70548fe2f`)** — `chat-slots.spec.ts` was asserting
`[data-testid=\"custom-assistant-message\"]` in 4 places (4 of 5 tests
would time out); the actual signal is
`data-slot-label=\"MessageView.AssistantMessage\"` from the SlotMarker
wrapper. `qa/gen-ui-tool-based.md` described a \"Haiku Generator\" demo
that doesn't exist anywhere; rewrote to match the chart-rendering demo.
- **Final cleanup (`2342fbb5a`)** — deleted 4 orphan QA files flagged
across multiple cr-loop rounds (`agentic-chat-reasoning.md`, `hitl.md`,
`hitl-in-chat-booking.md`, `shared-state-write.md`).
## Test posture
| Level | Result |
|---|---|
| **smoke** | green (health + agent endpoints) |
| **D4** (e2e-chat-tools / Tool Reasoning) | **green** — not regressed |
| **D5** (e2e-deep) | 38/40 green; the two reds
(`gen-ui-interrupt-sales-call` and `tool-rendering-reasoning-chain` turn
2) are pre-existing and explicitly out-of-scope per the original audit's
Phase F. **`mcp-apps` and `tool-rendering-reasoning-chain` turn 1 went
from red → green on this branch.** |
## Pre-existing reds (deliberately not addressed; track as follow-ups)
- **`gen-ui-interrupt-sales-call`** — agent reply lands but the
`interrupt(...)` payload never reaches the frontend so the picker never
enters its picked state. This is the **W8-6 LangGraph-Platform interrupt
deployment** ticket from the original audit's Phase F.
- **`tool-rendering-reasoning-chain` turn 2** — turn 1 was fixed in
`a6682ccd6` by adding non-empty `content` so aimock's Responses-API
dispatch routes through `buildContentWithToolCallsResponse` instead of
dropping reasoning. Turn 2 is a deeper aimock+deepagents+Responses-API
multi-turn state-management issue — production with real OpenAI works.
## Out-of-this-PR follow-up seeds
- Cross-codebase `byoc-* → declarative-*` rename across the other 17
integrations + shared registry (one of: complete the rename or roll it
back to keep the codebase coherent).
- e2e specs for `interrupt-headless`, `tool-rendering-reasoning-chain`,
`shared-state-read-write` (manifest demos with no Playwright spec).
- `gpt-4o → gpt-5.4` docstring sweep across `multimodal_agent.py` and
`mcp_apps_agent.py` (pre-existing comment drift).
## Checklist
- [x] I have read the Contribution Guide
- [x] If the PR changes or adds functionality, I have updated the
relevant documentation (e2e specs + QA markdowns)
- [x] \"Allow edits by maintainers\" is checked
D5 probes were red for two demos because the aimock fixtures didn't
exist or were dispatched through the wrong Responses-API path.
- `harness/fixtures/d5/mcp-apps.json` (new) — the MCP Apps probe sent
"Open Excalidraw and sketch a system diagram" with no matching
fixture, so aimock 404'd and the agent threw `NotFoundError`.
Added a two-leg fixture: leg 1 emits a `create_view` MCP tool call
with Excalidraw element JSON; leg 2 is the `hasToolResult: true`
content reply. Mirrored verbatim into `aimock/d5-all.json` so the
Docker-baked aimock has the same matches.
- `harness/fixtures/d5/tool-rendering-reasoning-chain.json` — added
non-empty `content` fields to the first-leg fixtures (Tokyo weather
+ SFO/JFK flights). aimock's Responses-API dispatch routes
tool-call-only responses (no `content`) through
`buildToolCallResponse`, which silently drops the `reasoning`
payload; with a non-empty `content`, dispatch shifts to
`buildContentWithToolCallsResponse`, which emits the
reasoning_summary events the v2 `<ReasoningBlock>` needs to mount.
Mirrored into `d5-all.json`.
- `harness/fixtures/d5/reasoning-display.json` — added a second
matcher for the `reasoning-default` e2e pill prompt ("sky appears
blue"), keyed alongside the existing D5 probe matcher ("show your
reasoning step by step"). Both responses include `reasoning` fields
so the built-in `CopilotChatReasoningMessage` "Thinking…/Thought
for…" header lands.
- `aimock/feature-parity.json` — added a fixture for the agentic-chat
multi-turn name-recall test ("What name did I just give") replying
"You said your name is Alice." The matcher is intentionally narrow
to avoid colliding with the generic showcase-assistant catch-all.
Net D5 effect: `mcp-apps` and `tool-rendering-reasoning-chain` turn 1
go from red → green. The remaining red on `tool-rendering-reasoning-
chain` turn 2 is the deeper aimock+deepagents+Responses-API multi-
turn state-management issue (production with real OpenAI works); out
of scope per the original audit's Phase F.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The specs and QA markdowns had drifted from the demos they describe.
This commit brings every test contract into line with the actual demo
source — eliminating false-greens, false-fails, and stale assertions.
False-fail spec assertions (would fail every run):
- `agentic-chat.spec.ts` — rewrote from the old `change_background` /
`weather-card` / `useAgentContext` flow that no longer exists. New
spec exercises the vanilla `<CopilotChat>` + three suggestion pills
contract the simplified demo actually exposes.
- `gen-ui-tool-based.spec.ts` — asserted on UI text ("Use the sidebar
to generate charts", "Chart Generator") that doesn't exist; switched
to suggestion-pill assertions and scoped the SVG check to inside the
assistant-message bubble (was matching CopilotChat's send-button
SVG).
- `agent-config.spec.ts` — asserted heading "Agent Config Object" but
the demo has "Agent Config".
- `multimodal.spec.ts` — asserted a non-existent "Multimodal
attachments" heading; switched to the `multimodal-demo-root` testid.
- `chat-slots.spec.ts` — asserted `[data-testid="custom-assistant-
message"]` and the bare text "slot" — neither exists. The actual
signal is `data-slot-label="MessageView.AssistantMessage"` from the
SlotMarker wrapper.
- `reasoning-default.spec.ts` — asserted `[data-testid="copilot-
reasoning-message"]` and `[data-message-role="reasoning"]`; neither
is emitted by `CopilotChatReasoningMessage`. Switched to the text-
based "Thinking…/Thought for…" header label.
False-green spec assertions (passed for the wrong reason):
- `shared-state-read.spec.ts` — was a complete false-green: asserted
on "Sales Pipeline", "Total Pipeline", "Active Deals" but the demo
has been a Recipe Editor for some time. Rewrote against the
recipe-card / ingredients-container / instructions-container testids.
- 11 specs (agent-config, beautiful-chat, frontend-tools-async,
gen-ui-tool-based, gen-ui-agent, gen-ui-interrupt, hitl-in-chat,
hitl-in-app, multimodal, readonly-state-agent-context, voice) used
`[data-role="assistant"]` to gate "agent responded" — but the v2
react-core bundle never emits that attribute (it ships
`data-testid="copilot-assistant-message"`). Mechanical sweep to the
correct testid.
- Deleted `shared-state-write.spec.ts` (route consolidated into
`shared-state-read-write` earlier on this branch — spec targeted a
removed demo) and `renderer-selector.spec.ts` (asserted on a radio-
pill UI that no longer exists; the four "Declarative UI" variants
are now separate manifest demos).
QA drift:
- `qa/gen-ui-tool-based.md` documented a "Haiku Generator" demo with
haiku-card / japanese-line / english-line / haiku-image testids — a
demo that doesn't exist anywhere on this branch. Rewrote to match
the chart-rendering demo's actual testids and pill prompts.
- `qa/chat-slots.md` referenced "Custom Slot" pill / "Welcome to the
Slots demo" heading / "This welcome card is rendered via the
welcomeScreen slot." body text — all of which the slot-wrappers
refactor on this branch removed. Updated to match the
`custom-welcome-message` sub-slot that's actually rendered. Also
fixed max-w-4xl → max-w-5xl to match the page.
- `qa/shared-state-read.md` said default instruction is "Preheat oven
to 350 F" but the source has "Preheat oven to 350°F (175°C)".
- `qa/agentic-chat.md` rewrote to match the simplified vanilla-chat
demo (the previous QA documented `change_background` / `WeatherCard`
flows that no longer exist).
- `qa/reasoning-default.md` cited `kind: "testing"` in feature-
registry.json for the `reasoning-default` entry; the registry entry
has no `kind` field. Rewrote without the false cross-file claim.
- Deleted 4 orphan QA files for demos that don't exist:
`agentic-chat-reasoning.md`, `hitl.md`, `hitl-in-chat-booking.md`,
`shared-state-write.md`.
- Renamed `qa/reasoning-default-render.md` → `qa/reasoning-default.md`
to match the manifest cell name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Martha Schumann