mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
tyler/workflow-observer-example
15047 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Tyler Slaton
|
50245e56de | test(react-core): correct feedback memoization coverage | ||
|
Tyler Slaton
|
40f4fdc25e |
feat(showcase): add travel workflow example
Add a minimal CopilotKit useAgent showcase backed by a Python LangGraph workflow. Stream attractions onto a Leaflet map with stable markers and Inspector support. |
||
|
Ben Taylor
|
f2c9a2ecfe |
docs(react-core): clarify system_prompt requirements for useFrontendTool (#5469)
## What does this PR do? Adds documentation explaining that `useFrontendTool` may require explicit `system_prompt` guidance in LangGraph agents to be reliably called. Includes a working code example showing both a well-described `useFrontendTool` registration and the matching system prompt instruction. Addresses the confusion reported in #4950 where users implement tools with only a `description` and find the agent doesn't call them. Changes: - `docs/integrations/langgraph/frontend-tools.mdx` — new section "Ensuring your agent reliably calls frontend tools" with a `Callout`, Python+TypeScript code examples, and a decision table (description vs. system_prompt) - `reference/hooks/useFrontendTool.mdx` — added "LangGraph agents: description vs. system_prompt" subsection linking to the full example - `docs/troubleshooting/common-issues.mdx` — expanded the "tool listed but agent never calls it" bullet to cross-link the new guide ## Related PRs and Issues - Closes #4950 ## Checklist - [x] I have read the Contribution Guide - [x] If the PR changes or adds functionality, I have updated the relevant documentation - [x] "Allow edits by maintainers" is checked |
||
|
Ben Taylor
|
6c955686eb |
feat(react-core): expose AG-UI raw event to feedback callbacks (#6289)
## Summary React v2 feedback callbacks receive an assistant message without the trace metadata carried by the direct AG-UI event that created it. This slice exposes that metadata to thumbs callbacks without changing canonical messages or future run inputs. ## Root cause AG-UI keeps `rawEvent` on events while reducer-created assistant messages remain protocol-clean. `StateManager` sees the direct start event but previously discarded its correlation before `CopilotChatMessageView` forwarded the message to feedback callbacks. ## Changes - Store defined direct `TEXT_MESSAGE_START.rawEvent` metadata by agent, thread, and message. - Replace repeated scoped entries and prune them with message removal and lifecycle cleanup. - Return a cloned sidecar value through `CopilotKitCore.getRawEventForMessage`. - Enrich only thumbs-up and thumbs-down callback arguments at click time across flat and virtualized rendering. - Add production-path regressions and document the callback-only type. ## Out of scope Canonical messages, future `RunAgentInput.messages`, render props, message identity, stream ordering, snapshots, transformed chunks, persistence, GraphQL, legacy React, Vue, Angular, and standardized trace semantics remain outside this slice. ## Related PRs and Issues Addresses #3039. The callback-only scope follows https://github.com/CopilotKit/CopilotKit/issues/3039#issuecomment-5086936452. Related trace-correlation contract: #4634. ## Test plan - [x] StateManager sidecar tests, 10 passed. Covers direct capture, falsey values, replacement, scope isolation, cleanup, snapshots, and chunks. - [x] React v2 feedback tests, 4 passed. Covers real callback routing, canonical and outbound cleanliness, render identity, and flat/virtual paths. - [x] Full package suites, 625 core tests, 1,475 React Core tests, and 2 script tests passed. - [x] Typecheck, formatting, lint, and whitespace validation passed; lint reported five pre-existing warnings. - [ ] CI green (`static / quality`, `test / unit` on Node 20/22/24). ## Notes The clean-base behavioral half of the reproduction remains unproved because temporary worktree setup hung behind unrelated Git processes. The PR makes no base execution claim for that half. |
||
|
Ben Taylor
|
d30983f406 |
refactor(vue): make tool-call memoization lint-valid (#5932)
## What does this PR do? Makes the existing Vue tool-call memoization lint-valid without changing rendering behavior. The existing `v-memo` placement on the fallback renderer inside the tool-call loop violates the `vue/valid-v-memo` placement constraint. This refactor introduces a Vue-valid component boundary while preserving the optimization contract: - Named `#tool-call-<toolName>` and generic `#tool-call` consumer slots remain reactive. - Only the registered fallback renderer remains memoized. - The fallback memo boundary matches the React counterpart's renderer-level optimization. The focused tests act as behavior-preservation and regression guards for slot updates and fallback rerender prevention. ## Related PRs and Issues - None. ## Verification - `CI=1 pnpm exec nx run @copilotkit/vue:test -- --run src/v2/components/chat/__tests__/CopilotChatToolCallsView.test.ts --reporter=dot` — 14/14 passed. - `CI=1 pnpm exec nx run @copilotkit/vue:check-types` — passed. - `CI=1 pnpm exec nx run @copilotkit/vue:build` — passed. - Direct ESLint from `packages/vue` on all touched source/test files — passed. - `git diff --check upstream/main...HEAD` — passed. - `CI=1 pnpm exec nx run @copilotkit/vue:lint` remains blocked by 172 pre-existing errors in unrelated files; no package-wide lint cleanup is included. - The broad commit-hook suite encountered an unrelated SSR timeout; the final tree was not changed afterward. Coverage preserves generic and named slot updates, unchanged fallback rerender prevention, tool-name changes, agent-specific renderer selection, status/result behavior, and renderer precedence. ## Scope and exclusions This is limited to the Vue tool-call rendering boundary, its parity note, and focused tests. It does not change React behavior, package-wide lint errors, attachment work, or unrelated rendering paths. ## Checklist - [x] Contribution guide and package instructions reviewed. - [x] Relevant Vue parity documentation updated. - [x] Allow edits by maintainers is enabled. |
||
|
Ben Taylor
|
27431412e6 |
fix(react-core): stop the compat CopilotKit wrapper pinning useSingleEndpoint (#6605)
Refs [OSS-888](https://linear.app/copilotkit/issue/OSS-888). ## The failure A correctly assembled v2 integration 404s on its first browser request while every static check passes and `GET /info` returns 200. `packages/react-core/src/v2/index.ts:28` re-exports the **v1-compat** `CopilotKit` wrapper, so it is the provider most integrations reach for. That wrapper pinned: ```tsx useSingleEndpoint={props.useSingleEndpoint ?? true} ``` which overrode the core's `"auto"` negotiation and forced single-route transport. But **every** v2 handler defaults to `mode: "multi-route"` (`endpoints/hono.ts:95`; `createCopilotEndpoint` is an alias at `:90`). Nothing serves the single-route envelope the client sends, so the runtime 404s while the provider looks connected. ## What this is *not* The library defaults do not actually disagree. `CopilotKitProvider` (the real v2 provider) leaves the flag undefined → `"auto"`, which probes `GET /info` and falls back to the single-route envelope (`core/agent-registry.ts` `fetchRuntimeInfoAutoDetect`) — it works against **either** handler mode. Only the compat wrapper defeated that. So this is one line of override, not a defaults mismatch needing a direction chosen. ## Why four onboarding runs hit it, not one The library bug alone doesn't explain a 100% failure rate. The shipped `react-core` skill does: `packages/react-core/skills/react-core/references/provider-setup.md` — bundled in the npm tarball (`files: ["dist","skills"]`) — **mandated** the compat wrapper, **forbade** `CopilotKitProvider` as "a subset of the functionality", and mentioned `useSingleEndpoint` **zero times** across ~10 code samples. An agent following it wrote the 404 configuration every time. Meanwhile `skills/copilotkit-setup/SKILL.md` got it right, so the two shipped skills contradicted each other and nothing gated either against the code. ## The change **Commit 1 — the library fix.** The prop already arrives through `v2Props`, so dropping the override lets it stay `undefined` and inherit `"auto"`. An explicit `useSingleEndpoint` still wins in both directions. **Commit 2 — the docs and skills.** Correcting the default made ~15 pages' explanations false. Code samples that pass `{false}` stay valid (they pin what negotiation would find anyway), so this corrects the *explanations* rather than the samples — keeping every page true both before and after release. Includes dropping the now-false causal claim from the single-route-envelope diagnostic added in #6579. ## Compatibility Safe for existing v1 apps. A v1 app on a single-route-only handler (`copilotRuntimeNextJSAppRouterEndpoint` and friends) now does one `GET /info` that 404s, then falls back to single-route and works. Cost is one extra request on connect. One edge case worth a reviewer's eye: if a deployment's `runtimeUrl` + `/info` returns 200 from something that is *not* a multi-route CopilotKit runtime (a catch-all proxy serving HTML, say), `"auto"` would resolve to `rest`. Setting `useSingleEndpoint` explicitly remains the escape hatch. Conventional-commit note: this lands as `fix`, but it *does* change a public default. Flag if you'd rather it carried a minor bump. ## Tests - New `copilotkit-transport-default.test.tsx` — omitted → `"auto"`, `{true}` → `"single"`, `{false}` → `"rest"`. Confirmed RED first (`expected 'single' to be 'auto'`). - `CopilotChat.readinessGate.test.tsx` depended on the old default to avoid a REST probe. Single-route transport is a **precondition of that fixture**, not the behaviour under test, so it now pins the flag explicitly and its stale comments are corrected. Its coverage (readiness gate across the real SSE boundary) is unchanged. - `react-core` 1512 passed · `runtime` 2073 passed · `core` 668 passed. - `pnpm check:plugin-skills` in sync (`skills/react-core/` is the generated mirror). `showcase/shell-docs`'s own vitest suite fails to load 35 files with `Cannot find package 'react/jsx-dev-runtime'` — reproduced identically on unmodified `origin/main`, so it is environmental in this checkout and unrelated. All 183 tests that do run pass. ## Not addressed here Nothing gates a shipped skill against the code it documents, which is why `provider-setup.md` could contradict both the library and the sibling skill indefinitely. Worth its own ticket. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Mark
|
0298616223 |
docs: add Claude Managed Agents cookbook (#6430)
## Summary - add an independently runnable Claude Managed Agents finance-assistant example - add a cookbook recipe that explains the CopilotKit runtime, managed-session mapping, and tool rendering flow - add the recipe to cookbook navigation, the overview grid, sidebar icon mapping, and render coverage - use the real Claude vector mark for the cookbook instead of the text-placeholder SDK asset - register the example's Vite configuration and managed-agent model with the repository CI allowlists - include a compact architecture diagram and links to the relevant rendering and CopilotKit Intelligence documentation - disable Claude's complete built-in toolset and expose only the scoped `show_growth_projection` runtime tool - make the provisioning model configurable through `ANTHROPIC_MODEL`, defaulting to `claude-fable-5` - bound CopilotKit request bodies to 256 KB and managed-agent turns to 90 seconds, while relying on the adapter's per-thread serialization - cap public run traffic at 20 provider-like attempts per client IP per minute and 2,000 successful starts per process per 24-hour window - restrict browser runtime requests with an exact Origin allowlist that supports same-origin or separately hosted frontends, and restrict iframe parents with CSP `frame-ancestors` - validate persisted managed-agent IDs at startup so malformed local configuration fails immediately - publish the interactive example on Railway and embed the live deployment in the cookbook - align the demo with the existing cookbook chat styling and show the `Project monthly investing` starter on first load ## Demo  ## Why This gives developers a focused example of connecting CopilotKit to Anthropic Claude Managed Agents without the extra surface area of a larger analyst application. The recipe follows the existing cookbook structure and keeps the live demo compact enough for the standard cookbook pane. Its managed environment has no outbound network or package-manager access, and its agent cannot use bash, filesystem, search, or fetch tools. The request, turn, per-IP, and process-wide limits bound public demo traffic without adding authentication or user friction. The traffic counters are intentionally in memory, reset on process restart, and are not shared across replicas, so the dedicated Anthropic workspace spend limit remains the durable cost backstop. The exact-Origin browser check reduces drive-by use but is explicitly documented as a control rather than authentication. The model override allows operators to select a lower-cost supported model during provisioning without editing source code. ## Validation - scoped formatting: passed - scoped lint: 0 warnings, 0 errors - shell-docs typecheck: passed - standalone example typecheck: passed - docs render tests: 26/26 passed - standalone example tests: 23/23 passed - shell-docs tests: 375/375 passed - shell-docs production build: passed (222/222 pages) - standalone example production build: passed - standalone npm lockfile validation: passed - build-config allowlist validator: passed - docs model-name validator: passed - exact-Origin regression coverage for run requests plus headerless same-origin runtime discovery: passed - malformed persisted agent-ID regression coverage: passed - live Railway root and iframe CSP: passed - live Railway runtime discovery, exact welcome copy, and first-load starter pill: passed - live three-turn AG-UI managed-agent run with `show_growth_projection`: passed - cookbook verified in the browser at desktop and narrow widths with no console errors or horizontal overflow |
||
|
Mike Ryan
|
b8b19834a2 |
fix(runtime): unify the Intelligence key name and publish the wiring (refs OSS-881) (#6595)
## What does this PR do? Closes the naming and documentation half of [OSS-881](https://linear.app/copilotkit/issue/OSS-881). Paired with CopilotKit/Intelligence#890, which adds `copilotkit verify` and tightens the evaluation rubric. ### 1. One name for the Intelligence key **Three** names for one value were live in CopilotKit's own documentation, and following the wrong one with a CLI-provisioned project yields an undefined key: | Name | Where | Code readers | | --- | --- | --- | | `INTELLIGENCE_API_KEY` | what `copilotkit project select` writes; all 34 integration examples; the docs site | 34 | | `COPILOTKIT_INTELLIGENCE_API_KEY` | 7 Channels package READMEs + packaged skills | **0** | | `COPILOTKIT_API_KEY` | `examples/slack`, `examples/teams`, and the TSDoc on `CopilotKitIntelligence` itself | 2 | `INTELLIGENCE_API_KEY` wins — it is the name the CLI provisions, and changing it would break every scaffolded project in the wild. - `COPILOTKIT_INTELLIGENCE_API_KEY` is **retired outright**. Nothing ever read it, so there is nothing to keep compatible. - `COPILOTKIT_API_KEY` stays **readable as a deprecated alias** in the two examples that consume it, so an existing `.env` keeps working, and is documented as deprecated everywhere it appears. The third name was the worst placed: it was in the TSDoc on `CopilotKitIntelligence`, which is what an IDE shows on hover. This was not only untidy. The CLI's own `channels-preflight` accepts `INTELLIGENCE_API_KEY` or `COPILOTKIT_API_KEY` — **not** `COPILOTKIT_INTELLIGENCE_API_KEY`, the name the Channels READMEs told people to set. So following a Channels README verbatim made `copilotkit channels` warn that no runtime API key was present while the key sat visibly in `.env`. After this PR the documented name is one preflight accepts. > [!NOTE] > `NEXT_PUBLIC_COPILOTKIT_API_KEY` is a **different value** — the legacy Copilot Cloud public key — and is deliberately left alone. ### 2. A real defect, not just naming skew `skills/runtime/references/intelligence-mode.md` documented `organizationId` as a `CopilotKitIntelligence` option, sourced from two further env names (`COPILOTKIT_INTELLIGENCE_ORG_ID`, `COPILOTKIT_ORG_ID`). `CopilotKitIntelligenceConfig` has no such field — the copy-pasteable sample it appeared in **would not compile**. Removed from the samples, and the prose telling readers to fetch a value for it corrected. That file is the only place those two names ever existed, which is very likely why the failing validation run reported that "the runtime reads `COPILOTKIT_INTELLIGENCE_API_KEY` and `COPILOTKIT_INTELLIGENCE_ORG_ID`". ### 3. Publish the Intelligence wiring The wiring instructions existed only inside `node_modules/@copilotkit/runtime/skills/`, and the only docs pages mentioning `CopilotKitIntelligence` at all were the two Channels frontends — so a developer on the plain web path had no page to reach it from. Adds **`/premium/connect-your-runtime`**: the wiring itself, how to confirm the credential is actually consumed, the self-hosted both-URLs-or-neither rule, and a troubleshooting table. Linked into both navs, and the skills reference now points at the published page. ### 4. A guard so it cannot drift back `scripts/validate-intelligence-env-names.ts` (`pnpm check:intelligence-env-names`), wired to lefthook and a new workflow. The workflow is **intentionally unfiltered**. The two workflows that would otherwise cover this both filter: `plugin-skills-check` by `paths:`, and `static/quality` by `paths-ignore: examples/**` — which is exactly where the deprecated alias lives. Scoping the job would re-open the hole it exists to close. Legitimate alias sites live in `ALIAS_ALLOWLIST`. ## Related PRs and Issues - [OSS-881](https://linear.app/copilotkit/issue/OSS-881) — needs **both** PRs; neither closes it alone - CopilotKit/Intelligence#890 — items 1 and 4 (`copilotkit verify` + rubric contract 1.3.0) ## Verification - Full lefthook pre-commit ran green: `check-plugin-skills`, `lint-fix`, the new `check-intelligence-env-names`, and `test`/`publint`/`attw` across **25 projects**. - `examples/slack` `managed.test.ts` extended to cover **both** the canonical name and the alias fallback, and proven non-vacuous — removing the fallback turns the new test red. - The drift guard proven non-vacuous the same way: reintroducing a retired name fails it, exit 1. - `oxfmt` and `oxlint` clean on every file touched (0 errors). ## Checklist - [x] I have read the [Contribution Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md) - [x] If the PR changes or adds functionality, I have updated the relevant documentation 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Mark
|
f36cb2b8ee | fix(showcase): validate persisted setup IDs | ||
|
Benjamin Taylor
|
f30d3bfae5 |
docs: correct the useSingleEndpoint default across docs and shipped skills
The compat `<CopilotKit>` wrapper no longer pins `useSingleEndpoint` to `true`,
so every statement that it "defaults to single-route" or that a multi-route
backend "needs `{false}`" is now wrong. Code samples that pass `{false}`
explicitly stay valid — they pin what negotiation would find anyway — so this
corrects the explanations rather than the samples, keeping the pages true both
before and after the release.
The shipped `react-core` skill is the load-bearing one. `provider-setup.md`
mandated the wrapper, forbade `CopilotKitProvider` as "a subset of the
functionality", and never mentioned `useSingleEndpoint` across ~10 samples — so
an agent following it wrote the 404 configuration every time. It now documents
the transport and stops steering readers off the negotiating provider.
Also drops the false causal claim from the runtime's single-route-envelope
diagnostic (added in #6579), which named the wrapper's old default as the cause.
`skills/react-core/` is the generated mirror of `packages/react-core/skills/`,
synced with `pnpm sync:plugin-skills`.
Refs OSS-888.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Mark
|
65f07610a1 | fix(showcase): allow runtime discovery request | ||
|
Benjamin Taylor
|
dc73af1dc4 |
fix(react-core): stop the compat CopilotKit wrapper pinning useSingleEndpoint
The `CopilotKit` wrapper is re-exported from `@copilotkit/react-core/v2`
(`v2/index.ts`), so it is the provider most integrations reach for. It pinned
`useSingleEndpoint={props.useSingleEndpoint ?? true}`, which overrode the core's
`"auto"` negotiation and forced single-route transport.
Every v2 handler defaults to `mode: "multi-route"`, so the pinned default made
the first browser request 404 while `GET /info` still returned 200 and looked
healthy. Four independent onboarding runs hit it and all four fixed it the same
way, with `useSingleEndpoint={false}`.
The prop already arrives through `v2Props`, so dropping the override lets it
stay `undefined` and resolve to `"auto"` — probe `GET /info`, fall back to the
single-route envelope — which works against either handler mode. An explicit
`useSingleEndpoint` still wins in both directions.
`CopilotChat.readinessGate.test.tsx` depended on the old default to avoid a REST
probe. Single-route transport is a precondition of that fixture rather than the
behaviour under test, so it now pins the flag explicitly and its comments are
corrected.
Refs OSS-888.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Mark
|
d50e8d7e7c | Merge branch 'main' into codex/claude-managed-agents-cookbook | ||
|
Mark
|
ff060eb97b | feat(showcase): polish managed-agent cookbook demo | ||
|
Ben Taylor
|
6787b30203 |
fix(core): preserve run IDs across SSE connect replay (#6253)
## What does this PR do? - Uses each `RUN_STARTED.runId` during Runtime SSE connect replay. - Preserves the first run association when later message snapshots are cumulative. - Keeps live message events able to correct an earlier provisional run association. - Adds `/connect` and StateManager regression coverage for multiple server runs. ## Why? A single `/connect` stream can contain multiple runs. This affects custom Runtime streams and the built-in in-memory runner, which replays historic runs through one reconnect stream. StateManager previously stored replayed state under the connection input ID and reassigned earlier snapshot messages to the latest run. ## Related PRs and Issues - Closes #6252 ## Validation - `@copilotkit/core` tests: 59 files, 661 tests passed - `@copilotkit/core` type check - Pre-commit lint, test, publint, attw, and commitlint ## Checklist - [x] I have read the Contribution Guide - [x] Documentation is not required for this internal bug fix - [x] Allow edits by maintainers is enabled |
||
|
Mark
|
7089b3cf53 | fix(showcase): harden managed-agent deployment setup | ||
|
Martha Kelly Schumann
|
197dbe292a |
Complete the tool-rendering docs example (#6597)
## Summary - provide one dependency-complete, docs-only frontend example for tool rendering - keep runtime showcase frontends byte-identical across Google ADK, LangGraph Python, and Mastra - retain each selected framework's backend tool definition - cover visual MDX inlining, frontend parity, and generated LLM-text routes This is the clean FAC-71 replacement for stale PR #6217; it does not reuse that branch or its obsolete route ownership. ## Validation - `npm run pretypecheck` - `npx vitest run src/lib/__tests__/llm-text.test.ts src/lib/__tests__/tool-rendering-docs.test.ts` (41 tests) - `npm run typecheck` - `npm run lint` (passes with existing warnings) - `npm run build` (223 pages generated) Linear: [FAC-71](https://linear.app/copilotkit/issue/FAC-71/google-adk-tool-call-rendering-docs-missing-weathercard-and) |
||
|
David McKay
|
35ea9eee36 |
feat(reskinnable-demo): run banking on a nested LangChain deep agent with a streaming CLI console (#6581)
Replaces the Codex CLI harness with **LangChain deep agents (Python)**:
sandboxed shell execution, parallel research subagents, no external
harness binary, our own API key.
All ten beats have been walked in a browser, and the demo now has its
own CI job. **Ready for review.**
## What banking's agent is now
```
banking gpt-5.4, temp 0
│ banking's own prompt · the browser's frontend tools · Intelligence
│ memory tools · render_report for the canvas
└─ expense-analyst gpt-5.6-sol, reasoning_effort=high
│ sandboxed shell (LocalShellBackend) · submit_expense_report
└─ merchant-researcher gpt-5.4, one per merchant
search_merchant (Tavily)
```
Six skins still run in-process as `BuiltInAgent`s. `banking` is an
`HttpAgent` pointed at `agent/`.
Each level exists because something about it must differ from its parent
— prompt, model, reasoning effort, tool set. A single agent had nowhere
to put any of that.
### Why the whole agent moved, not just the expense beat
The obvious design was a second agent id for the long-running beat,
leaving banking's `BuiltInAgent` alone. That was built first, and it
does not survive the requirement *"start the analysis, switch threads,
run other pills, come back to it."*
**Threads are scoped per agent.** `listThreads` takes `agentId` as a
required parameter, and measured against the running app the two lists
were disjoint — `banking` 46 threads, `banking-expenses` 10. The v2
runtime has no handoff mechanism, and `defineTool`'s `execute` takes
`(args)` with no emitter, so a tool cannot stream a multi-minute run
into the conversation. One conversation list means one agent.
### Why the beat is a subagent rather than prose in one prompt
It started as a section appended to banking's prompt. That left per-beat
configuration homeless (model, effort and recursion limit are all
agent-level, and there was one agent), and made banking's
~21,000-character rulebook ride every one of the ~20 model calls the run
makes — re-sending rules about markdown tables while the agent read a
CSV.
Reached as a `CompiledSubAgent`, because a raw `SubAgent` spec has no
`subagents` field and this one needs its own: the per-merchant fan-out
is a headline of the beat, and a flat subagent could only research
serially. A probe confirmed nesting survives — the analyst's `task`
dispatches, the researchers' `search_merchant` calls and the final
report tool all reach `astream_events`.
## The console
One CLI window in the transcript carries the whole run: narration, `$
execute`, `search "…"`, `→ merchant-researcher: …`, results, indented by
depth so a ten-way fan-out reads as a fan-out. Subagent narration is
suppressed from the conversation so the console is the single place the
harness is visible; banking's own replies still render normally.
`shell/subagents/subagent-activity.tsx` subscribes to the agent's
**event stream**. Reading `agent.messages` — the first design — was
wrong twice: messages materialise at the `MESSAGES_SNAPSHOT` (two per
run), so the pane sat still for minutes then filled at once; and
persisted messages carry no `subagentRunId`, so the harness's narration
could not be told from banking's reply. The fold is pure and idempotent,
so the same code serves the live subscription and a replay of the
thread's stored events when a conversation is reopened.
Three heuristics were deleted along the way, each replaced by identity
the protocol already carried:
| heuristic | replaced by |
|---|---|
| console anchored on "the first tool call" | the run's first `task`
call, from **message order** (durable; the event-derived version
rendered one console *per delegation* on a restored thread — six,
measured) |
| `CONSOLE_TOOL_NAMES` allowlist | presence of `subagentRunId` |
| `disable_streaming` on the researchers | the canary's per-lane state |
The message filter suppresses **prose** and keeps **tool calls**: an
agent routinely narrates and calls a tool in one message, and returning
`null` for the whole message hid the report card — the run looked
perfect and ended with nothing to show.
## Canary stack, contained to this app
The subagent surface only exists on the canary line, and a released
`@ag-ui/client` ≤ 0.0.57 rejects `SUBAGENT_*` events **in the HTTP
transport before any middleware runs**, killing the stream. So the demo
leaves the root pnpm workspace and ships its own lockfile:
- `@copilotkit/* 1.68.3-canary.1786716392`
- `@ag-ui/* 0.0.59-canary.1786716392.0`
- `ag-ui-langgraph 0.0.43.dev1786716392` → `ag-ui-protocol
0.1.20.dev1786716392`
That keeps an unreleased protocol out of every other package in the
monorepo.
A 1.62.2-based canary was tried first and **could not compile the app**:
`workspace:*` is not a version, so the demo had no recorded lower bound
on the CopilotKit API it needs, and pointing it at 1.62.2 silently
rewound that API five minors. `OpenGenerativeUIActivityRenderer` — a
public `/v2` export since ~1.63 — was the first thing to break, and
there was no reason to think it was the only one. Rebasing the canary on
1.68.3 collapsed that whole class of risk.
## Verified against the running app
Every row is a measurement, not a claim.
| | |
|---|---|
| Registry shape | `/info`: `banking` = HttpAgent, other six =
BuiltInAgent |
| Sandboxed execution | 8 `execute` calls; agent writes and runs its own
Python |
| Parallel subagents | 6 `SUBAGENT_STARTED`/`FINISHED` pairs, 10
`search_merchant` |
| Canvas beat | `render_report` → a2ui middleware emits `activityType:
a2ui-surface` |
| Frontend tools | given two, picks `showTransactions`, emits no result,
emits **no prose** — the prompt's restraint rule surviving the port |
| Memory (Intelligence) | run is handed `recall_memory`, `save_memory`,
`forget_memory`, `knowledge_base_shell` |
| HITL round-trip | tool call out, answer back in, agent continues |
| Durable background run | client disconnected at 8s; run finished
unattended; thread replayed |
| Thread restore | 61 messages persisted incl. the report tool (an
earlier flat-subagent attempt collapsed to 4) |
| Report correctness | 14 rows, 9 researched, 6 filed with ids read out
of real 201 bodies, totals reconciling against their own rows |
| Run duration | 86s reported vs 98s wall clock |
## Correctness bugs found by running it
Each produced a confident, complete-looking wrong answer rather than an
error — the characteristic failure of a multi-minute agentic beat.
- **Beat 3d was dead, and it took the whole thread with it.** The bet
against this one was right. `ag_ui_langgraph` routes *every* attachment
to the model as an `image_url` block, documents included, so the Q2
invoice was rejected before the first token: `400 Invalid MIME type.
Only image types are supported`. The exception is raised inside the
model node, which kills the SSE stream — the runtime sees `RUN_ERROR:
terminated` with no cause and the browser renders **nothing**: no error
bubble, no failed message. And the crashed run is still checkpointed, so
every later message on that thread replays the rejected content and dies
the same way. One click on the pill killed the conversation permanently;
only restarting the service cleared it. Fixed here by a
`wrap_model_call` middleware that rewrites those blocks into LangChain
standard `file` blocks, and upstream in
[ag-ui#2476](https://github.com/ag-ui-protocol/ag-ui/pull/2476) (both
adapters, plus the return leg so a non-image attachment survives
`MESSAGES_SNAPSHOT`). The middleware is a stopgap with its deletion
condition written into its header — this service installs the adapter
from PyPI, so the upstream fix cannot reach it until published.
- **Totals did not match their own rows.** Every per-row amount matched
the CSV while the headline totals came back $1.00 and $0.20 high: the
model authored them instead of adding them. The card prints the total
directly above the rows it is the sum of. Fixed structurally — derived
in `submit_expense_report`, removed as parameters.
- **`amount` arrived as a string** on all 14 rows, silently defeating
`toLocaleString(…currency…)` so it printed `842.10` with no currency.
- **`merchantKind` non-answers.** With no search tool the model wrote a
bare `"unclear"`; with Tavily live it hedges in prose (`"unknown (likely
bookbindery/bookshop retail, but not established for this exact
merchant)"`), which an exact-match filter passed into a 60-character
label glued to the merchant name. Now rejected on leading token, hedging
language, and a 40-character cap.
- **The run clock reported 333s for a two-minute run.** It took the
oldest open stamp across the process because the tool could not name its
own run; model calls *after* the report re-stamped it and the leftover
became the next run's start. Now keyed per run via an injected
`ToolRuntime`.
- **`graph.with_config({"recursion_limit"})` is silently dropped** by
the AG-UI adapter. The agent completed the entire analysis, streamed
every argument of the final report, then died at LangGraph's default of
25 supersteps.
- **A sync `wrap_model_call` under `astream`** surfaces as a bare
`RUN_ERROR: terminated`, cause only in the service log.
- **`emit_raw_events` defaults to `True`**, piggybacking LangChain's
internal events onto the stream: 27,950,261 bytes → 374,086 with it off,
identical report. Matters because the thread *persists* those events for
replay.
- **`gpt-5.6-sol` + `reasoning_effort` + function tools 400s** on
`/v1/chat/completions`; needs `use_responses_api=True`. The first model
probe missed it by binding no tools — a model probe for an agent has to
bind one.
## Upstream findings (reported separately, not fixed here)
1. **`@copilotkit/runtime` drops `subagentRunId` when persisting
messages.** 2888 of 3026 stream events carry it; 0 of 53 persisted
messages do. Reproduced with Intelligence removed entirely, so it is the
runtime's message shape rather than the platform store — and
`@copilotkit/runtime`'s dist contains no occurrence of the field, while
`@ag-ui/core`, `ag-ui-protocol` and `@copilotkit/core` all model it. One
field threaded through would let the console rebuild from message
history and delete the event-replay seeding here.
2. **`copilotkit` 0.1.95 × `ag-ui-langgraph` 0.0.43** — the FastAPI
endpoint calls `agent.clone()` per request; 0.0.43's base `clone()`
hard-passes three kwargs the SDK subclass does not accept, so **every
request 500s**. Verified with a minimal repro on stock classes and by
reading published wheels (0.0.41/0.0.42 are fine — the window is 0.0.43
only). `sdk-python` requires `>=0.0.42` unbounded, so fresh installs
break, and `examples/showcases/deep-agents{,-finance-erp,-job-search}`
are one `uv lock --upgrade` away. Now open as #6592 (`**kwargs`
passthrough + 9 regression tests incl. a forward-compat guard); it needs
a `0.1.96` bump to reach PyPI before `main.py`'s
`BankingAGUIAgent.clone()` workaround can go — and note `agent/uv.lock`
pins `copilotkit 0.1.95`, so removing the workaround is a re-lock as
well as a delete. That branch also fixes 4 pre-existing failures in the
sdk-python suite from the same root cause — **`test_unit-python-sdk` may
currently be red on `main`; worth checking independently.**
3. **Both LangGraph AG-UI adapters send non-image attachments as
`image_url`**, so a PDF, audio clip or video is rejected on the block
kind — see beat 3d in *Correctness bugs*. Fixed in
[ag-ui#2476](https://github.com/ag-ui-protocol/ag-ui/pull/2476); carried
here as a middleware until that publishes.
Also verified and **not** a problem: pnpm 10.33.4 still applies
`package.json` `pnpm.overrides` despite warning that it ignores them —
the ~70 root overrides including the security pins are live, nothing was
silently unpinned. (Migration to `pnpm-workspace.yaml` is worth doing
anyway, since the installed pnpm is 11.21.0 where the field genuinely is
dropped; branch exists, byte-identical lockfile.)
## What is not done
1. **Docs are stale.** `CLAUDE.md` still says `AgentRegistration` is `{
createAgent: () => BuiltInAgent }` and that the route "builds one
`BuiltInAgent` per registered skin"; the reskin skill says the same in
three places, and `templates.md` scaffolds it. The real type is `() =>
AbstractAgent`. This app has a standing rule that every change answers
whether the skill went stale. It did — the skill's launcher step is
fixed, the `BuiltInAgent` claims are not.
3. **Pre-release dependencies**, now on both halves: the JS canaries in
this app's own `pnpm-lock.yaml`, and `ag-ui-langgraph` /
`ag-ui-protocol` pinned `==` to the matching `.dev` canaries in
`agent/pyproject.toml`. Both move to stable when the subagent work
ships.
4. `pnpm test:e2e` has not been run.
### Closed since this PR was opened
- **The demo has its own CI job** —
`.github/workflows/test_reskinnable-demo.yml`. Nx discovers projects
*through* the pnpm workspace (no `workspaceLayout` in `nx.json`), so
leaving it also left the repo-wide `nx run-many -t build`
(`static_compat.yml`), `-t check-types` (`static_quality.yml`) and `-t
test` (`test_unit.yml`) sweeps, and both static workflows carry
`paths-ignore: ["examples/**"]` besides. One job runs the four gates
(lint, typecheck, 2460 unit tests, build); a second syncs
`agent/uv.lock` with `--frozen` and asserts the subagent *capability*
rather than a version string, so it keeps meaning something after the
pin moves.
The build gate is the one a developer can least run locally: `next
build` corrupts a concurrently running dev server's PostCSS/Turbopack
cache — measured, `globals.css` transforms to garbage, every route 500s,
and a dev-server restart does not clear it because the corruption is on
disk.
**Its first two runs failed, both usefully**, and both bugs were
pre-existing:
1. `pnpm/action-setup@v6.0.10` with `package_json_file:` pointed at this
app still installed the **root's** pnpm 10.33.4 rather than this app's
pinned 10.10.0 — the resolver that ignores `pnpm.overrides`, which is
the only place three of the five `@ag-ui/*` canaries are pinned. Caught
only because the job asserts `pnpm --version` against the pin; it would
otherwise have installed under the wrong resolver and stayed green. Now
activated through corepack, which reads the nearest `package.json`.
2. `pnpm install` in this directory **never installed this app**. pnpm
walks *up* for a workspace root, found the repo's, and installed all 70
monorepo projects (4645 packages) while leaving this directory with no
`node_modules` — the next command failing as `sh: 1: eslint: not found`.
So the app that ships its own lockfile was uninstallable by its own
documented instruction. Fixed by giving it its own
`pnpm-workspace.yaml`; `ignore-workspace=true` in an `.npmrc` does
**not** work (CLI-only in pnpm 10.10, measured).
- **The canary `overrides` now live in `pnpm-workspace.yaml`**, their
supported home, as a side effect of that fix. They no longer depend on
`packageManager: pnpm@10.10.0` being the version that still reads
`package.json` — which matters because `@ag-ui/core`, `@ag-ui/encoder`
and `@ag-ui/proto` are pinned nowhere else and a released `@ag-ui/core`
rejects `SUBAGENT_*` in the HTTP transport.
- **All ten beats walked in a browser**, including 3d — which was
broken, for the reason in *Correctness bugs* above.
- **`./run-demo.sh` starts a complete demo.** It now brings up the
Python agent between the compose wait and `pnpm dev`, guarded on
`/health` so a re-run reuses a live one, and dies with `(cd agent && uv
sync)` when the venv is missing. `./stop-demo.sh` had the mirror gap and
now stops it too — that one mattered more than it looks: the start
script *reuses* a live :8124, so an orphan surviving teardown is
silently adopted by the next cold start, serving whatever code it was
launched with.
- **The agent's Python deps are pinned and locked.** They were not, and
the JS half was — so a colleague's fresh `uv sync` resolved the
*release* `ag-ui-langgraph 0.0.43`, which does not accept
`emit_subagent_events` and ships no subagent symbols. That is this PR's
headline feature, and it would have failed silently: `main.py` sets the
flag as an attribute, so the assignment succeeds against an object
nobody reads and the service starts clean. Now `==` pins plus a
committed `agent/uv.lock`, verified by building a venv from only the
tracked files and replaying a captured Q2-with-PDF payload through it.
- **The README told people to install from the repo root** "as a
workspace package". It is deliberately not one, so a root install did
nothing for this app — and installing in the right place did nothing
either, until the `pnpm-workspace.yaml` above.
## Relationship to #6501
Based on `b94e4bfb5d`, the last commit before Codex appears, which is
**not on `main`**. So this PR carries **9 commits: the 4 here plus the 5
foundation commits it shares with #6501** — harness types + fixture,
prompt/workspace, the OFFSITE-to-fixture invariant guard, the
summary-shape/filing-contract fix, and `POST /transactions`. Whichever
merges first shrinks the other. #6501 and #6565 are deliberately
untouched.
## A demo-design question, not a bug
The fixture's merchant names are invented, so `Cardinal & Ash` — the
prompt's own worked example of *"could be a restaurant or a law firm,
find out"* — cannot be resolved by real web search and stays `unclear`
alongside `Bluebonnet Provisions`. The beat's headline claim is that the
agent researches every merchant; real merchant names in the CSV would
make that land harder.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01JRubZT6AS6LCGkcE2KzcfA
|
||
|
Tyler Slaton
|
34b10737b0 |
chore: release monorepo v1.68.3 (#6601)
## Release monorepo v1.68.3 **Scope:** `monorepo` | **Bump:** `patch` --- ### How this release process works 1. **This PR was created automatically** by the "release / create-pr" workflow. It bumped the `monorepo` packages to `1.68.3` and generated AI-enhanced release notes. 2. **CI runs on this PR** — the full test suite (unit tests, lint, type checks, build) must pass before merging. This is the review gate. 3. **Review the release notes** in `release-notes.md` in this PR. If a Notion draft was created, you can edit the release notes there before merging. 4. **When this PR is merged**, the `release / publish` workflow automatically: - Builds all packages - Publishes the `monorepo` packages to npm at version `1.68.3` - Creates git tag `monorepo/v1.68.3` - Creates a GitHub Release with the final release notes ### Before merging - [ ] CI is green (tests, lint, types, build) - [ ] Version bumps look correct - [ ] Release notes are accurate (edit in Notion if a draft was created) --- > **Do not merge until CI is fully green.** The full test suite runs automatically on this PR.v1.68.3 |
||
|
Maxim
|
6623dcfd4f |
fix(reskinnable-demo): make the app its own pnpm root
`pnpm install` in this directory did not install this app. Leaving the root workspace's member list was only half of it: pnpm walks UP from the cwd for a workspace root, found the repo's, and installed THAT — measured in CI (run 32398378642), "Scope: all 70 workspace projects", 4645 packages resolved for the monorepo, and this app left with no node_modules. The next command then failed as `sh: 1: eslint: not found`, which reads as a broken toolchain rather than an install that went elsewhere. A `pnpm-workspace.yaml` here stops the walk. `ignore-workspace=true` in an `.npmrc` does not — it is CLI-only in pnpm 10.10 (measured: the Scope line was unchanged), so the alternative would have been a flag every human and job had to remember. The five canary `overrides` move into that file, their supported home. They stay duplicated in `package.json`'s `pnpm` field for now, which is read only because this app pins `packageManager: pnpm@10.10.0` — three of the five (`@ag-ui/core`, `@ag-ui/encoder`, `@ag-ui/proto`) are pinned nowhere else, and a released `@ag-ui/core` rejects SUBAGENT_* events in the HTTP transport, so a packageManager bump would have silently killed the harness console. Verified read-only: `pnpm install --frozen-lockfile --lockfile-only` in this directory resolves the single project, satisfies the committed lockfile, and leaves it byte-identical. |
||
|
Maxim
|
80e83c8186 |
ci(reskinnable-demo): activate the app's pnpm via corepack, key the import smoke
First run (32398188492) failed twice, both usefully:
* `pnpm/action-setup@v6.0.10` with `package_json_file:` pointed at this
app still installed the ROOT's pnpm 10.33.4 rather than the app's pinned
10.10.0, and it cannot take a `version:` alongside a `packageManager`
field. The version assertion caught it instead of the job silently
installing under a resolver that ignores the three override-only
`@ag-ui/*` canary pins. corepack reads the nearest package.json, so it
resolves the app's pin and honours its +sha512 hash. The pnpm store is
now cached explicitly, keyed on this app's lockfile, since setup-node's
implicit pnpm cache needs pnpm installed before it runs.
* The import smoke needs OPENAI_API_KEY. `main.py` calls `build_agent()`
at import time and that raises without it. A placeholder is enough —
constructing a ChatOpenAI does not validate the key or touch the network.
|
||
|
copilotkit-qa-bot[bot]
|
589239686e | make tool-rendering docs layout executable | ||
|
Maxim
|
a36bef985d |
ci(reskinnable-demo): gate the demo's four gates and its agent's lockfile
Nothing in CI built or type-checked this app. Leaving the root pnpm
workspace — which was right, it contains the canary line — also removed it
from every repo-wide sweep, because Nx discovers projects THROUGH the
workspace and there is no `workspaceLayout` in `nx.json`:
nx run-many -t build static_compat.yml does not see it
nx run-many -t check-types static_quality.yml does not see it
nx run-many -t test test_unit.yml does not see it
Both static workflows also carry `paths-ignore: ["examples/**"]`, so this
is a new workflow rather than an edit to either. It follows
`test_unit-showcase.yml`, which exists for the same reason applied to a
different directory.
Two jobs.
`gates` runs lint, typecheck, unit (2460 tests) and build, cheapest first,
inside the app because no root task reaches it. Measured locally at ~2
minutes of gates; the 20m budget is install headroom.
The build gate is the one that matters most and the one a developer can
least run: `next build` corrupts a concurrently running dev server's
PostCSS/Turbopack cache — measured, `globals.css` transforms to garbage
and every route 500s, and a dev-server restart does not clear it because
the corruption is on disk. CI is the only safe home for it. It needs no
env: the route constructs one agent per skin at module load and none of
them requires a key (banking's is an `HttpAgent` whose URL is never called
during a build).
`agent-resolve` syncs `agent/uv.lock` with `--frozen` and then asserts the
subagent surface is actually present. That is not hypothetical: until this
branch pinned it, `ag-ui-langgraph>=0.0.43` resolved the RELEASE, which
accepts no `emit_subagent_events` and exports no subagent symbols — the
demo's headline feature, failing silently because the flag is set as an
attribute on an object nobody reads. The assertion is on the CAPABILITY
rather than the version string, since a version assertion goes stale the
moment the pin moves and the surface is what must stay true.
One uncertainty is made into a gate rather than left to trust: this app
pins pnpm@10.10.0 while the root pins pnpm@10.33.4, and the older resolver
is load-bearing here — it still reads `pnpm.overrides` from package.json,
the only place `@ag-ui/core`, `@ag-ui/encoder` and `@ag-ui/proto` are
pinned to the canary (`@ag-ui/client` is a direct dependency; those three
are not). `pnpm/action-setup` is pointed at the app's package.json, and
because an unexpected action input is a WARNING in Actions rather than an
error, a step then compares `pnpm --version` against the pin and fails
loudly if they differ.
No `continue-on-error` and no `|| true` anywhere in the file.
|
||
|
copilotkit-qa-bot[bot]
|
08b3e15b77 | Merge remote-tracking branch 'origin/main' into codex/fac-71-tool-rendering-docs | ||
|
BenTaylorDev
|
aa3fb29dce | chore: release monorepo v1.68.3 | ||
|
Tyler Slaton
|
7efa3b64f0 | fix(runtime): send global telemetry properties on the field the sink reads (#6603) | ||
|
David McKay
|
16e2c3a69a |
fix(runtime): send global telemetry properties on the field the sink reads
Folded into `properties`, they arrived in the per-event slot. That works and it is the wrong place: the sink treats `global_properties` as the pass-through bag for `oss.runtime.*` and spreads it into the analytics event, and v1's client sends package name and version there for the same reason. Sending them as their own field keeps a process-level fact separable from an event's own properties the whole way to the warehouse. It also moves conflict resolution. Two fields cannot collide in the SDK, so a shared key survives on both and the sink decides, spreading the global bag last and therefore letting the global win. That is the opposite of what most readers expect from the word global, so the field docs now say not to reuse a key an event already sets, and a test pins the behaviour rather than leaving it to be discovered. |
||
|
Ben Taylor
|
dcaface1ed |
feat(runtime): let a caller name itself on the telemetry it already sends (#6599)
## The problem
The v2 telemetry client sends exactly the properties each call site
passes. There is no way for a product built on this runtime to be told
apart in the events that already go.
That leaves one option open to such a product: send its own events.
Which means a second pipeline describing the same runs, a second
namespace at the ingest gate, and two sources of truth for "how much
traffic came through us".
## What this adds
`telemetryProperties` on the runtime, merged into every event the client
sends.
```ts
new CopilotRuntime({
agents,
telemetryProperties: { accessibility_title: "OpenBot" },
});
```
Set beside the license token, in the shared base, for the same reason
that is: it describes the caller rather than the call, so every event
carries it whichever handler fired, `instance_created`,
`copilot_request_created` or any of the `agent_execution_stream_*`.
## Two decisions worth naming
**Per-event properties win on conflict.** A call site describing one
event knows more than a value set once at construction, so the general
must not overwrite the specific. Tested.
**No egress on its own.** Unset, nothing changes. Telemetry off, nothing
is sent, so nothing carries this. It adds a field to existing events
rather than adding events.
## Why
OpenBot needs to be separable from other runtime traffic in the existing
OSS analytics, and the ask there was explicitly *one field, no new
events, no new pipeline, no new namespace*. Without a seam here, the
only way to answer "which requests came through OpenBot" is to build the
thing nobody wanted.
The client is a private module singleton and the package `exports` map
does not expose it, which is correct, so a consumer cannot reach it to
set this itself. Verified by probing the deep import from a consuming
package: blocked.
## Tests
Six new, at the send boundary rather than on the instance, because a
field held correctly and dropped on the way out is the failure that
matters:
- carried on an event that sets none of its own
- carried alongside an event's own properties
- successive calls merge rather than replace
- an event's own property wins on conflict
- nothing sent at all when telemetry is disabled
- nothing added when none are set
`packages/runtime`: 11 telemetry tests pass, 1270 v2 tests pass, 0
failures. Build clean, `telemetryProperties` present in the emitted
`.d.mts`. Pre-commit gate green (lint 0 warnings, monorepo tests,
commitlint).
|
||
|
copilotkit-qa-bot[bot]
|
1698420360 | keep tool-rendering example docs-only | ||
|
David McKay
|
86a9f9b016 |
feat(runtime): let a caller name itself on the telemetry it already sends
The v2 telemetry client sends exactly the properties each call site passes, so there is no way for a product built on this runtime to be told apart in the events that already go. The only route open to one was to send its own events, which is a second pipeline describing the same runs. `telemetryProperties` on the runtime is merged into every event. Set beside the license token and for the same reason: it describes the caller rather than the call, so every event should carry it whichever handler fired. Per-event properties win on conflict. A call site describing one event knows more than a value set once at construction, and letting the general overwrite the specific would be the wrong way round. No behaviour change when unset, and nothing is sent when telemetry is off, so this adds no egress on its own. |
||
|
github-actions[bot]
|
cfde633882 | style: auto-fix formatting | ||
|
copilotkit-qa-bot[bot]
|
43fe5fde9d | make tool-rendering docs dependency complete | ||
|
Benjamin Taylor
|
ed031949ca | docs(shell-docs): state the Intelligence wiring steps directly | ||
|
Maxim
|
cf9de905ac |
fix(reskinnable-demo): pin and lock the agent's Python canaries
A colleague cloning this branch could not reproduce the demo. The JS half
is pinned exactly — this app left the root pnpm workspace and ships its
own `pnpm-lock.yaml` with `@ag-ui/client 0.0.59-canary.1786716392.0` — but
the Python half pinned nothing: `ag-ui-langgraph>=0.0.43` and no
`uv.lock`, so a fresh `uv sync` resolved the RELEASE.
Measured, on the release that `>=0.0.43` actually selects:
ag-ui-langgraph==0.0.43
emit_subagent_events accepted by LangGraphAgent.__init__: False
subagent symbols in ag_ui.core: NONE
That is this branch's headline feature — the streaming CLI console — and
it would have failed SILENTLY. `main.py` sets `emit_subagent_events` as an
attribute (copilotkit's subclass takes only four kwargs), so on a release
without the feature the assignment succeeds, lands on an object nobody
reads, and the service starts clean. No `subagentRunId` reaches the
browser, the console cannot separate the harness's work from the parent's,
and a reopened thread collapses a multi-minute run to one tool message.
Every gate stays green.
So `ag-ui-langgraph` and `ag-ui-protocol` are now `==` pins, and
`agent/uv.lock` is committed. `ag-ui-protocol` is pinned as a DIRECT
dependency although nothing imports it by name: it carries the SUBAGENT_*
event types and the adapter asks only for `>=0.1.15`, so left transitive
it resolves the release and undoes the other pin.
README: the quick start said `pnpm install # from the repo root — this is a
workspace package`. It is not one — it is absent from
`pnpm-workspace.yaml`, deliberately, so the canary line cannot leak into
the rest of the monorepo. A root install therefore installs nothing for
this app, which is a confusing first five minutes for anyone who reads it
and follows it.
Verified by cold start rather than by inspection: copied ONLY the files
git tracks (the five .py modules, pyproject.toml, the new uv.lock) into an
empty directory, ran `uv sync --frozen`, and got
ag-ui-langgraph 0.0.43.dev1786716392 with `emit_subagent_events accepted:
True` and the five SUBAGENT symbols present. Then booted that venv on a
spare port and replayed the browser's captured Q2-with-PDF payload
through it: RUN_FINISHED, with createReport carrying the invoice's real
line items. The live stack was not touched.
Reskin skill: checked, no impact. Its install/verify step is `pnpm dev`
inside this app, which is correct either way; the root-vs-here distinction
is a README concern and the skill never mentions the workspace.
|
||
|
Benjamin Taylor
|
cba6bad69b |
docs(shell-docs): confirm Intelligence use without an unreleased command
The page told the reader to run `npx copilotkit verify` and said it "reports whether the runtime is wired for Intelligence at all". Neither half holds yet: `verify` is not in the published CLI — `latest` is 4.8.3 and the command landed after that tag — and the Intelligence-consumption check it referred to is still in review, so even on `main` the command does not report that. Publishing it would have documented a seam that does not exist, which is the class of defect this change set exists to remove. The dashboard check works today and needs no CLI at all, so it becomes the instruction: send a message and confirm a thread appears. A runtime in SSE mode produces none, whatever the browser showed. The `verify` route can be documented once the command ships with the check in it. |
||
|
Maxim
|
ffbb01be08 |
fix(reskinnable-demo): stop banking's agent in stop-demo.sh
The teardown mirror of the previous commit. `./stop-demo.sh` stopped the
dev server, the docker stack and the native TEI — never :8124 — so
banking's Python agent survived every teardown.
That leftover is not merely litter. `run-demo.sh` health-checks :8124
before starting (so a re-run reuses a live agent instead of colliding on
the port), which means the next cold start silently ADOPTS the orphan and
serves whatever code it was launched with. Edit `agent/`, re-run the
script, observe no change, conclude the edit did nothing.
No `--keep-agent` flag to match `--keep-tei`: TEI has that flag because it
is slow to warm, and the agent boots in seconds, so keeping it would only
reintroduce the failure above.
Also corrects two things in the same breath:
* The Ctrl-C claim I got backwards one commit ago. MEASURED this time,
with the same shell construct the script uses: SIGINT reaches the
foreground process group, which the backgrounded children are still in,
but a NON-INTERACTIVE shell sets background jobs to ignore SIGINT
(POSIX) — so only the exec'd dev server dies (exit=-2) and the stack,
TEI and agent all survive. `nohup` is not what saves them; that covers
SIGHUP, a different signal. Both scripts and the README now say this.
* `ok "docker stack down${PURGE:+ (volumes removed)}"` printed "(volumes
removed)" on EVERY teardown, because the flag holds the string "0" when
unset and `:+` expands on non-empty. The action was always right
(`--volumes` is gated on `-eq 1`) — verified: the postgres/redis/minio
volumes are still there after a flagless run — but the line told anyone
reading it that their seeded data had just been deleted.
Verified with a full cycle through both scripts: `./stop-demo.sh
--keep-tei` reported the agent stopped and left the volumes in place, then
`./run-demo.sh` came back with `banking agent ready (200)` and `stack
healthy`. The idempotency guard was exercised against the live service and
reports "already up" rather than starting a second uvicorn.
Reskin skill: checked, no impact. It documents authoring a skin, not
running the stack; its one launcher line got the note it needed in the
previous commit.
|
||
|
Ben Taylor
|
9df63beeef |
fix(runtime): name useSingleEndpoint when a single-route envelope hits a multi-route runtime (#6579)
Closes [OSS-882](https://linear.app/copilotkit/issue/OSS-882/add-to-existing-journeys-reach-for-the-v1-compat-copilotkit-wrapper). ## The failure The v1-compatible `<CopilotKit>` provider pins `useSingleEndpoint` to `true` ([`copilotkit.tsx:108`](https://github.com/CopilotKit/CopilotKit/blob/main/packages/react-core/src/components/copilot-provider/copilotkit.tsx#L108)), so its startup handshake POSTs `{ method: "info" }` at the base path. A multi-route runtime — the default — matches no route for that path and answered a bare `{"error":"Not found"}`, indistinguishable from a wrong `basePath` or an unmounted handler. Two independent onboarding validation runs hit this on their first browser attempt and each had to guess the cause. Both were *add-to-existing-app* journeys; the greenfield one reached for `CopilotKitProvider` and never saw it. ## What changed **The runtime says what happened.** `detectSingleRouteEnvelope` recognises a POST whose JSON body carries a `method` the single-route endpoint accepts, and the multi-route handler uses it at the one point routing gives up. The 404 now carries a `code` and a message naming the prop, plus a `logger.warn` so it lands in the dev-server terminal too. Deliberately conservative — wrong verb, non-JSON, unknown method, or a JSON POST that isn't an envelope all stay ordinary 404s, unchanged in status and shape. **The client stops discarding it.** All four `/info` callers (two in `agent-registry.ts`, two in `agent.ts`) threw away the response body and reported only the status, so a server-side diagnosis reached nobody. They now go through `runtimeInfoError`, which folds a string `message` from the body into the thrown error. Any future server-side diagnosis reaches the developer for free. **Docs.** Five pages paired a v2 multi-route handler with `<CopilotKit>` and never mentioned the prop. Rather than a warning under a snippet that is still wrong to copy, the snippets themselves now pass `useSingleEndpoint={false}`, with a short callout linking to the provider/handler mapping. Two pages were deliberately left alone: `backend/runtime-endpoints.mdx` already documents the pairing in full, and `cookbook/arcade.mdx` uses `mode: "single-route"` on purpose and already explains it. `backend/copilot-runtime.mdx` keeps its snippet as-is — it pairs with the v1 endpoint, where the default is correct — and gains the caveat only on its "switch to v2 handlers" note. Option 3 in the issue (reconsidering the compat default) is **not** in this PR. ## Testing ### Both halves connect, end to end Real `createCopilotRuntimeHandler` + real `CopilotKitCore` configured the way the v1 wrapper configures it — no mocks on either side: ``` code : runtime_info_fetch_failed message: Runtime info request failed with status 404: Received a single-route request envelope ({ method: "..." }) but this runtime is mounted in multi-route mode, so the request matched no route. If the frontend uses <CopilotKit> from @copilotkit/react-core/v2, pass useSingleEndpoint={false} — that provider defaults it to true. Otherwise mount the runtime with mode: "single-route" to serve this envelope. PASS — the diagnostic reached the client ``` The server-side `logger.warn` fired in the same run, carrying `{ url, path, method: 'info' }`. ### Unit tests `packages/runtime` — `single-route-envelope-diagnostic.test.ts` (2 positive, 5 control): ``` ✓ src/v2/runtime/__tests__/single-route-envelope-diagnostic.test.ts (7 tests) 26ms Tests 7 passed (7) ``` `packages/core` — `runtime-info-error-detail.test.ts` (2 positive, 5 control): ``` ✓ src/__tests__/runtime-info-error-detail.test.ts (7 tests) 267ms Tests 7 passed (7) ``` ### Mutation checks Every new test was verified to fail when its mechanism is broken, in both directions. Detector forced to `return null` — the two positives die, the four controls hold: ``` × names useSingleEndpoint when the envelope is an info call × diagnoses every method the single-route envelope accepts ✓ leaves an ordinary unmatched route as a plain 404 ✓ leaves a JSON POST that is not an envelope as a plain 404 ✓ leaves an unrecognized method name as a plain 404 ✓ does not diagnose a non-JSON POST ``` Detector forced to `return "info"` — the controls die instead, proving they are not vacuous: ``` ✓ names useSingleEndpoint when the envelope is an info call ✓ diagnoses every method the single-route envelope accepts × leaves an ordinary unmatched route as a plain 404 × leaves a JSON POST that is not an envelope as a plain 404 × leaves an unrecognized method name as a plain 404 × does not diagnose a non-JSON POST ``` `runtimeInfoError` with the detail dropped, then with the `typeof message === "string"` guard removed — each kills a different pair: ``` mutation: detail dropped → 2 failed | 5 passed mutation: accept any message field → 2 failed | 5 passed restored → 7 passed ``` ### Full suites, builds, docs | Check | Result | |---|---| | `packages/core` full suite | `Test Files 60 passed (60)` / `Tests 662 passed (662)` | | `packages/runtime` full suite | `Test Files 142 passed (142)` / `Tests 2067 passed (2067)` | | `packages/core` `tsc --noEmit` | clean | | `packages/runtime` `tsdown` | `416 files` — build complete | | MDX compile, 5 edited pages | all `OK` | | pre-commit `nx run-many -t test,publint,attw` | passed across affected projects | | CI on `f94d1ab0` | 72 pass, 3 skipping, 0 fail | Both suites are fully green. An earlier revision of this description reported 6 runtime failures as pre-existing on `main`; they were not. They were artifacts of a worktree whose `node_modules` had been assembled by hand, and a proper `pnpm install` cleared all of them along with the inspector-metadata failures from a stale `@copilotkit/shared` dist. `main` is clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Maxim
|
b96a9449de |
fix(reskinnable-demo): start banking's agent from run-demo.sh
`./run-demo.sh` brought up the embedder, the Intelligence stack and the dev server, then handed over an app whose DEFAULT skin could not answer a single message. Banking's agent is a Python service (`agent/`, :8124) and it is not a compose service, so it had to be launched by hand — and nothing said so: no line in the script, no line in the README, no line in any markdown in this tree. The failure mode is the expensive kind. Nothing errors at startup: the stack comes up healthy, the app boots, the dashboard renders off the REST ledger, every pill is present. Only sending a message fails, and the six in-process skins keep working, so the obvious reading is "my machine is fine, the demo is broken". The script now starts the agent between the compose wait and `pnpm dev`, guarded on `/health` so a re-run reuses a live one instead of colliding on the port, and dies with `(cd agent && uv sync)` when the venv is missing — the same shape as the native-TEI branch above it. Also corrects the Ctrl-C line, which claimed Ctrl-C "stops only the dev server". Measured: the docker stack survives, and the dev server, the native TEI and now the agent all go down with the script. README: the quick start said `pnpm dev` and described OSS mode as needing only `OPENAI_API_KEY`. True for six skins, false for the default one. It now starts the agent too and says why the whole agent lives out of process. Reskin skill: updated, one sentence in the Verification list's step 2. A skin author runs `pnpm dev`, gets redirected from `/` to banking, sends a test message to see if anything works, and gets silence — then debugs their own registration. The skill now points them at `/<their-id>` or `./run-demo.sh`. Verified by doing it: stopped the dev server and the agent, re-ran the script, and it reported `banking agent ready (200)` and `stack healthy` without touching the warm TEI. Then walked beat 3d in the browser — the Q2 pill filed a report citing "the Meridian Creative Agency invoice from page 1", i.e. the model read the attached PDF. |
||
|
Maxim
|
11838db020 |
fix(reskinnable-demo): carry an attached PDF to the model as a file block
Beat 3d was dead on banking: clicking the Q2 pill staged the invoice,
sent the message, and then nothing happened at all — no report, no error,
no failed message in the transcript.
`ag_ui_langgraph` hands every attachment to the model as an `image_url`
block, documents included, so the PDF was rejected before the first
token:
openai.BadRequestError: 400 - Invalid MIME type. Only image types are
supported. (code: invalid_image_format)
The exception is raised inside the model node, which kills the SSE
stream. The runtime sees `RUN_ERROR: terminated` with no cause and the
browser renders nothing. Worse, the crashed run is still checkpointed, so
every LATER message on that thread replays the rejected content and dies
the same way: one click on the pill killed the whole conversation, and
only restarting this service cleared it (`MemorySaver` is in-process).
`_repair_document_attachments` rewrites those blocks into LangChain
standard `file` blocks before the model call. It walks every message, not
just the newest, because the offending content comes back from the
checkpoint on later turns.
STOPGAP, not a design. The real fix is open upstream as
ag-ui-protocol/ag-ui#2476 (both adapters, plus the return leg so a
non-image attachment survives MESSAGES_SNAPSHOT). This service installs
the adapter from PyPI, so that fix cannot reach this venv until it is
published; the middleware's header says when to delete it.
No test: `agent/` has no python test runner, and standing one up for code
whose deletion is already scheduled is the wrong trade. The durable
tests — PDF, audio, video, filename, round-trip, legacy binary — ship
with the upstream PR instead. Verified here by replaying the browser's
real captured run payload against the service: 400 before, RUN_FINISHED
with `createReport` carrying the invoice's line items after.
Reskin skill: checked, no impact. Its beat-3d guidance is entirely the
CLIENT half (staging into the composer, the `AttachmentFailureCause`
union, do not copy `@/shell/attach`), and it names no model-side
conversion. A skin authored from the skill gets a `BuiltInAgent`, whose
converter already maps documents onto file parts — banking is the only
skin whose agent is a LangGraph service, so this failure is unreachable
from the skill's path.
Gates: lint clean, typecheck clean, 2460 unit tests pass. `pnpm build`
deliberately not run — the diff touches no TypeScript, and `next build`
would clobber the `.next` state of the dev server currently serving the
demo.
|
||
|
Benjamin Taylor
|
f36deaaa61 |
fix(docs): keep the useSingleEndpoint guidance mode-aware on multi-mode pages
Two pages document both transport modes and carry a single "point your
frontend at it" snippet serving every front door on the page. Baking
`useSingleEndpoint={false}` into those snippets traded one silent mismatch for
its mirror image: correct for the multi-route majority, wrong for anyone who
followed the `mode: "single-route"` example.
Both now state the rule conditionally next to the snippet instead of asserting
one side of it. Pages with a single handler mode (auth, custom-agent) are
unambiguous and keep the prop inline.
Also pins the one path where the diagnostic could have cost more than it gives:
`clone()` throws once a before-request middleware has drained the body, so the
detector must return null and let the plain 404 stand rather than surfacing a
500. The guard existed; nothing held it in place.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
renovate[bot]
|
76892a03b9 |
chore(deps): update depot/setup-action digest to 91bc849 (#6593)
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [depot/setup-action](https://redirect.github.com/depot/setup-action) ([changelog](https://redirect.github.com/depot/setup-action/compare/15c09a5f77a0840ad4bce955686522a257853461..91bc8495a33ebfc504ffc89e5674379ccf23c29c)) | action | digest | `15c09a5` → `91bc849` | --- ### Configuration 📅 **Schedule**: (in timezone America/Los_Angeles) - Branch creation - "before 9am every weekday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/CopilotKit/CopilotKit). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4zNS40IiwidXBkYXRlZEluVmVyIjoiNDQuMzUuNCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==--> |
||
|
renovate[bot]
|
d4f147aee8 | chore(deps): update depot/setup-action digest to 91bc849 | ||
|
Murat Sari
|
6a36bfda7d |
feat: implement interrupt handling in AgentStore and add injectInterr… (#6538)
## Summary Closes #6509. Expose an interrupt controller directly on Angular’s `AgentStore`: ```ts const store = injectAgentStore("ticketing"); store().interruptController.hasInterrupt(); store().interruptController.resolve({ approved: true }); ``` This keeps messages, state, run status, and pending interrupts on the same conversation-scoped object. Consumers no longer need to separately inject a controller and repeat the agent ID for the common case. ## What changed - Added an eager, readonly `AgentStore.interruptController`. - Bound the controller to the store’s agent and thread. - Destroy the controller when its store is replaced or destroyed. - Made store teardown idempotent and unregister its `DestroyRef` callback. - Prevent stale interrupts from resuming after the agent changes threads. - Kept `injectInterrupt` for typed, filtered, and preprocessed interrupt handling. - Added a convenient agent-selection API matching `injectCapabilities`: ```ts injectInterrupt(); injectInterrupt("ticketing"); injectInterrupt(agentIdSignal); injectInterrupt("ticketing", { enabled, handler }); ``` - Preserved the original options-only form: ```ts injectInterrupt({ agentId: "ticketing", enabled, handler }); ``` - Moved `injectInterrupt` into its own module to avoid an `agent`/`interrupt` circular dependency. - Documented the coexistence limitation between store and specialized controllers. ## Design The store controller is created eagerly with the store. This avoids global coordination, lazy getters, and shared controller registries. Each store owns two local agent observers: 1. Store projection for messages, state, and run status. 2. Interrupt lifecycle observation. These are in-memory observers on the same agent, not additional network or SSE connections. Both are released during teardown. ## Specialized controllers `injectInterrupt` remains the advanced API for: - Typed interrupt payloads. - `enabled` filters. - Asynchronous or synchronous `handler` preprocessing. - Components following an ambient or explicitly selected agent. Controllers do not claim interrupts from each other. If a store controller and specialized controller observe the same agent, both can expose the same decision. Applications should render only one controller for a given decision. ## Testing - Three focused `AgentStore` integration tests. - Two `injectInterrupt` convenience and lifecycle tests. - Stale-thread behavior added to the existing controller lifecycle test. |
||
|
Rainer Hahnekamp
|
bf18677145 | preserve interrupted run IDs when resuming Angular agents | ||
|
Rainer Hahnekamp
|
736742f6f9 | test(angular): document unobservable thread changes | ||
|
Murat Sari
|
ba41a31d7c | feat: implement interrupt handling in AgentStore and add injectInterrupt function | ||
|
JYbill
|
61ac927bae | Merge branch 'main' into fix/state-manager-run-id | ||
|
Maxim
|
0163beab8e |
feat(reskinnable-demo): stream the harness into a CLI console via AG-UI subagents
Makes the offsite-expenses beat legible while it runs, and gives it its own
model, by taking the AG-UI subagent surface from the canary line.
## The expense analyst is now a real subagent
banking gpt-5.4, temp 0
│ banking's prompt; browser frontend tools; Intelligence memory tools
└─ expense-analyst gpt-5.6-sol, reasoning_effort=high
│ sandboxed shell, submit_expense_report
└─ merchant-researcher gpt-5.4, one per merchant, Tavily
Previously the beat was a section of banking's prompt, which left nowhere to put
per-beat configuration: model, effort and recursion limit are all agent-level and
there was one agent. It also meant banking's ~21,000-character rulebook rode
every one of the ~20 model calls the run makes, re-sending rules about markdown
tables while the agent read a CSV.
Reached as a `CompiledSubAgent` because a raw `SubAgent` spec has no `subagents`
field and this one needs its own — the per-merchant fan-out is a headline of the
beat, and a flat subagent could only research serially. Verified nesting
survives: a probe showed the analyst's `task` dispatches, the researchers'
`search_merchant` calls and the final report tool all reaching `astream_events`.
`gpt-5.6-sol` additionally needs `use_responses_api=True`: with function tools
and `reasoning_effort` it 400s on /v1/chat/completions. The first probe missed
that by asking the model a question with NO tools bound — a model probe for an
agent has to bind one.
## The console: one CLI window, streaming
`shell/subagents/subagent-activity.tsx` subscribes to the agent's event stream
and folds it into console lines. Reading `agent.messages` (the previous design)
was wrong twice over: messages materialise at the `MESSAGES_SNAPSHOT`, two per
run, so the pane sat still for minutes and then filled at once; and persisted
messages carry no `subagentRunId`, so the harness's narration could not be told
from banking's own reply.
The fold is pure and idempotent — every line keyed by the id of the thing that
produced it — so the same code serves the live subscription and a replay of the
thread's stored events when a conversation is reopened.
Three heuristics are deleted, each replaced by identity the protocol already
carried:
- the console's "first tool call" anchor -> the run's first `task` call, from
MESSAGE order (durable; the
event-derived version rendered
one console per delegation on a
restored thread — six, measured)
- `CONSOLE_TOOL_NAMES` suppression list -> `subagentRunId` presence
- `disable_streaming` on the researchers -> the canary's per-lane state
`shell/subagents/subagent-message-filter.tsx` keeps subagent narration out of the
conversation. It suppresses the PROSE and keeps the TOOL CALLS: an agent
routinely narrates and calls a tool in one message, and returning null for the
whole message hid the REPORT CARD — the run looked perfect and ended with nothing
to show. Shell-level and inert for a skin whose agent has no subagents.
## Canary stack, contained to this app
The subagent surface only exists on the canary line, and a released
`@ag-ui/client` <= 0.0.57 rejects `SUBAGENT_*` events in the HTTP transport
before any middleware runs, killing the stream. So the demo leaves the root pnpm
workspace and ships its own lockfile, pinning `@copilotkit/* 1.68.3-canary` and
`@ag-ui/* 0.0.59-canary` locally instead of imposing an unreleased protocol on
every package in the monorepo.
A 1.62.2-based canary was tried first and could not compile the app: it silently
rewound the CopilotKit API five minors under a demo written against 1.67.1, and
`OpenGenerativeUIActivityRenderer` (a public `/v2` export since ~1.63) was the
first thing to break. `workspace:*` is not a version, so the app had no recorded
lower bound on the API it needs.
KNOWN GAP, deliberately not fixed here: Nx discovers projects THROUGH the pnpm
workspace (there is no `workspaceLayout` in `nx.json`), so leaving it also
removes the demo from the repo-wide `nx run-many -t build` and `-t check-types`
sweeps. Verified — `nx show project deep-agents` and the other standalone
showcases return "Could not find project". No workflow names this demo, so it is
currently unbuilt and untype-checked in CI and needs its own job. Run the four
gates locally until that lands. Documented in `pnpm-workspace.yaml`.
## Fixes
- The run clock is keyed per run and read through an injected `ToolRuntime`
instead of taking the oldest open stamp across the process. Model calls AFTER
the report re-stamped the clock and that leftover became the next run's start:
a two-minute run reported 333s. Now 86s reported against 98s wall clock — the
gap is thread-naming and delegation, before the analyst's first model call,
which is what the tile claims to measure.
- `merchantKind` non-answers are rejected on the leading token, on hedging
language, and over 40 characters. With no search tool the model wrote a bare
"unclear"; with Tavily live it hedges in prose ("unknown (likely
bookbindery/bookshop retail, but not established for this exact merchant)"),
which an exact-match filter passed into a 60-character label glued to the
merchant name.
- `vitest` no longer externalises `@copilotkit/*`. Installing them from npm moved
`src/app/layout.tsx`'s stylesheet import under `node_modules/.pnpm/`, where
Node's ESM loader threw `Unknown file extension ".css"` and took out 16 suites
while naming a stylesheet nobody had touched.
- `agent/main.py` reads the demo's `.env` as well as its own, so `TAVILY_API_KEY`
works wherever an operator puts it. Two env files to keep in sync is a trap
whose failure mode is "the agent ignores a key that is plainly sitting in .env".
## Upstream finding (reported separately, not fixed here)
`@copilotkit/runtime` drops `subagentRunId` when persisting messages: 2888 of
3026 stream events carry it, 0 of 53 persisted messages do. Reproduced with
Intelligence removed entirely, so it is the runtime's message shape rather than
the platform store — and `@copilotkit/runtime`'s dist contains no occurrence of
the field at all, while `@ag-ui/core`, `ag-ui-protocol` and `@copilotkit/core`
all model it. One field threaded through would let the console rebuild from
message history and delete the event-replay seeding added here.
`CLAUDE.md`'s appended block is generated by `next dev`
(`next/dist/server/lib/generate-agent-files.js`) and committed per its own
instruction to keep the tree clean.
Gates: lint 0, typecheck 0, test:unit 2460 passed across 216 files, build 0.
Measured end to end in Intelligence mode: 14 rows, 9 merchants researched, 6
charges filed with ids read out of real 201 bodies, totals reconciling against
their own rows, 3220 events with 6 SUBAGENT_STARTED/FINISHED pairs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JRubZT6AS6LCGkcE2KzcfA
|
||
|
Mark
|
85af4bedfb |
fix(showcase): gen-ui-tool-based uses shared pie-chart contract for all slugs (#6587)
## The bug (shared-probe-rule violation)
`showcase/harness/src/probes/scripts/d5-gen-ui-custom.ts` branched on
`integrationSlug` via a stale `CHART_INTEGRATIONS` allowlist: only ~5
slugs (langgraph-python, ms-agent-python, spring-ai, google-adk, mastra)
got the pie-chart prompt + assertions. Every other `gen-ui-tool-based`
slug got an **obsolete** `generate_haiku` prompt + `HaikuCard`
assertion.
This violates Showcase **iron rule 1** (one shared probe, no per-slug
`if slug === …` branch in the test) and no longer matches the product:
- All **21** `gen-ui-tool-based` pages register `render_bar_chart` +
`render_pie_chart` via `useComponent`.
- Every committed D6 `render-a2ui.json` fixture already carries the
chart exchange.
- On staging the non-allowlisted cells failed: the assistant container
renders but the `HaikuCard` assertion reports "rendered but has no text
content". Routing the cell through the pie-chart path greens it (SVG
renders, pie validated, narration appears).
## The change (probe only)
- Every `gen-ui-tool-based` integration now sends
`PIE_CHART_USER_MESSAGE` ("Show me a pie chart of revenue by category").
- The pie-chart SVG shape assertions + second-leg narration token check
run for **every** slug.
- Removed the slug-dependent `CHART_INTEGRATIONS` set /
`isChartIntegration` branch.
- Removed the obsolete haiku prompt + `HaikuCard` fallback (verified no
other usage anywhere in `showcase/harness/src`).
No allowlist was substituted with a larger allowlist. **No** fixture
re-record, backend, frontend, npm, or aimock change. Net `-248` lines
across the probe + its unit test.
## Red → Green (local proof)
The unit test was updated **before** the implementation so a
formerly-non-allowlisted slug (`ms-agent-dotnet`) fails pre-fix.
**RED (test updated, old implementation):** 6 failed / 2 passed
```
× buildTurns sends the pie chart message for a formerly-non-allowlisted slug (ms-agent-dotnet)
→ expected 'Write me a haiku about nature' to be 'Show me a pie chart of revenue by category'
× NO slug selects a different probe contract — every gen-ui-tool-based slug sends the pie chart message
→ slug ms-agent-dotnet must send the shared pie chart message: expected 'Write me a haiku about nature' to be 'Show me a pie chart of revenue by category'
× pie chart: assertion FAILS when the rendered component has no <svg> (ms-agent-dotnet)
→ got 'gen-ui-custom: matched cascade selector … but no haiku card or rendered component found in DOM'
× pie chart: assertion FAILS when SVG has too few drawing children (pydantic-ai)
× pie chart: assertion FAILS when assistant follow-up is missing expected tokens (ms-agent-dotnet)
× pie chart: assertion PASSES on a healthy donut render with full narration (ms-agent-dotnet)
```
**GREEN (after implementation):** 8 passed / 8
```
✓ src/probes/scripts/d5-gen-ui-custom.test.ts (8 tests) 23ms
Test Files 1 passed (1)
Tests 8 passed (8)
```
**Mutation-verified:** setting `PIE_CHART_USER_MESSAGE = "MUTANT"` fails
the two contract tests; disabling the narration token check (`if
(false)`) fails the missing-tokens test. Clean restore confirmed by
diff.
## Command results (from a fresh worktree off origin/main)
| Command | Result |
| --- | --- |
| `nx run @copilotkit/showcase-harness:test` (target file) | ✅ 8 passed
|
| `nx run @copilotkit/showcase-harness:test` (full) | ✅ 177 files / 3721
tests passed, 2 skipped, 0 failed |
| `nx run @copilotkit/showcase-harness:typecheck` | ✅ clean |
| `nx run @copilotkit/showcase-harness:build` | ✅ success |
| `showcase/bin/showcase fixtures validate` | ✅ exit 0, "All fixtures
valid" |
> Note: the full suite / typecheck initially reported failures in
`frontend-matrix.test.ts` and `d0-gone-predicate.test.ts`. These are
**pre-existing** and depend on gitignored generated artifacts
(`frontend-catalog.json`, `registry.json`) absent from a fresh worktree
— confirmed by reproducing them on pristine `origin/main` with my
changes stashed. After running the repo's `generate-registry` step they
pass. Neither file references gen-ui-custom.
## Value tests — HANDED OFF (control-plane unreachable here)
The production-shaped control-plane value tests must run **without**
`--direct`. Docker is not reachable in my environment (`docker info` /
`docker ps` exit 1; `--isolate` fails at `docker compose … ps`), so I
did **not** run them and did **not** substitute `--direct`. Please run
live:
```
showcase/bin/showcase test ms-agent-dotnet:gen-ui-tool-based --d5 --isolate --verbose
showcase/bin/showcase test pydantic-ai:gen-ui-tool-based --d5 --isolate --verbose
showcase/bin/showcase test langgraph-typescript:gen-ui-tool-based --d5 --isolate --verbose
showcase/bin/showcase test langgraph-python:gen-ui-tool-based --d5 --isolate --verbose # control
```
Expect: RED on the formerly-haiku cells (ms-agent-dotnet, pydantic-ai,
langgraph-typescript) before this change, GREEN after; langgraph-python
stays GREEN.
## Rollout
Needs a harness / control-plane deploy + a D5 sweep rerun to flip the
affected `gen-ui-tool-based` cells. **No** npm publish, backend, aimock,
or fixture re-record required.
Refs the `gen-ui-tool-based` shared-probe contract. Separate/unrelated:
LlamaIndex `done-signal-missing` (not touched here).
|
||
|
Jordan Ritter
|
8f4adc9d1a |
fix(showcase): gen-ui-tool-based uses shared pie-chart contract for all slugs
The D5 gen-ui-custom probe branched on integrationSlug via a stale CHART_INTEGRATIONS allowlist: ~5 slugs got the pie-chart prompt + assertions, everyone else got an obsolete generate_haiku prompt + HaikuCard assertion. That violates Showcase iron rule 1 (one shared probe, no per-slug branching in the test) and no longer matches the product — all 21 gen-ui-tool-based pages register render_bar_chart + render_pie_chart, and every committed D6 render-a2ui fixture carries the chart exchange. Collapse to the single shared contract: every integration sends "Show me a pie chart of revenue by category" and runs the SVG/pie-chart shape + second-leg narration assertions. Remove the CHART_INTEGRATIONS allowlist / isChartIntegration branch and the now-unused haiku prompt + HaikuCard fallback (verified no other usage). No fixture, backend, frontend, npm, or aimock changes. |
