Semantically-identical reformat of `renovate.json` (compact single-line,
same content) to nudge Mend/Renovate to re-evaluate against the NEW
github-actions-only central preset.
Background: Renovate hasn't re-scanned since 2026-06-16 due to the
cached-old-preset gotcha, so no Dependency Dashboard has appeared yet
after the onboarding cutover (#5031). A no-op touch of the consumer
config forces re-evaluation.
Content is unchanged:
```json
{"$schema":"https://docs.renovatebot.com/renovate-schema.json","extends":["local>CopilotKit/renovate"]}
```
No behavior change; npm remains human-controlled.
Migrates this repo's Renovate configuration to extend the org-wide
central config at https://github.com/CopilotKit/renovate.
Phase 1 of the migration ([Notion
plan](https://www.notion.so/3613aa38185281a38863fcff2907021c)) scopes
Renovate to the github-actions ecosystem only; npm/pip remain on
Dependabot.
Replaces the previous renovate.json (which had `extends:
config:recommended` plus a global "ignore all packages initially" rule
that effectively disabled the prior Renovate install). The new config
inherits from the org-wide central preset, which is scoped to
github-actions for Phase 1.
Pre-existing open Renovate PRs (#4751, #3295) from the prior install can
be closed separately once this lands.
## Root Cause
`agno 2.6.20` removed `agno.os.interfaces.agui.utils`. The floating
`agno>=2.5.17` pin in `requirements.txt` caused staging to pull the
breaking version on the next build, causing a startup failure.
## Red-Green Proof
**RED** — with `agno>=2.6.20` installed:
```
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'agno.os.interfaces.agui.utils'
```
**GREEN** — with `agno==2.6.19` installed:
```
GREEN: all 3 symbols OK
```
(Symbols confirmed: `async_stream_agno_response_as_agui_events`,
`extract_agui_user_input`, `validate_agui_state`)
## Changes
- `showcase/integrations/agno/requirements.txt`: pinned `agno>=2.5.17` →
`agno==2.6.19` (exact pin, last version with `agui.utils`)
- `showcase/integrations/agno/src/agent_server.py`: added TODO comment
at line 73 import site noting migration to agno 2.6.20+ API is a
follow-up; no structural changes to imports
## Follow-up
Migration of `agent_server.py` imports to the agno 2.6.20+ API (once the
replacement for `agui.utils` is identified) is tracked in the TODO
comment at line 73.
## Note on CI
The `validate-pins` CI check will likely flag pre-existing non-exact
pins across ~15 other integrations (`openai ^5.9.0`, `crewai` ranges,
etc.). This is pre-existing debt not introduced by this PR.
The threads-list internal-scroll fix (grid-template-rows: minmax(0,1fr)) for the
integration examples is pulled out of the drawer-redesign PR. It ships in a
separate PR (branch ben1/ent-1051-example-grid-followup) once the redesigned
drawer packages are released, alongside the examples' dependency bumps to the
new versions.
## What & why
Resolves [OSS-132](https://linear.app/copilotkit/issue/OSS-132).
Investigated with systematic-debugging; every conclusion verified
against the **real** OpenAI Responses API.
**Net change: a TanStack version bump only.** No showcase schema change.
- `@tanstack/ai` `0.18.0` → `0.35.0`
- `@tanstack/ai-openai` `0.9.1` → `0.15.6`
- `package-lock.json` regenerated (Dockerfile uses `npm ci
--legacy-peer-deps`)
## The bug
The built-in-agent showcase 400s on every prompt against real OpenAI.
The state tools (`AGUISendStateSnapshot` / `AGUISendStateDelta` /
`set_steps`) declare arbitrary payloads as `z.any()`, which serializes
to a **typeless** JSON-Schema property (`{ "description": ... }`, no
`"type"`).
The old `@tanstack/openai-base`'s `isStrictModeCompatible()` only
screened for `oneOf/allOf/not/$ref/$defs`, so it missed the missing
`type`, sent the tool with `strict: true`, and OpenAI rejected it:
```
400 Invalid schema for function 'AGUISendStateSnapshot':
In context=('properties','snapshot'), schema must have a 'type' key.
```
This was **masked in production** because the deployed showcase runs
against aimock, which replays fixtures without validating the request
schema — a raw `curl` to prod returns a clean `RUN_FINISHED`, green for
the wrong reason.
The ticket's original framing (zod3/zod4 drift → typeless *root*, `got
"None"`) was already fixed by the zod-4 migration; this is the same
symptom one layer down (typeless *property*).
## The fix is upstream
`@tanstack/ai-openai@0.15.6` (via `@tanstack/openai-base@0.9.2`) fixes
`isStrictModeCompatible`: it now detects typeless / `z.any()` properties
and sends `strict: false`. OpenAI accepts typeless properties under
`strict: false` — so `z.any()` works again with no schema change on our
side.
(`@tanstack/ai-openai@0.15.5` also dropped `@tanstack/ai-client` from
its peerDependencies, so no `ai-client` dep is added.)
## Verification (real OpenAI, gpt-4o)
| Probe | Result |
|---|---|
| Typeless property, `strict: true` (raw OpenAI) | **400** — `schema
must have a 'type' key` |
| Typeless property, `strict: false` (raw OpenAI) | **ACCEPTED** —
confirms it was the strict flag, not the schema |
| `z.any()` tool on old adapter (0.9.1/0.15.4) | adapter sends `strict:
true` → **400** |
| `z.any()` tool on new adapter (0.15.6) | adapter sends **`strict:
false`** → **ACCEPTED**, model calls the tool |
| All 3 `z.any()` state tools attached, new adapter | **ACCEPTED**, no
400 |
## Not covered here
The showcase's aimock + Playwright e2e suite was **not** run locally
(this worktree has no installed toolchain). CI runs it on this PR;
please confirm the gen-ui / shared-state demos still pass before merge.
---
_Branch history shows an interim `z.string()` workaround that was
reverted once the upstream fix shipped; the net diff is the version bump
only. Squash-merge recommended._
One-line fix: `bin/railway`'s `run_staging_probe` invoked `npx --yes tsx
verify-deploy.ts` from the repo root with no `chdir`, so under Node 22
tsx failed to resolve (MODULE_NOT_FOUND in the ESM preload) → the
promoter misread it as 'staging not green' → hard REFUSE. This
tier-gated all prod promotes (incl. the langgraph fix in #5825). Fix
adds `chdir: File.expand_path("../scripts", __dir__)` so tsx resolves
from `showcase/scripts/node_modules`.
Red-green: from repo root `npm ls tsx` is empty and `npx tsx` crashes;
from showcase/scripts it resolves (tsx declared in
showcase/scripts/package.json).
NOTE: the agno `<2.6.20` pin (originally bundled here) was split out —
it edits a requirements file which trips the fleet-wide validate-pins
ratchet (pre-existing non-exact-pin debt across ~15 integrations).
Tracking separately.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
agno 2.6.20 removed agno.os.interfaces.agui.utils; the floating
agno>=2.5.17 pin in requirements.txt caused staging to pull the
breaking version. Pinned to 2.6.19 (last version with the module).
Added TODO comment at the import site for future migration.
Without chdir, npx resolves tsx from the repo root where it is not installed.
tsx is a dev dependency of showcase/scripts; chdir ensures npx resolves it correctly.
Add grid-template-rows: minmax(0, 1fr) to the drawer-hosting grid layout in the integration examples that were missing it, mirroring the langgraph-js fix. Without a bounded row the single grid track sizes to content, so a long thread list makes the drawer grow past the viewport (page scrolls, header scrolls away) instead of scrolling internally with the header pinned.
Examples touched:
- a2a-middleware
- adk
- agno
- crewai-crews
- crewai-flows
- langgraph-python
- llamaindex
- mastra
- mcp-apps
- ms-agent-framework-dotnet
- ms-agent-framework-python
- pydantic-ai
- strands-python
The confirm-delete overlay was a CSS-positioned div trapped in the drawer
host's stacking context, so it could paint under other UI (e.g. the chat's
welcome view). Render it as a native <dialog> opened with showModal(), which
lives in the browser top layer and can never be occluded. The <dialog> is
always present so updated() can drive showModal()/close() from
_confirmingDeleteId; jsdom implements neither method, so a feature-detect falls
back to toggling the `open` attribute (which HTMLDialogElement.open reflects)
to keep unit tests observing open/closed state. Native Escape is handled via
the dialog's `cancel` event (the manual confirm-Escape branch in _onKeyDown is
removed); backdrop clicks dismiss via a target===currentTarget check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## What & why
Showcase services were being killed on Railway. Root causes, all fixed
here:
1. **langgraph-python / langgraph-fastapi — watchfiles log flood →
Railway 500-logs/sec replica kill.** `langgraph dev` ran with
hot-reload, emitting "1 change detected" per request; under D6 probe
fan-out this blew past Railway's 500 logs/sec cap and killed the
replica. Fix: `--no-reload` + `export
LANGGRAPH_DISABLE_FILE_PERSISTENCE=true` (also stops unbounded
pickle-state OOM).
2. **langgraph-typescript — `FileSystemPersistence` RangeError crash
loop.** `@langchain/langgraph-api` serialized unbounded thread state via
`JSON.stringify`; past V8's ~512MB string ceiling it threw `RangeError`
in a timer, hung the event loop, and the watchdog kill-looped (state
persisted on disk, so restarts re-crashed). Fix: boot-purge stale state
+ a **size-gated** restart (checks dir size, only restarts near the
ceiling — no in-flight-wiping timer, no unpinned `/internal/truncate`).
3 & 4. **Per-request proxy log flood across all integrations.**
`[copilotkit/route] POST` + `Response status` logged on every
sub-request, unconditionally, in 19 `route.ts`. Fix: gate them behind
`SHOWCASE_ROUTE_DEBUG` (off in prod) — **but keep non-2xx responses
logged unconditionally** so production errors stay visible, and gate the
health-probe GET too.
## Verification
- Every fix carries local red-green. langgraph-typescript entrypoint:
**18 mutation-sensitive subprocess tests** (reversed comparison / broken
du|awk / wrong-kill-target all caught; orphan-cleanup reaped). route.ts
gating verified on the real Next.js surface across ≥3 integrations
(non-2xx logged, 2xx+health gated, `SHOWCASE_ROUTE_DEBUG=1` restores
verbose).
- Code review: Round 1 (7 agents) → fixes → Round 2 (7-agent
confirmation) → fix → Round 3 (3-lens targeted) → fix → converged to
zero mandatory findings.
## ⚠ Before merge
The two entrypoint changes (`--no-reload` +
`LANGGRAPH_DISABLE_FILE_PERSISTENCE` on pinned `langgraph-cli 0.4.21`)
are **source-verified but could not be run locally** (the langgraph
packages are on a private index; `0.4.21`'s `--no-reload` was confirmed
only in public `0.4.3`). **Requires live-Railway validation** (branch
deploy: boots, serves 200, no watchfiles spam, no pickle files) before
merge. Kept as a **draft** until validated and the maintainer approves.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The thread panel is a persistent always-visible sidebar on desktop; the Figma
"closed" mockup is the MOBILE state, already covered by off-canvas behavior.
- web-components: remove the `collapsed`/`collapsible` properties,
`_toggleCollapsed`, the header collapse-toggle button, and the
collapsed-cluster render branch; render() always paints the full panel body.
Gate the now control-less header on a `_hasHeader` slotchange flag so no empty
bar renders. Drop the unused `iconSidebar`, the `CollapseChangeDetail` type +
`collapse-change` event-map entry, the index re-export, and the
`.root.collapsed`/`.collapsed-cluster` styles.
- react: drop the `collapsible` prop + property assignment, the
`onCollapseChange` prop + `collapse-change` listener/handler, and the local
`CollapseChangeDetail` type.
- vue: drop the `collapsible` prop + element binding, the `collapse-change`
emit + `@collapse-change` handler, and the local `CollapseChangeDetail` type.
- angular: drop the `collapsible` input + property push, the `collapseChange`
@Output + event wiring (and now-unused EventEmitter/Output imports), and the
local `CollapseChangeDetail` type.
- tests: remove all collapse tests across the four packages; add an element
header-gating test. Mobile off-canvas (open-driven) behavior is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gates per-request POST + 2xx Response-status + GET health-probe logs behind SHOWCASE_ROUTE_DEBUG across 19 integrations to stay under Railway's 500-logs/sec cap, while logging non-2xx responses unconditionally so production errors stay visible.
Boot-purge of stale .langgraph_api state plus a size-gated restart (du > threshold -> kill agent -> container restart -> purge), replacing an in-flight-wiping periodic truncate loop. Adds mutation-sensitive subprocess tests for the watchdog.
Also fixes the collapsible-default test assertion (element defaults collapsible=true,
mirroring licensed; the prior undefined assertion only passed vs a stale dist).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Element-only; downstream wrappers' search removal lands in the paired commit.
Verified in isolation: @copilotkit/web-components test (86) + build green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror the existing recentLabel (passthrough) and onSearch (element-event)
props with two additive props targeting the element's forthcoming
`collapsible` property and `collapse-change` event.
- react: add `collapsible?: boolean` (pushed as an element PROPERTY, like the
default-true boolean `licensed`) and `onCollapseChange?: (collapsed) => void`
(wired via the handler-ref addEventListener block, like onSearch).
- vue: add `collapsible?: boolean` (imperative property push in the
watchEffect, like `licensed`) and re-emit the element's `collapse-change`
event as `collapse-change(collapsed)` (matching the `search` emit convention).
- angular: add a `collapsible` signal input (property push in the effect, like
`licensed`) and `@Output() collapseChange = new EventEmitter<boolean>()`
wired from the element's `collapse-change` event (like `search`).
CollapseChangeDetail is declared locally in each wrapper with a TODO to switch
to the package export once the parallel element PR that adds the collapse
feature lands and is published (the built element types in this worktree
predate it).
Testing: added mirrored tests per framework (property-set + event-passthrough);
full nx test suites green (react-core 1420, vue 1070, angular 178); check-types
and build green for all three packages.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Search: the magnifier now expands an inline search input in the header icon
row and focuses it; it auto-collapses back to the icon on blur when the query
is empty/whitespace, but stays open while a non-empty query is active.
Toggle-click and Escape still close + clear + emit search{query:''}.
- Remove all clipped-name tooltip machinery (data-tooltip attr,
_syncNameClipping + its updated() call, name-clipped toggling, the tooltip
::after/::before CSS, and the name-clipped:hover z-index lift). Ellipsis
truncation is kept.
- Add reactive collapsible property (default true, mirrors licensed): when
false the header omits the collapse toggle and the collapsed cluster never
renders (drawer stays expanded). Mobile off-canvas is unaffected.
- Add collapse-change event + CollapseChangeDetail (types + index re-export);
the collapse toggle and cluster-expand route through _toggleCollapsed(), which
flips collapsed and emits collapse-change with the new value (user-driven only).
## What this does
Re-platforms the banking showcase's self-learning off the **abandoned**
offline-distill path (which targeted the now-closed Intelligence #192
`record → /annotate → sl-worker → /knowledge` pipeline) onto the
**shipped** memory substrate (`libs/memory`, Intelligence #294/#321).
The agent now saves a demonstrated over-limit procedure as a
`project`-scoped, `procedural` memory via `save_memory`, and
`recall_memory`s it at the start of later over-limit requests — so a
**fresh thread, or a different user on the same team, completes the
approval unaided**. That's the FOR-149 durable cross-thread + cross-user
proof.
## Verified live (local stack)
- Vendored memory-enabled Intelligence stack comes up healthy; `POST
/api/memories` → `201`, `/recall` → `200`, and
`save_memory`/`recall_memory`/`forget_memory` MCP tools attach
(`SL_ENABLED` + embedder).
- Cross-user: a project memory saved by one user recalls for a different
user.
- App boots in Intelligence mode; OSS fallback (`InMemoryAgentRunner`)
untouched and still the default.
## Changes
- **`docker-compose.yml`** — vendored stack cloned from the proven
`memory-chat` recipe (postgres/pgvector, redis, minio, TEI, composite
app-api + gateway). Hardened during a real bring-up: `minio-init`
DNS-race retry, **pluggable embedder** (`MEMORY_EMBEDDINGS_URL` + `tei`
dependency `required:false`, so RAM-constrained / Apple-Silicon machines
can point at a host TEI), and non-colliding `715x` host ports.
- **Runtime** (`route.ts`) — Intelligence branch gains `licenseToken` +
lock config + `generateThreadNames`; recall-first / save-on-teach
prompt; `recall_memory`/`save_memory` added to the tool list.
- **`saveLearnedWorkflow`** resolves a `status: saved` result that
drives the agent's `save_memory` call (Option A — agent-initiated),
keeping the already-approved guard.
- **Removed** the dead `record-user-action` `/annotate` seam (kept the
visual `useRecording` UX).
- **README** rewritten: one-command stack, host-TEI override, ports,
`.env`, cross-thread + cross-persona walkthrough, testing notes. Adds
`.env.example`.
## Tests
- **Deterministic E2E (CI gate):** `e2e/memory-learning.spec.ts` +
aimock fixtures — agent LLM served by `@copilotkit/aimock` (fixtured
`recall_memory` → exception → approve tool calls) against the **real**
local memory backend; asserts a fresh thread unlocks from recalled
memory with no recording offer.
- **Real-LLM drift smoke (manual, non-gating):**
`scripts/memory-drift-smoke.mjs`.
## ⚠️ Why draft — needs a green E2E run
The Task 7 E2E is **authored + statically validated** (`playwright test
--list` compiles spec + config; fixtures/JSON/launcher all valid) but
**has not had a green run yet** — it needs `@copilotkit/aimock`
installed, the docker stack up, and the dev server in Intelligence mode
(a 4-process orchestration). Each E2E file carries a `VERIFY ON FIRST
GREEN RUN` checklist (aimock fixture schema/launch API, chat + HITL
selectors, the `sequenceIndex` ordering key). Marking draft until that
passes.
## Out of scope (deferred)
- Per-run demo reset for a repeatable public embed (user-scope memory /
periodic DB reset / dashboard control).
- Managed-Intelligence target: PRD/handoff prefer
`api.intelligence.copilotkit.ai`; this PR ships the local-vendored stack
per direction. Reconciling for the V1 website/Railway deploy is a
follow-up.
- Pre-existing demo `tsc` looseness (`page.tsx`, `copilot-context.tsx`)
— untouched.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---
## Update — booth-bundle pass (2026-06-30)
Follow-up to make the demo booth-ready and reproducible from the
CopilotKit repo by teammates. Four commits on top of the above:
- **`/api/v1/dev/reset` now clears durable memory**, not just the
transaction store — so the full *fail → teach → succeed* arc replays for
each booth visitor. New scope-complete `forgetAllMemories` helper
enumerates via a bare `GET /api/memories` (the backend `400`s on
`?scope=` filters, so a single bare GET is inherently scope-complete)
and `DELETE`s each id; the route returns
`{ok,reset:["store","memory"],forgot:N}`, or a `502` on partial failure
so a half-reset state is never silently used. Live-validated
(`forgot:2`). **This resolves the "per-run demo reset" item listed as
deferred above.**
- **Memory `kind` migrated `operational` → `procedural`** to match the
Intelligence demo branch's current schema (`semantic | episodic |
procedural`). *(Supersedes the "operational" wording earlier in this
description.)*
- **Fixed the aimock E2E launcher** — `new LLMock({ fixtures })` ignores
`options.fixtures`, so the mock was serving 0 fixtures; now registers
via `addFixtures()`.
- **E2E status:** `test:unit` green; `test:self-learning` — the Glass
Engine inspector test passes; the autonomous-recall test has a known
aimock fixture-sequencing flake (harness-only, not a demo/backend bug).
The booth relies on the manual real-LLM arc.
**Build the Intelligence backend from `david/for-162-splat-demo`, not
`main`.** Verified by building both: the demo branch boots healthy and
runs the arc; `main` crash-loops with this compose — it requires the new
`INTELLIGENCE_DEPLOYMENT_MODE=self_hosted` auth contract (rejecting the
`DEPLOYMENT_MODE` + `DEFAULT_ORGANIZATION_ID` env this compose sets) and
its memory `kind` vocabulary is `topical/episodic/operational`.
Targeting `main` is a separate migration (compose auth env + org-seed
model + `kind` taxonomy). A full local-setup runbook exists for
teammates (internal Notion).
---
## Update — CR pass (2026-07-03)
A 7-agent review-and-fix loop converged (2 rounds + a bucket-(c)
promotion audit; 0 mandatory findings remaining). Four fixes landed,
each its own commit; `tsc`, unit tests (41/41), eslint, and `next build`
all green:
- **Recorder feed** — `handleApprove` in `transactions-list.tsx` and
`pending-approvals-chat.tsx` called `logStep()` *before*
`beginRecording()`, so the "Approved the charge" line was silently
dropped (`logStep` no-ops when inactive; `beginRecording` then resets
the feed). Reordered to `beginRecording → logStep → endRecording`; added
`recording-context.test.tsx` with red-green coverage.
- **Docs** — corrected the memory-learning spec path `tests/e2e/` →
`e2e/` (README, `.env.example`, smoke script), and the
top-of-description memory `kind` `operational` → `procedural`.
- **docker-compose header** — infra host-port comments corrected
`705x/706x` → the actual `715x/716x` mappings.
Deferred (pre-existing, out of this PR's subject; candidates for a
follow-up): the dual/divergent "current page" agent readable
(`copilot-context.tsx:96` vs `layout.tsx:147`), and the `PUT
/api/v1/transactions/[id]` error-swallow returning `undefined`.
---
## Update — migration to Intelligence `main` + presenter reset +
Apple-Silicon fresh-setup (2026-07-06)
This branch now targets Intelligence **`main`** (the earlier sections
assumed the `david/for-162-splat-demo` branch). Changes on top of the
above:
**Migration to `main`'s contract**
- **Compose auth:** `INTELLIGENCE_DEPLOYMENT_MODE=self_hosted` (legacy
`DEPLOYMENT_MODE` / `DEFAULT_ORGANIZATION_ID` removed — `main`'s
`loadAuthEnv` rejects them).
- **Memory `kind` renamed `semantic|procedural` →
`topical|operational`** to match `main`'s closed enum (`topical |
episodic | operational`). ⚠️ *This supersedes the earlier "migrated
operational → procedural" note (that was for the old branch): the
over-limit procedure is now **`operational`**, general facts
**`topical`**.*
- **Self-hosted memory is license-gated on `main`.** New
`scripts/mint-dev-license.mjs` (`pnpm mint-dev-license --write`) signs
an enterprise dev license (`features.memory=true`) with a throwaway key
and bakes the public half via `BAKED_LICENSE_KEYS_JSON`, which the local
(unbaked) app-api trusts. Drives the signer from the private
Intelligence source via `INTELLIGENCE_REPO` — no signing code vendored
into this public repo. Managed-Intelligence users instead supply a
CopilotKit-issued token and omit the baked key.
**Presenter reset button** (finishes the deferred "per-run demo reset",
now UI-driven)
- New `PRESENTER_RESET_ENABLED` flag gates **both** a sidebar reset
button **and** the `/api/v1/dev/reset` endpoint (403/hidden by default —
safe-off for public hosts).
- Full clean slate: re-seeds transactions + forgets memory for **both**
seeded personas (`SEEDED_USER_IDS`), with partial-progress reporting on
a mid-clear failure. TDD; spec + code-quality reviewed.
**Apple-Silicon fresh-setup fix**
- The bundled amd64 `tei` crash-loops under arm64 emulation (Candle
backend unavailable → ONNX/ORT backend → 404 on ONNX files
`Qwen3-Embedding-0.6B` doesn't publish). Gated it behind the
`cpu-fallback` profile (a bare `up` skips it), and added `run-demo.sh`
that runs a native Metal TEI on Apple Silicon (same 1.9.3 + model →
byte-identical embeddings) and the docker `tei` on amd64/CI. README
diagnosis corrected (was mis-attributed to OOM).
**Verification:** 55 unit tests green, `tsc` clean, `eslint` clean. A
from-scratch run (Intelligence `main` rebuild + clean `pnpm install` +
native TEI) was validated end-to-end — memory save/recall through the
native embedder, teach→recall arc, and reset all working. The
deterministic aimock e2e still has the known fixture-sequencing flake
(see follow-up comment below).
## Problem — the leak
The v2 runtime's `shouldForwardHeader` forwarded `authorization` **and
any header whose name starts with `x-`** onto the outgoing agent call.
In a real deployment the inbound request has already traversed a
browser, CDN/edge, load balancer, and hosting platform — each stamping
its own `x-*` headers — so the wide `x-*` wildcard silently forwarded:
- **Hop-by-hop / topology:** `x-forwarded-for`, `x-real-ip`,
`x-forwarded-proto/host/port`
- **Cloud / CDN tracing:** `x-amzn-trace-id`, `x-amz-cf-id`,
`x-cloud-trace-context`, `x-azure-*`, `x-fastly-*`, `x-request-id`
- **Platform-injected:** `x-vercel-*`, `x-middleware-*`
- **CopilotKit Cloud platform credential:**
`x-copilotcloud-public-api-key`
The last item is a real credential-exfiltration concern: a platform key
scoped to Copilot Cloud reaching a third-party agent URL. This is the
**breadth** half of #5712 (option 3); the **precedence** half was fixed
in #5782.
## Design — denylist default + config knob, both paths
- **Default denylist (safe default).** Keep the `authorization` + `x-*`
base eligibility, but strip a curated, greppable set of known
infra/proxy/platform headers (exact names + prefix families) before
forwarding. Legitimate custom `x-*` application headers (`x-tenant-id`,
`x-api-key`, …) keep flowing untouched. The authoritative list is a
single exported constant in `header-utils.ts`.
- **Configurable policy (`forwardHeaders` runtime option).**
- `useDefaultDenylist?: boolean` (default **true**) — `false` restores
the previous wide-open behavior.
- `deny?` / `denyPrefixes?` — extend the default denylist.
- `allow?` — opt into strict allowlist mode (only listed headers
forward).
- **Resolve once.** The constructor resolves `forwardHeaders` into a
`forwardHeadersPolicy: ResolvedForwardHeadersPolicy` field (mirroring
the existing `debug` → `ResolvedDebugConfig` resolve-once), exposed on
`CopilotRuntimeLike` / `BaseCopilotRuntime` with a passthrough getter on
the `CopilotRuntime` shim.
- **Both paths.** The resolved policy is read at **/run**
(`configureAgentForRequest`) and **/connect** (`handleSseConnect`) via
`mergeForwardableHeaders`, so the two can never diverge. Server-wins
precedence and server-self case-dedup from #5782 are untouched.
## Semver
**Minor with an opt-out.** Removing a leak is a fix, not a contract
change, and we ship a documented escape hatch: `new CopilotRuntime({
agents, forwardHeaders: { useDefaultDenylist: false } })` restores the
prior behavior. Custom-header forwarders (the common case) are
unaffected.
## Red-green proof (real surface, both paths)
RED — with the predicate reverted to the old wide-open `authorization ||
x-*` (policy ignored), the new behavior assertions fail; the leak
reproduces (`x-forwarded-for: 203.0.113.7` forwards on both /run and
/connect):
```
❯ header-utils.test.ts (19 tests | 8 failed)
× strips known infra/proxy/platform headers by exact name → expected true to be false
× strips known infra/platform header families by prefix → expected true to be false
× strips denylisted headers case-insensitively → expected true to be false
× deny extends the default set → expected true to be false
× denyPrefixes extends the default set → expected true to be false
× allow switches to allowlist mode → expected true to be false
× extractForwardableHeaders drops denylisted x-* infra → expected {…4} to deeply equal {…1}
❯ agent-utils-header-forwarding.test.ts (/run) (10 tests | 1 failed)
× strips denylisted infra/platform headers (#5712 breadth) → expected '203.0.113.7' to be undefined
❯ sse-connect-agent-id.test.ts (/connect) (5 tests | 1 failed)
× strips denylisted infra/platform headers → expected '203.0.113.7' to be undefined
```
GREEN — with the real policy in place:
```
✓ header-utils.test.ts (19 tests)
✓ agent-utils-header-forwarding.test.ts (10 tests) # /run path
✓ sse-connect-agent-id.test.ts (5 tests) # /connect path
✓ agent-header-precedence.test.ts (2 tests)
Test Files 4 passed (4)
Tests 36 passed (36)
```
Full `@copilotkit/runtime` suite: **113 files / 1593 tests passed.**
Typecheck, oxlint (0 errors), oxfmt, and build all green.
## Builds on #5782
This branches off #5782's head (`636bcad05`) and reuses that PR's
`mergeForwardableHeaders` (server-wins precedence + server-self
case-dedup). It should land **after #5782**. It addresses the
**forwarding-breadth half of #5712** — #5712's precedence core is fixed
by #5782; this is the breadth follow-up (not `Fixes #5712`).
Bucket A — mobile open-flash:
- Default the drawer element's `open` property to `false`. On a mobile viewport
the previous `open = true` default made the first render satisfy
_isMobileModalOpen(), painting the modal + body scroll-lock + focus steal for
one frame before any wrapper effect could close it. Desktop is unaffected
(only .root.mobile.open / _isMobileModalOpen() consume `open`).
- Element tests: added an `open` option to setup(); updated the 6 mobile/desktop
tests that relied on the old open=true default to opt in explicitly; added a
regression test asserting a fresh mobile element defaults open=false and paints
no backdrop / applies no scroll-lock until opened.
Bucket B — dead CSS + inert `confirming` machinery:
- styles.ts: removed the `.row-action[data-tooltip]` hover/focus tooltip rules
and the `.root.confirming .row-action[data-tooltip]…` suppression rules — no
rendered .row-action carries data-tooltip anymore (row actions moved into the
kebab menu as labeled .row-menu-items). Kept the .row-action base rules (still
used by the confirm-dialog Cancel button + fetch-more retry).
- element: removed the now-inert `confirming` root class (it gated only the
dead CSS above) and its stale comment; deleted the test asserting the no-op
tooltip suppression. Refreshed two comments that referenced the removed
row-action tooltip lineage.
Bucket B — comment/test hygiene:
- react-core CopilotThreadsDrawer.tsx: reworded the imprecise event-rebind
comment to describe the actual [mounted] deps.
- angular spec: added a beforeEach resetting the module-level threadsState
signals + clearing mock fns to remove order-coupling (parity with react/vue).
- vue use-threads.test.ts: aligned MockThreadStore.unarchiveThread + its
assertion to the real core contract (PATCH /threads/{id} { archived:false },
not POST /threads/{id}/unarchive).
Verified: web-components (89), vue (1068), angular (176), react-core suites all
green; web-components + react-core builds green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Problem
When a self-hosted v2 `CopilotRuntime` is configured with a server-side
agent (an `@ag-ui/client` `HttpAgent` with static `headers` for
service-to-service auth), the runtime forwards inbound
`authorization`/`x-*` request headers onto the agent's outgoing call
**and lets them override the headers the server configured** — silently
breaking service-to-service auth to a secured backend (e.g. a private
Cloud Run agent behind IAM).
`Fixes #5712`
## Root cause
`packages/runtime/src/v2/runtime/handlers/shared/agent-utils.ts:125-128`
merged forwarded inbound headers **last**, so they won on collision:
```ts
agent.headers = {
...agent.headers, // server-configured
...extractForwardableHeaders(request), // inbound — overrode the above
};
```
There are actually **two** failure modes:
1. **Same-case collision** — inbound `authorization` overwrites a server
`authorization` (last-write-wins).
2. **Case-mismatch collision** — `extractForwardableHeaders` lowercases
inbound keys (`authorization`), while the server typically configures
canonical casing (`Authorization`). A plain spread treats those as
*distinct* keys and emits **both** — which undici downstream comma-joins
into a single invalid `"Bearer A, Bearer B"` ("multiple JWTs") value.
Flipping the spread order alone does **not** fix this case.
## Fix
In `agent-utils.ts`, make server-configured `agent.headers`
authoritative on collision, matched **case-insensitively**: drop any
forwarded inbound header whose name (case-insensitively) is already set
on the agent, and let non-colliding inbound headers pass through
unchanged. This preserves the existing forward-for-auth behavior for
headers the server does *not* set, while guaranteeing a server-set token
is never overridden or duplicated.
The merge logic lives in a shared
`mergeForwardableHeaders(serverHeaders, request)` helper in
`packages/runtime/src/v2/runtime/handlers/header-utils.ts` so the
precedence semantics are defined in exactly one place.
### Scope note
This is the conservative precedence + case-insensitive-dedup fix (the
issue's suggested fix#1). I did **not** tighten the default allowlist
to drop hop-by-hop/platform `x-*` headers (`x-serverless-*`,
`x-forwarded-*`, …) or add an opt-out — those alter existing forwarding
behavior and are worth a separate, deliberate change. The precedence fix
alone resolves the reported breakage (the server-set token now wins
regardless of what the platform injects on a colliding header name).
A documented workaround already exists for users on released versions:
pass a custom `fetch` to the `HttpAgent` that builds outgoing headers
from scratch (it runs after `configureAgentForRequest` and survives the
per-request `agent.clone()`).
## Red-green proof (the real fix — `/run` path)
The load-bearing assertion: there must be exactly **one** authorization
header carrying the **server** value.
### RED (fix stashed, against unmodified `agent-utils.ts`)
```
❯ src/v2/runtime/__tests__/agent-header-precedence.test.ts (2 tests | 1 failed)
× configureAgentForRequest — header precedence (#5712) > server-configured agent headers win over a colliding inbound header
AssertionError: expected [ 'Authorization', 'authorization' ] to have a length of 1 but got 2
81| expect(authKeys).toHaveLength(1);
Test Files 1 failed (1)
Tests 1 failed | 1 passed (2)
```
The pre-existing `agent-utils-header-forwarding.test.ts` also failed,
because it explicitly encoded the buggy behavior
(`expect(...["x-aimock-context"]).toBe("new-context")` — inbound
winning):
```
FAIL src/v2/runtime/__tests__/agent-utils-header-forwarding.test.ts > ... > request forwardable headers override matching pre-existing agent headers
AssertionError: expected 'old-context' to be 'new-context'
```
### GREEN (fix applied)
```
✓ src/v2/runtime/__tests__/agent-header-precedence.test.ts (2 tests) 2ms
✓ src/v2/runtime/__tests__/agent-utils-header-forwarding.test.ts (8 tests) 3ms
Test Files 2 passed (2)
Tests 10 passed (10)
```
The colliding test (`agent-utils-header-forwarding.test.ts`) was updated
from asserting the old bug to asserting corrected precedence + a new
case-insensitive-dedup guard. The non-colliding-forward test is retained
unchanged as a regression guard.
## Quality gates
```
NX Successfully ran target check-types for project @copilotkit/runtime
NX Successfully ran target test for project @copilotkit/runtime — Test Files 113 passed (113), Tests 1576 passed (1576)
```
---
## `/connect`-path change — forward-looking plumbing, inert today
The original issue and a prior eval flagged the same forwarding pattern
at `handlers/sse/connect.ts`. To keep the two paths' merge semantics
consistent, the `/connect` path now builds the same server-wins merged
headers (via the shared `mergeForwardableHeaders` helper) and passes
them into `runner.connect()`.
**This is not an active auth fix, and it is not red-green-proven as one
— because there is no live bug to fix on the connect path today.** No
shipped runner consumes the `headers` field of
`AgentRunnerConnectRequest`: the in-memory, intelligence, telemetry, and
sqlite runners all destructure only `threadId` from the connect request
and ignore `headers` entirely. Connect is a thread replay/reconnect, not
a fresh outgoing agent call. So whatever headers we pass into
`runner.connect()` are dropped on the floor by every runner that ships.
What this change actually does:
- Threads the per-request agent clone through `handle-connect.ts →
handleSseConnect` so the connect path *has access to* the
server-configured `agent.headers` (it previously did not).
- Passes `mergeForwardableHeaders(agent?.headers, request)` into
`runner.connect()` — the correct, server-wins argument **shape** for a
future outbound-connecting runner that *would* consume connect-path
headers.
- Rewrites the comments/JSDoc on this path to say this plainly, rather
than implying an active auth fix. It also documents that the
connect-site `cloneAgentForRequest` call is the sole `agentId`-existence
guard (the intelligence branch never re-validates the id), and documents
`cloneAgentForRequest`'s `AbstractAgent | Response` (404) dual-return
contract that both callers depend on.
The real outbound header forwarding — the thing that fixes#5712 — is
the `/run` path's `agent.headers` mutation described above. The connect
change is staged plumbing so that if/when a runner starts honoring
connect-path headers, it inherits the same server-wins precedence
without a second fix.
### Tests on the `/connect` path
The connect tests assert the *merge shape* that reaches
`runner.connect()` (server value wins on collision, exactly one
`authorization` key, non-colliding `x-*` still forwards) and that the
agent-undefined case (no server `agent.headers`) degrades to forwarding
allowlisted inbound headers only and does not crash. These verify the
argument we construct is correctly shaped — not that any shipped runner
consumes it.
## Files
- `packages/runtime/src/v2/runtime/handlers/header-utils.ts` — shared
`mergeForwardableHeaders` helper (case-insensitive, server-wins).
- `packages/runtime/src/v2/runtime/handlers/shared/agent-utils.ts` —
`/run` path uses the helper so server headers win on collision (**the
real fix**).
- `packages/runtime/src/v2/runtime/handlers/sse/connect.ts` — `/connect`
path uses the helper; forward-looking plumbing, inert until a runner
consumes connect-path headers.
- `packages/runtime/src/v2/runtime/handlers/handle-connect.ts` — threads
the per-request agent clone into `handleSseConnect`.
-
`packages/runtime/src/v2/runtime/__tests__/agent-header-precedence.test.ts`
— `/run` regression test exercising the real `configureAgentForRequest`
surface with a real `HttpAgent`.
-
`packages/runtime/src/v2/runtime/__tests__/agent-utils-header-forwarding.test.ts`
— updated the test that encoded the old (buggy) precedence; added a
case-mismatch dedup guard.
-
`packages/runtime/src/v2/runtime/handlers/sse/__tests__/sse-connect-agent-id.test.ts`
— connect-path merge-shape + agent-undefined coverage.
### Notes
- A documented `@ag-ui/client` `HttpAgent` `fetch` workaround already
exists for attaching service-to-service auth the runtime can't override
(see the issue). This change makes the workaround unnecessary for the
`/run` precedence case.
- Conservative scope: this is the **precedence flip on `/run`** plus
forward-looking connect plumbing. Tightening the default allowlist
(dropping hop-by-hop / platform `x-serverless-*`, `x-forwarded-*`,
`x-cloud-trace-context`, …) and an opt-out switch — issue suggestions
#2/#3 — are intentionally left as a follow-up to keep the
security-policy change minimal.
Managed bots ran single-turn: conversationStore.getOrCreate returned a bare
agent with no messages, so the agent never saw prior turns ("chart the CSV I
sent above" failed). Mirror bot-slack — seed agent.messages from thread history:
- add DeliverySource.getHistory + HttpDeliverySource.getHistory, which GETs
the Intelligence /api/bots/history endpoint (returns [] when threadTs is
absent, i.e. a root turn; best-effort — a 4xx logs a loud misconfig warning,
5xx/429/network degrade quietly; never throws into the turn)
- getOrCreate unwraps ManagedReplyTarget.route and seeds agent.messages
- historyLimit adapter option (default 20)
- extract buildContentParts into content-parts.ts as the single source of truth
so historical files (incl. images) hydrate identically to the live turn
Document the v2 runtime's inbound-header forwarding behavior on the
Copilot Runtime page: the default denylist (authorization + x-* minus
known infra/proxy/platform headers), the x-request-id upgrade note,
server-configured header precedence (#5782), and the forwardHeaders
config option (deny/denyPrefixes/allow/useDefaultDenylist) with the
allowlist-mode denylist-bypass footgun.
Refs #5712, #5783
D2 — wire fetchMoreError end-to-end:
- core: add a dedicated `fetchMoreError` channel to the thread store, tracked
separately from the initial-list `error`. `nextPageFailed` now writes
`fetchMoreError` (was `error`), so a paginated-load failure preserves the
loaded list and drives the element's inline "couldn't load more — retry"
panel instead of a full-panel error. Cleared on fetch-more request (retry),
on success, and reset on context change / stop. New symbols:
* `ThreadState.fetchMoreError`
* `ThreadSelectors.fetchMoreError` + `ɵselectFetchMoreError`
Call sites of `ɵselectFetchMoreError`:
* packages/core/src/threads.ts (export)
* packages/react-core/src/v2/hooks/use-threads.tsx (selector read)
* packages/vue/src/v2/hooks/use-threads.ts (bindThreadStoreSelector)
* packages/angular/src/lib/threads.ts (bridge to signal)
* packages/vue/src/v2/hooks/__tests__/use-threads.test.ts (core mock)
Call sites of `ThreadSelectors.fetchMoreError` (mock objects updated):
* packages/core/src/__tests__/core-thread-store-auto-unregister.test.ts
* packages/core/src/__tests__/thread-store-registry.test.ts
- react/vue/angular: expose `fetchMoreError` on the hook/composable/store and
push it onto `el.fetchMoreError`, making the dead `retry{scope:"fetch-more"}`
handler reachable. Initial-list error behavior unchanged.
D1 — Angular wrapper open-state coordination: drive `el.open` from the config's
`drawerOpen` (default CLOSED) so the element no longer springs open full-screen
and scroll-locks on mobile load; handle `(open-change)` -> `config.setDrawerOpen`
with a provider-less local-state fallback; call `config.registerDrawer()` with
cleanup on destroy. The config's drawer members were fully functional (only
marked "RESERVED/unwired") — wiring them makes them consumed, so their comments
were updated accordingly (no reservation conflict).
D3 — Angular focus-return: add a `findChatInput` scoped to the Angular chat
selectors (`copilot-chat-view` container, `textarea[copilotChatTextarea]`) and
focus it on thread select, mirroring React/Vue.
A6 — react wrapper comment rot: "nine outbound events" -> "eleven" (2 spots).
DEFAULT_AGENT_ID parity (react): import `DEFAULT_AGENT_ID` from
`@copilotkit/shared` instead of hardcoding `"default"` (equal value).
Angular test isolation: three list-path tests now set `licenseStatusSignal`
explicitly instead of relying on inherited module-level state.
Tests: core 552, react-core 1417, vue 1068, angular 176 — all pass;
check-types passes for all four packages.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Search toggle-close clears the query and emits search{query:""} via a shared
_closeSearch() helper, so it no longer leaves a stale, invisible list filter.
- Escape-close routes through _closeSearch() too, so a consumer's onSearch is
never left holding a stale query.
- Composed document pointerdown listener (bound/removed in connected/
disconnectedCallback) dismisses the funnel popover and row kebab menu on
outside clicks; selecting a thread row also closes an open menu.
- Kebab popover no longer clipped by the list overflow for bottom rows:
lower-half rows open the menu upward (menu-up -> bottom-anchored popover).
- Locked/unlicensed view no longer renders the search toggle, search input, or
"New Conversation" row — only the Upgrade panel (collapse toggle kept).
- Row-action aria-labels use the shared hasName?name:"New thread" fallback so an
empty-string name never announces a trailing blank.
- Hardened CSS-contract tests: archived-italic and :host height:100% match
selector+declaration as a unit; muted-not-struck checks color + no line-through.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the 'a filter is applied' dot (part=filter-indicator, --cpk-drawer-indicator,
default #5b94e4) shown on the funnel when the active filter is not the default,
matching the Figma archived view.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Also re-exports SearchDetail from @copilotkit/web-components/threads-drawer
so the React wrapper's type import resolves.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the optional header projection slot back to the redesigned icon-row
header (flex:1 so empty keeps icons right-aligned, filled fills the left),
keep `label` driving the region + listbox accessible names only, drop the
retired visible-title expectations from the two label tests, and remove the
orphaned --_rail-width token and .row-action.danger rule.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Jordan Ritter