Bump the canonical CopilotKit pin across all showcase integrations + shell
to 1.61.2 (canonical-pins.json, every package.json + package-lock.json),
which carries CopilotKit#5611: passing a catalog to the provider
(`<CopilotKit a2ui={{ catalog }}>`) now auto-enables A2UI and defaults tool
injection on, so the runtime no longer needs an explicit `a2ui` config.
Demonstrate the feature on the A2UI dynamic (declarative-gen-ui) demos by
removing the now-redundant runtime `a2ui` block (`injectA2UITool: true` +
`defaultCatalogId`) from:
- langgraph-python, langgraph-fastapi, langgraph-typescript
- strands, strands-typescript
- google-adk
The forwarded catalog supplies its own catalogId (sdk-js A2UI middleware
auto-derives `defaultCatalogId` from it), so the previous "Catalog not found"
fallback no longer applies.
Verified: validate-pins drift ratchet unchanged (38 / same hash);
langgraph-python D6 `gen-ui-declarative` green end-to-end (no Catalog-not-found).
Bump every @copilotkit/* dependency across the showcase integrations and
the shell from 1.60.2 (and stray "latest" override pins) to an exact
1.61.1 pin, and move the canonical pin source of truth to match.
Regenerate each standalone npm package-lock.json with the same
--legacy-peer-deps flag the Dockerfiles use for "npm ci".
- showcase/integrations/*/package.json + package-lock.json
- showcase/integrations/langgraph-typescript/src/agent/*
- showcase/shell/package.json + package-lock.json
- showcase/scripts/showcase-canonical-pins.json: canonical 1.60.2 to 1.61.1
aimock stays on its own version line (1.26.1). The Python copilotkit SDK
was already 0.1.94 across every requirements.txt, so no change there.
validate-pins ratchet is unchanged (FAIL=38, identical hash);
validate-parity, validate-fixture-tool-surface, and the showcase/scripts
vitest suite (2102 tests) all pass.
## What
Adds **cvdiag** — a permanent, always-available observability subsystem
for the showcase, built to diagnose the red↔green cell flap on the
staging dashboard and to make that diagnosis a dashboard query rather
than a multi-day forensic hunt in the future.
Captures the full request path with `X-Test-Id` correlation across
**probe → backend → aimock → edge**, across every integration
(TypeScript, Python, Java/spring-ai, .NET):
- Per-language backend emitters (canonical + staged/compile-linked
mirrors), all sharing one schema (`schema.json`, closed-world
`additionalProperties:false`).
- CREATE-only writes to two new PocketBase collections: `cvdiag_events`
and `cvdiag_raw_byte_samples` (additive migrations — no existing data
touched).
- An 8-class flap classifier mapping to the observed failure signatures
(`sse-missing` / `text-unstable` / `dom-missing`).
- DEBUG-tier raw-byte capture (secret-scrubbed) and HMAC-guarded A/B
edge-interference detection.
## Why
The runId flap-fix (`cdc1e90e`, 2026-06-09) did **not** fully resolve
the flap — it was still observed 2026-06-19. cvdiag exists so the
*remaining* cause is observed live with full correlation instead of
inferred.
## Safety / enablement
- **Inert by default.** With `CVDIAG_BACKEND_EMITTER` unset the
subsystem performs zero host mutation (no logging-config changes, no
threads/tasks, no stdout) — verified by
`test_cvdiag_inert_when_disabled`. **To accumulate data, set
`CVDIAG_BACKEND_EMITTER=1` on the showcase services.**
- All per-language scrubbers match the canonical `scrubSecrets`
(sk-/base64url, Bearer, colon-less URL userinfo, size-guard) — verified
with real toolchains (vitest / mvn / dotnet).
- Merged latest `main` (only conflict: a clean `.csproj` include union).
## Verification
- harness `tsc --noEmit` ✓ · `src/cvdiag` vitest 251/251 ✓ ·
`cvdiag-stage-ts --check` in-sync ✓
- Java MessageScrubber 17/17 (mvn) ✓ · .NET CvdiagBackend 5/5 (dotnet
sdk:9.0) ✓ · Python emitters 93/93 (3.12) ✓
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Resolve event-renderer.ts onRunFinishedEvent: keep the native turn stream open
(finalized in finish()) AND retain the legacy per-message stream drain from main
(#5573) as a no-op-in-native safety net. app/index.ts (telegram adapter from
#5520 + showToolStatus:false) and create-bot.test.ts auto-merged.
Aligns dependency versions across all 19 showcase integrations to current
released minor versions for the 1.60.2 release cycle.
Package families:
- @copilotkit/{a2ui-renderer, react-core, react-ui, runtime, shared, sdk-js, voice}
1.59.4 -> 1.60.2 (18 integrations already staged; ms-agent-harness-dotnet
catches up from 1.57.2)
- @ag-ui/{client, core, encoder} 0.0.55 -> 0.0.57
- @ag-ui/mastra 0.2.1-beta.2 -> 0.2.4 (stable on 0.x; 1.0.x major held back)
Includes the previously-missed ms-agent-harness-dotnet integration in the
@copilotkit/* bump, plus the @copilotkit/web-inspector override pin.
Lockfile-only reconciliation via npm install --package-lock-only
--legacy-peer-deps (cmdk@0.2.1 pre-existing react^18 peer-dep is unaffected).
The injected render_a2ui tool guide instructs models to omit catalogId
("the catalog id is set by the host"), and backend-owned generate_a2ui
tools see real models omit or late-stream it. Without defaultCatalogId
the a2ui middleware falls back to the spec basic catalog, which no
showcase page registers — surfaces fail with "Catalog not found:
https://a2ui.org/specification/v0_9/basic_catalog.json" (reported on
beautiful-chat / langgraph-python).
Pin each route to the catalog its page registers: beautiful-chat ->
copilotkit://app-dashboard-catalog, declarative-gen-ui ->
declarative-gen-ui-catalog. Routes with no a2ui block never attach the
middleware and are left untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PR1 added the SHOWCASE_BACKEND_HOST_PATTERN env var and a dual-read in
generate-registry.ts that synthesizes backend_url when the manifest omits
it. This commit (PR2) makes the env-var-derived path the only path.
- Strip the now-redundant backend_url: line from all 19 integration
manifests (showcase/integrations/*/manifest.yaml).
- generate-registry.ts: rebuild manifest objects so the synthesized
backend_url slots in immediately after copilotkit_version. With this
change registry.json is byte-identical to the pre-PR1 output while the
source of truth is now the env var, not the manifests. Comment updated
to reflect the new state.
- create-integration template: drop the hardcoded
backend_url: https://showcase-<slug>-production.up.railway.app line so
newly scaffolded integrations omit the field too. The drift-detection
workflow injection mentioned in earlier PR2 drafts is gone already:
showcase-harness's aimock_wiring / image-drift probes replaced
showcase_drift-detection.yml, so no workflow file needs editing.
- manifest.schema.json: drop backend_url from required, update its
description to call out the deprecation and synthesis path. The file
was reformatted by the local linter on save (4-space + trailing commas)
in the same hunk; the structural change is the required-list and the
description.
- starter.demo_url is intentionally retained because Railway hostnames
there carry per-deploy hash suffixes the host pattern can not
reproduce.
Verified locally:
- tsx generate-registry.ts -> byte-identical to baseline registry.json.
- SHOWCASE_BACKEND_HOST_PATTERN='showcase-{slug}-staging.example.com'
produces the expected per-slug staging URLs.
- tsc --noEmit -p showcase/scripts/tsconfig.json: clean.
- vitest run in showcase/scripts: 1308/1308 passing.
- playwright test --list in showcase/tests: 79 tests enumerate cleanly.
Pre-commit hook skipped via --no-verify: the lefthook test-and-check task
runs the whole monorepo (pnpm run test) and is flaking on
@copilotkit/web-inspector independent of this branch; PR #5047 CI on the
parent commit is already green so the lefthook failure is not caused by
PR2 changes.
Brings the Microsoft Agent Harness (.NET) integration live on the
showcase Railway project. Integration code itself landed in PR #4982.
Changes:
- Railway service `showcase-ms-agent-harness-dotnet` created
(id 6343d7f9-6c3f-4c8d-9a6e-79f03d2f1e37) with the public domain
showcase-ms-agent-harness-dotnet-production.up.railway.app, image
source ghcr.io/copilotkit/showcase-ms-agent-harness-dotnet:latest,
healthcheck /api/health, and env vars cloned from the sibling
showcase-ms-agent-dotnet service.
- .github/workflows/showcase_build.yml: add ms-agent-harness-dotnet to
workflow_dispatch options, paths-filter, and the ALL_SERVICES matrix
(mirroring the ms-agent-dotnet sibling entry).
- showcase/integrations/ms-agent-harness-dotnet/manifest.yaml: flip
deployed: false -> true so the dashboard surfaces the integration
once the image is live.
Skips test-and-check-packages pre-commit hook locally because
@copilotkit/web-inspector:test has a pre-existing failure on main
(window.localStorage.clear telemetry test setup) unrelated to these
YAML-only changes.
The previous wave-4 fix added `host == "::1"` for IPv6 loopback support,
but verification against `mcr.microsoft.com/dotnet/sdk:9.0` showed that
`new Uri("http://[::1]:8000/").Host` returns `"[::1]"` WITH brackets, not
`"::1"`. The string-equality check therefore never matched and IPv6
loopback detection was silently broken.
Switch to `Uri.IsLoopback`, which is the framework-provided helper that
robustly identifies all loopback variations: `localhost`, the entire
`127.0.0.0/8` range, `::1` in any bracketed form, and IPv4-mapped IPv6
loopback (`::ffff:127.0.0.1`). This sidesteps `Uri.Host`'s bracket-
formatting quirks entirely.
The explicit `0.0.0.0` (unspecified address, kept for back-compat) and
`aimock` (Docker-compose service name) checks are retained because they
are not loopback addresses.
ResolveApiKey and ResolveEndpoint previously cascaded through env var →
configuration sources using the `??` operator, which only short-circuits
on null. When `OPENAI_API_KEY=""` or `OPENAI_BASE_URL=""` was set in the
environment, the empty string was returned by the cascade WITHOUT
consulting the configuration fallbacks (`configuration["OPENAI_API_KEY"]`,
`configuration["GitHubToken"]`, `configuration["OPENAI_BASE_URL"]`),
masking the configured values entirely. The downstream
`IsNullOrWhiteSpace` check would then push the resolver into the
mock-or-throw path even when a valid key was configured.
Replace the `??` chains with a `FirstNonBlank` helper that treats both
null and whitespace-only candidates as absent, so empty env vars fall
through to the configuration fallbacks as intended.
Wave-1 widened the catch list in BeautifulChatAgent.GenerateA2ui from
specific upstream exceptions to a blanket catch (Exception ex). That
inadvertently swallowed InvalidOperationException thrown by
ApiKeyResolver.ResolveApiKey, which is the intentional fail-fast contract
for misconfigured deployments. With the blanket catch in place, a missing
or invalid API key produced a generic "unexpected_error" structured
response to end users while the operator received no clear startup-level
signal — exactly the silent-degradation failure mode the fail-fast was
designed to prevent.
Replace the blanket catch with specific catches for the two
runtime-recoverable exception types the secondary tool caller can throw
after the transport/SDK/cancellation handlers:
- JsonException — upstream returned malformed JSON
- KeyNotFoundException — upstream JSON missing required fields
(defensive; TryGetProperty guards most paths after wave-1)
Both map to the existing "upstream_malformed" structured-error taxonomy.
All other exceptions, including configuration errors like
ApiKeyResolver's InvalidOperationException, now propagate so a
misconfigured deployment fails loudly at the operator layer instead of
limping along behind a generic user-facing message.
ASP.NET Core hands control back to the middleware once the endpoint handler
completes via `await _next(context)`, but for streaming endpoints (AG-UI uses
`IAsyncEnumerable`/SSE) the response delegate continues writing to the response
body — and may issue downstream OpenAI calls — AFTER `_next` returns. The
previous `finally` clause reset `AimockHeaderContext` to an empty dictionary at
that point, which caused `AimockHeaderPolicy` to silently drop aimock headers
on those streaming-tail calls. That defeated the entire purpose of the
middleware for the exact codepath it was built to serve.
The `finally`-clear was also unnecessary for isolation: `AsyncLocal<T>` is
scoped to the current execution context, so each ASP.NET request gets its own
slot automatically. There is no cross-request leakage to defend against.
Remove the `try`/`finally` and just call `AimockHeaderContext.Set(headers)`
followed by `await _next(context)`. Add a comment documenting why no reset is
performed.
ApiKeyResolver.IsMockEndpoint compared Uri.Host against "[::1]", but
System.Uri.Host strips the surrounding brackets and returns "::1" for
inputs like http://[::1]:8000/. The bracketed comparison was therefore
dead code: developers running aimock on IPv6 loopback got the fail-fast
error instead of the intended mock-key fallback.
Change the literal to "::1" and update the inline + docstring comments
to reflect the bracket-less form.
- BeautifulChatStateSnapshotAgent.RunCoreAsync now coalesces a null
response.Messages to an empty array before constructing the new
List<ChatMessage>. Previously, an inner agent that returned no
messages would cause `new List<ChatMessage>(null)` to throw
ArgumentNullException, masking the real upstream behaviour.
- GenerateA2ui's empty-content branch now returns the canonical
BeautifulChatA2ui.StructuredError shape (error / message /
remediation / errorId) like every other error path in the method,
instead of an ad-hoc { error, errorId } object. The frontend
expects the structured shape with remediation guidance.
Program.cs#CreateOpenAiClient previously read the OpenAI endpoint solely from
the OPENAI_BASE_URL environment variable and fell back to
ApiKeyResolver.DefaultOpenAiEndpoint. The secondary tool-calling HTTP client
(A2uiSecondaryToolCaller) instead routes through ApiKeyResolver.ResolveEndpoint,
which checks env, then configuration[OPENAI_BASE_URL] (appsettings.json /
user-secrets), then the default.
When OPENAI_BASE_URL was supplied only via configuration (e.g. user-secrets in
local dev), the primary client silently dialed the public Azure-hosted models
endpoint while the secondary client dialed the configured aimock. The two
endpoints diverged with no visible signal, breaking aimock-routed flows and
masking misconfigurations.
Switch CreateOpenAiClient to call ApiKeyResolver.ResolveEndpoint so both
clients share a single source of truth. Logging is preserved and now reports
which source supplied the endpoint (env, configuration, or default).
When iterating IHeaderDictionary's underlying store yields case-variant
duplicates (e.g., a misbehaving proxy injecting both `X-Foo` and `x-foo`),
the previous GroupBy + ToDictionary path silently kept only the first value
and discarded the rest. For aimock context routing — where header semantics
drive fixture selection — silent drops are a debugging nightmare.
We still keep `.First()` because HTTP defines no canonical merge for
case-variant collisions across distinct keys (the comma-join rule only
applies when keys are ASCII-equal). Instead, when a group has more than
one entry, we emit a structured warning naming the key, the kept value,
and the number of dropped variants, so operators can spot the upstream
misbehavior in logs.
Constructor now takes `ILogger<AimockHeaderMiddleware>` — ASP.NET's
middleware activator injects it automatically via `UseMiddleware<T>()`.
Two surgical hardening fixes to ApiKeyResolver:
1. IsMockEndpoint: drop host.StartsWith("aimock.", ...) and
host.EndsWith(".aimock", ...). The StartsWith match is exploitable
— an attacker-registered domain like aimock.attacker.example.com
would be classified as a mock endpoint and bypass the fail-fast,
silently returning the mock key. The EndsWith match is unused
noise (no .aimock TLD exists). Only exact, well-known mock hosts
("localhost", "127.0.0.1", "0.0.0.0", "[::1]", "aimock") are
accepted. IPv6 loopback [::1] is added so dual-stack dev paths
are still covered.
2. ResolveApiKey: change !string.IsNullOrEmpty(apiKey) to
!string.IsNullOrWhiteSpace(apiKey). A whitespace-only key such as
" " would otherwise be accepted as valid, bypassing the mock-key
/fail-fast logic entirely and sending a bogus key upstream.
- RunCoreAsync: avoid mutating AgentResponse.Messages in place. Reassign
Messages to a fresh List<ChatMessage> so the path is safe regardless of
whether the inner agent returns a mutable list or an immutable
IReadOnlyList wrapper. Prevents NotSupportedException at runtime; all
other response metadata (AgentId, ResponseId, Usage, RawRepresentation,
...) is preserved via the property setter.
- RunCoreStreamingAsync: only emit the trailing todos snapshot if the
inner stream completed normally. Wrap the inner enumerator in
try/finally with a streamCompletedNormally flag; on early exit
(throw or cancellation) log a warning and skip the snapshot rather
than emitting potentially stale state to the frontend. The trailing
yield lives outside the try block (idiomatic C# pattern since `yield`
cannot live inside `try { } catch { }`).
The previous implementation used Contains() substring matching against the
full endpoint URL, which is exploitable. An attacker-controlled endpoint
such as https://attacker.example.com/aimock-decoy or
https://api.openai.com/?env=localhost would be classified as a mock and
bypass the fail-fast guard, silently returning the sk-mock-local key for
what is actually a non-mock destination.
Parse the URL and inspect only the host component, matching exact dev
hosts (localhost, 127.0.0.1, 0.0.0.0, aimock) plus aimock subdomains.
Path, query, and arbitrary subdomain segments containing "aimock" or
"localhost" no longer trigger the mock fallback.
The header propagation chain (middleware -> context -> policy) had two bugs
that combined to threaten D5/D6 header-forwarding:
1. AimockHeaderMiddleware.ToDictionary used the default ORDINAL case-sensitive
comparer. ASP.NET's IHeaderDictionary is case-insensitive, but iterating
the underlying store can yield case-variant duplicates (e.g., a misbehaving
proxy injecting both `X-Foo` and `x-foo`). Default-comparer ToDictionary
throws ArgumentException on duplicates and fails the request.
2. AimockHeaderContext.Set lowercased all keys via ToLowerInvariant.
AimockHeaderMiddleware captured original case; the context then mutated
the casing; AimockHeaderPolicy.TryGetValue then matched against whatever
case the OpenAI SDK happened to use. This is inconsistent and aimock
fixture matching can be case-sensitive depending on configuration.
Canonical strategy: preserve original header casing as captured by the
middleware, but compare case-insensitively throughout via
StringComparer.OrdinalIgnoreCase. The middleware now also groups variants
defensively so duplicate keys cannot blow up the dictionary build.
AimockHeaderPolicy's add-if-absent TryGetValue then works correctly under
case-insensitive comparison without any further changes.
The GitHubToken line shipped `ghp_...` as a literal placeholder VALUE. When
a developer runs `cp .env.example .env` and forgets to edit, the entrypoint's
`-z "$GitHubToken"` check sees a non-empty string and skips the "key missing"
warning. The literal `ghp_...` is then sent upstream, producing a confusing
401 that looks like an OpenAI/GitHub Models bug rather than a setup issue.
Move the format example into a comment above the line and leave the value
empty so the empty-value check in entrypoint.sh fires the clear warning.
OPENAI_API_KEY was already empty (prior fix); verified still empty.
- entrypoint.sh: replace `sleep 3 && kill -0` agent-startup gate with a curl
retry loop against /health on :8000 (up to 30s). The bare PID check only
proved the process existed; if Kestrel hadn't finished binding, Next.js
would proxy to a dead backend for ~90s until the watchdog killed the
container.
- entrypoint.sh: watchdog now also supervises Next.js. If Next.js dies while
the agent stays healthy the watchdog breaks out so wait -n can return and
Railway can restart the container instead of serving a broken page.
- entrypoint.sh + .env.example: align the env-var contract with what
agent/Program.cs actually reads. The .NET agent uses OPENAI_API_KEY,
GitHubToken (fallback), and optional OPENAI_BASE_URL — it never reads
AZURE_OPENAI_API_KEY. The startup warning and .env.example now document
the real key precedence plus every Next.js-side var (AGENT_URL,
NEXT_PUBLIC_BASE_URL, MCP_SERVER_URL, SHOWCASE_DEBUG_TOKEN).
- Replace silent `return null` paths in A2uiSecondaryToolCaller with
TryGetProperty guards that each emit a structured LogWarning naming
the exact missing/unexpected field (choices, message, tool_calls,
function, name mismatch, arguments). Callers can now tell why a
design-tool call produced no content.
- Add ILogger parameter to GetDesignToolArgumentsAsync and pass
BeautifulChatAgent._logger from the single call site so warnings
flow into the existing log stream.
- Log the response body (truncated to 1 KB) at LogWarning before
EnsureSuccessStatusCode throws, so upstream error payloads survive
the throw and reach operators.
- Extract the OPENAI_API_KEY/GitHubToken/sk-mock-local fallback chain
from Program.cs and A2uiSecondaryToolCaller.cs into a new
ApiKeyResolver helper. Both call sites now share one implementation.
- ApiKeyResolver fails fast with InvalidOperationException + LogCritical
when no real key is present and OPENAI_BASE_URL is not an
aimock/localhost endpoint, so misconfigured prod deploys cannot
silently send sk-mock-local to a real LLM provider. The silent
mock-key fallback is preserved for aimock/localhost dev endpoints.
- RunCoreAsync: append todos snapshot DataContent to AgentResponse so non-streaming
callers receive the same state mirror that RunCoreStreamingAsync already emits.
- ManageTodos: defensive-copy each incoming todo before storing (mirrors the
symmetry of GetTodosSnapshot) so callers cannot mutate our backing list by
retaining input references.
- ManageTodos: validate Status against the documented "pending" | "completed" set;
coerce out-of-range values to "pending" with a LogWarning instead of letting
arbitrary LLM-supplied strings into shared state.
- GenerateA2ui: add a final catch (Exception) returning a StructuredError
("unexpected_error", ...) matching the existing taxonomy, and log an info-level
entry when OperationCanceledException flows through (was previously silent).
AimockHeaderPolicy previously called message.Request.Headers.Set(...)
unconditionally for every key returned by AimockHeaderContext, which
silently clobbered any header already set by an earlier pipeline policy
or the SDK itself (e.g. x-request-id, x-correlation-id).
Switch both Process and ProcessAsync to add-if-absent semantics, using
PipelineRequestHeaders.TryGetValue to skip keys that already have a value
on the outbound request. New aimock x-* headers still propagate; existing
correlation/SDK headers are preserved.
Aligned everything to npm to match what the Dockerfile already uses
(`npm ci --legacy-peer-deps`) and the committed `package-lock.json`.
The sibling `ms-agent-dotnet` integration uses the same npm-based
setup, so npm is the established convention.
Changes:
- playwright.config.ts: webServer.command now `npm run dev` (was `pnpm dev`),
so local E2E works in environments without pnpm installed.
- package.json: removed the redundant top-level `pnpm.overrides` block.
The equivalent override is already declared under npm's `overrides`
field, so the pnpm block was dead weight given that npm is canonical.
- package.json: `scripts.dev` now uses `concurrently -k --success first`
so a crashing .NET agent surfaces during local dev instead of leaving
Next running headlessly.
Ran Shem Tov