## What
Adds **cvdiag** — a permanent, always-available observability subsystem
for the showcase, built to diagnose the red↔green cell flap on the
staging dashboard and to make that diagnosis a dashboard query rather
than a multi-day forensic hunt in the future.
Captures the full request path with `X-Test-Id` correlation across
**probe → backend → aimock → edge**, across every integration
(TypeScript, Python, Java/spring-ai, .NET):
- Per-language backend emitters (canonical + staged/compile-linked
mirrors), all sharing one schema (`schema.json`, closed-world
`additionalProperties:false`).
- CREATE-only writes to two new PocketBase collections: `cvdiag_events`
and `cvdiag_raw_byte_samples` (additive migrations — no existing data
touched).
- An 8-class flap classifier mapping to the observed failure signatures
(`sse-missing` / `text-unstable` / `dom-missing`).
- DEBUG-tier raw-byte capture (secret-scrubbed) and HMAC-guarded A/B
edge-interference detection.
## Why
The runId flap-fix (`cdc1e90e`, 2026-06-09) did **not** fully resolve
the flap — it was still observed 2026-06-19. cvdiag exists so the
*remaining* cause is observed live with full correlation instead of
inferred.
## Safety / enablement
- **Inert by default.** With `CVDIAG_BACKEND_EMITTER` unset the
subsystem performs zero host mutation (no logging-config changes, no
threads/tasks, no stdout) — verified by
`test_cvdiag_inert_when_disabled`. **To accumulate data, set
`CVDIAG_BACKEND_EMITTER=1` on the showcase services.**
- All per-language scrubbers match the canonical `scrubSecrets`
(sk-/base64url, Bearer, colon-less URL userinfo, size-guard) — verified
with real toolchains (vitest / mvn / dotnet).
- Merged latest `main` (only conflict: a clean `.csproj` include union).
## Verification
- harness `tsc --noEmit` ✓ · `src/cvdiag` vitest 251/251 ✓ ·
`cvdiag-stage-ts --check` in-sync ✓
- Java MessageScrubber 17/17 (mvn) ✓ · .NET CvdiagBackend 5/5 (dotnet
sdk:9.0) ✓ · Python emitters 93/93 (3.12) ✓
🤖 Generated with [Claude Code](https://claude.com/claude-code)
setup() ran logging.basicConfig(force=True) unconditionally at import (module
calls setup() at the bottom), tearing down the host app's root-logger handlers
on every backend even when cvdiag was disabled (CVDIAG_BACKEND_EMITTER off,
canary-safe default) — violating the byte-for-byte-inert contract. Replaced the
root basicConfig(force=True) with a scoped StreamHandler attached to the
"agents" logger, installed ONLY on the ENABLED path (after _ENABLED=True), so a
disabled/degraded setup performs zero logging mutation and the host root
handlers survive. capture-when-enabled (agents.* → stdout) preserved.
Call sites: setup() defined cvdiag_bootstrap.py:133, invoked at import-time
:273 (now no longer touches root). is_enabled() :206 unchanged (gates emit).
basicConfig() call REMOVED from :128; only doc references remain. New scoped
_install_agents_log_capture() runs on the enabled path; reset_for_test() now
detaches the handler.
The four §6-VERBOSE-only backend boundaries (request.ingress, llm.call.start,
llm.call.response, sse.first_byte) called _emit with no tier_gate, so they
over-emitted at DEFAULT tier — 4 extra events/request vs the middleware family,
breaking the §7 tier budget and cross-backend apples-to-apples parity. Gate
them with tier_gate=_VERBOSE_TIERS, matching emit.ts:58-63 and the agno
_BOUNDARY_TIER. langgraph-fastapi received the identical change (the two LGP
files differ only by docstring/plan-unit/_SLUG). Adds default-suppressed +
verbose-emits red-green coverage; updates the pre-existing first_byte
correlation test to drive at VERBOSE tier (the boundary is VERBOSE-only).
emit_cvdiag now early-returns when not is_enabled(), in addition to the
per-integration CVDIAG_BACKEND_EMITTER env check, so a degraded setup()
(_ENABLED=False) emits nothing — the degrade wins over the live env toggle.
Call sites: shared emit_cvdiag (this file, the single chokepoint) now gates on
is_enabled() (the previously-dead _ENABLED flag, set False by setup()'s
fail-closed degrade). Per-integration emitter_enabled() (langgraph-python,
langgraph-fastapi) and cvdiag_backend_enabled() (10 other _cvdiag_backend.py
modules) remain env-only and call into emit_cvdiag — out of scope here; the
shared gate is the defense-in-depth backstop for all of them.
Now that strands-typescript ships as "AWS Strands (TypeScript)", rename the
existing Python integration's display name from "AWS Strands" to
"AWS Strands (Python)" so the two appear as a matched Python/TypeScript pair
in the docs framework selector and shell (mirrors LangGraph's
"(Python)"/"(TypeScript)" pairing). Slug, content, and behavior are
unchanged. The shared aws-strands docs language tabs already default per
framework (strands -> Python, strands-typescript -> TypeScript) via
TAB_DEFAULTS_BY_SLUG.
Add a new node/TypeScript-backed AWS Strands showcase integration at
showcase/integrations/strands-typescript.
Backend: a node/TS agent server (src/agent/) built on @strands-agents/sdk
`Agent`/`tool` wrapped in @ag-ui/aws-strands `StrandsAgent` and served via
@ag-ui/aws-strands/server (`createStrandsApp`/`addStrandsExpressEndpoint`),
modeled on the upstream ag-ui aws-strands TS example server and the
langgraph-typescript infra. A single shared agent at "/" serves most demos
(tools, shared state via toolBehaviors/stateContextBuilder, HITL,
sub-agents), with tool-free specialized agents mounted at /voice,
/byoc-hashbrown, /byoc-json-render. model-factory targets OpenAI chat
completions and honors OPENAI_API_KEY / OPENAI_BASE_URL so it works behind
the showcase aimock proxy. Node-based Dockerfile + entrypoint run the agent
server (:8000) alongside the Next.js frontend.
Frontend mirrors the strands (Python) sibling's demo set and the
langgraph-typescript conventions, with HttpAgent routes proxying to the TS
agent server.
Scope: base integration + standard demos only. A2UI / declarative-gen-ui /
a2ui-fixed-schema is intentionally excluded (no A2UI agents, routes, demos,
or deps) and layered on later.
Platform wiring (mirrors langgraph-typescript): docker-compose local/dev
services on host port 3119, local-ports.json, packages.json, slug-map.ts
(born-in-showcase), showcase_build.yml matrix + path filter + metadata,
shell-docs/dashboard registries, and a logo asset. The python strands
integration is untouched.
Resolve event-renderer.ts onRunFinishedEvent: keep the native turn stream open
(finalized in finish()) AND retain the legacy per-message stream drain from main
(#5573) as a no-op-in-native safety net. app/index.ts (telegram adapter from
#5520 + showToolStatus:false) and create-bot.test.ts auto-merged.
- showcase-canonical-pins.json: bump canonicalCopilotKitVersion 1.59.4 -> 1.60.2;
remove ms-agent-harness-dotnet override (caught up to canonical in prior commit).
- fail-baseline.json: re-ratchet validatePinsFailCount 39 -> 38 and hash to match
the one-item drop (ms-agent-harness-dotnet override no longer counted).
- @ag-ui/mastra: revert 0.2.4 -> 0.2.1-beta.2. 0.2.4 imports
'@mastra/core/runtime-context' which the pinned @mastra/core@1.41.0 does not
export, breaking 'next build' (failing mastra build-check in CI). Holding
@ag-ui/mastra at the prior pin until a coordinated @mastra/core upgrade lands.
Aligns dependency versions across all 19 showcase integrations to current
released minor versions for the 1.60.2 release cycle.
Package families:
- @copilotkit/{a2ui-renderer, react-core, react-ui, runtime, shared, sdk-js, voice}
1.59.4 -> 1.60.2 (18 integrations already staged; ms-agent-harness-dotnet
catches up from 1.57.2)
- @ag-ui/{client, core, encoder} 0.0.55 -> 0.0.57
- @ag-ui/mastra 0.2.1-beta.2 -> 0.2.4 (stable on 0.x; 1.0.x major held back)
Includes the previously-missed ms-agent-harness-dotnet integration in the
@copilotkit/* bump, plus the @copilotkit/web-inspector override pin.
Lockfile-only reconciliation via npm install --package-lock-only
--legacy-peer-deps (cmdk@0.2.1 pre-existing react^18 peer-dep is unaffected).
Pre-existing crewai backend defect: tool-rendering-custom-catchall agentId was routed to the shared LatestAiDevelopment ChatWithCrewFlow at /, with no get_weather/get_stock_price handlers. tool_result never returns, second LLM call never fires, toolCallId-gated narration fixture entries unreachable, A7 requireContentPhrase=true probe fails.
Fix: add get_stock_price_impl in src/agents/tool_rendering.py mirroring LGP-python tool shape; re-route 'tool-rendering-custom-catchall' agentId in src/app/api/copilotkit/route.ts to createAgent('/tool-rendering').
Local verification on cr5495/fix-a14-crewai-green:
- /tmp/cr/a14-green-crewai.log: 1 passed (6.8s), INNER_EXIT=0
- /tmp/cr/a14-green-lgp.log (LGP regression): 1 passed (9.0s), INNER_EXIT=0
(cherry picked from commit 208d90018a3fc41c263fd4514b104c78815f8303)
Each integration's interrupt-headless demo defines a local useHeadlessInterrupt
hook around the framework useInterrupt. Slot-2 originally identified 8
quarantined integrations (claude-sdk-typescript, langgraph-{fastapi,python,
typescript}, langroid, pydantic-ai, spring-ai, strands); review-round
follow-ups extended the sweep to llamaindex, mastra, ag2, agno, and
crewai-crews (5 more integrations sharing the same byte-identical hook).
The demo-local resolve() previously fire-and-forgot copilotkit.runAgent(...)
via `void runAgent(...).catch(() => {})`. Mirroring the framework fix:
- Make resolve async, return await copilotkit.runAgent(...).
- Use a pendingRef so resolve has stable identity (drop pending from
useMemo deps).
- Type signature: resolve: (response: unknown) => Promise<unknown>.
- Wrap in try/catch + setPending(null) + console.error + rethrow,
symmetric with the framework hook.
- onRunFailed also setPending(null).
13 integrations patched byte-identically.
- Scope clickPill locator to data-message-role='user' bubble so the pill
button itself can no longer satisfy the dispatch guard
- Dedup clickPill retry: skip click if the user bubble already exists
- Hero pill: assert declarative-card count=0 (OSS-136 no-Card rule),
metric count >=4 (was >=3 — KPI strip is 4 tiles per composition rule)
- At-risk pill: assert no chart and no table testids (composition rule)
- Top-account pill: assert no data-table and no status-badge testids
- Rename hero test title to 'KPI strip + pie + bar (no surrounding card)'
so the title no longer falsifies the body
- QA docs: replace 'card + metrics + pie + bar' Expected Results with
'4 KPI metrics + 1 PieChart + 1 BarChart, no surrounding Card per OSS-136'
- Probe responseTimeoutMs derived from FIRST_SIGNAL_TIMEOUT_MS so it
matches the e2e 90s budget
- DataTable rowKey uses first-column value + index instead of bare index,
with JSON.stringify(row) fallback (stops re-mount on dynamic A2UI re-emits)
- Card emits data-card-id={props.title} so multi-card pills no longer
collide on a single declarative-card testid
- PieChart/BarChart value coercion replaced 'Number(x) || 0' with
finite-number check + console.warn on drift (no longer masks legitimate 0)
Jordan Ritter