The matrix tab can freeze the browser on load and during update bursts
because every PB SSE delta triggers a full re-render of all ~720 cells
(18 integrations × ~40 features), and PocketBase fires the subscribe
callback once per record — a probe finishing dozens of services or an
initial-state replay produces N consecutive React commits.
- ComposedCell: wrap in React.memo with a custom equality check that
compares overlays/catalogCell refs, ctx scalars, and (when the
liveStatus Map identity changes) only the 5 row keys this cell
actually reads. Because upsertByKey preserves row identity for
unchanged keys, deltas that don't touch a cell's slug/featureId
short-circuit at the memo boundary.
- useLiveStatus: buffer SSE callbacks into a per-key Map and flush via
a single 16ms setTimeout. A burst of N deltas now produces 1 React
commit instead of N. Last-write-wins per key. Buffer is cleared on
teardown and on reconnect kickoff so post-reconnect initial fetches
never land on top of stale buffered rows.
When a dashboard cell has no probe data for a depth (D5, D6), render
the dimension name with CSS line-through instead of appending a "?"
label. Strikethrough communicates "this depth doesn't exist / hasn't
run" more clearly than "D5 ?".
Changes:
- Badge component: when label is "?", render name with line-through
styling and suppress the question mark
- CellDrilldown BadgeRow: render "n/a" with line-through instead of "?"
- Update cell-drilldown test to verify strikethrough rendering
## Summary
Fixes the root causes for **30+ live-dashboard cells stuck at D2**
across 31 integration rows on https://dashboard.showcase.copilotkit.ai/.
Five surgical fixes plus one structural depth-walk fix; deploy-stale
cells (~40) are flagged separately for ops since they are not
code-fixable.
## Why these fixes ship together
A scrape of the live dashboard found 100 D2 chips. Bucketing them by
failure mode revealed the pattern is **structural + per-framework**, not
100 independent failures. Five frameworks ship genuine probe/integration
bugs we can fix in code; three frameworks are deploy-stale (Railway
image is older than the recent commit landings); one cell is a
per-feature slot-API contract bug. The dashboard itself also has a
structural bug that under-reports cells whose D5 is green when D3 has no
row yet.
## What ships
### Slot-agent fixes (Phase A/B/C of the blitz)
| Commit | What |
|---|---|
| `1de1556be` D5-green bypass for missing/red D3 in depth walk |
`deriveDepth` in
`showcase/shell-dashboard/src/components/depth-utils.ts` was
contiguous-walk-only, so 13 cells whose `d5:<slug>/<feature>` row is
green but whose `e2e:<slug>/<feature>` row is missing or red were stuck
at D2. Per the harness model, the e2e-deep driver only emits D5 after D3
+ D4 have passed at some point; D5-green is sufficient evidence. After
the D2 (agent) gate passes, `deriveDepth` now checks D5 first; if green,
achieved jumps to 5 and proceeds to D6. D1+D2 remain hard gates - D5
cannot bypass health/agent. 7 new tests pin every asymmetry. |
| `e0d89e0fe` fail loud on slug missing from registry in e2e-demos
driver | `E2eDemosResolver` previously returned `E2eDemoEntry[]`,
conflating "registry has the slug with zero demos" (legitimate brand-new
package) with "slug absent from registry" (operational fault - manifest
gap, stale mount, slug rename). Both collapsed silently into "aggregate
green / no per-feature rows" - exactly the symptom shape on the live
dashboard for several frameworks (`mastra`, `langroid`, `llamaindex`,
`langgraph-typescript`, `langgraph-fastapi`,
`ms-agent-{python,dotnet}`). Resolver now returns `{ present: boolean,
entries: E2eDemoEntry[] }`; when `present === false`, the driver emits a
synthetic red `e2e:<slug>/__missing-registry` side row + flips the
aggregate red with `errorClass: "registry-missing"`. Read-failure path
still goes through the existing green/silent branch to avoid swamping
the dashboard with N noisy red dots when the registry isn't mounted at
all. |
| `00e801209` claude-sdk-python missing openai dep crashed agent on
import | `src/agents/a2ui_dynamic.py` had a top-level `import openai`
but `openai` was never declared in `requirements.txt`. `agent_server.py`
imports `a2ui_dynamic` at module load, so `ModuleNotFoundError` cascaded
through `entrypoint.sh`'s "Agent failed to start - exiting" gate,
preventing Next.js from booting at all. Result: every `/demos/<id>`
route across the integration was unreachable - explaining all 9 red E2E
cells in the `claude-sdk-python` framework column uniformly. Two-line
fix: add `openai>=1.50.0` to `requirements.txt` and move `import openai`
to a lazy import inside `_generate_a2ui` (mirrors the existing pattern
in `agents/agent.py`). |
| `cb2ef9411` remove __future__ annotations from ag2 agent_config_agent
| Same class of bug as commit `e38fab4d6` (Apr 28) which fixed
`shared_state_read_write.py` and `subagents.py`: `from __future__ import
annotations` turns `ContextVariables` in the `@tool`-decorated
`get_current_config(context_variables: ContextVariables) -> str`
signature into a ForwardRef. Autogen's `TypeAdapter` can't resolve it,
registration raises `PydanticUserError` at module import,
`agent_server.py` fails to load, the whole AG2 integration goes
unreachable. Drop the `__future__` import. |
| `b377b78a6` render input slot in google-adk chat-slots welcome screen
| The custom welcome screen at
`src/app/demos/chat-slots/custom-welcome-screen.tsx` did not accept or
render the `input` ReactElement that `CopilotChatView` passes into the
`welcomeScreen` slot. `CopilotChatView` only mounts the chat composer
inside the welcome screen on the empty-thread path
(`CopilotChatView.tsx:301-348`), so when the empty-state welcome was
active no `<textarea>` ever reached the DOM. The e2e-readiness probe's
structural selectors all timed out, flipping the D2 chat-slots cell red.
Mirrors the canonical pattern used by 16+ other integrations (strands,
langgraph-python, claude-sdk-python). |
### cr-loop fixes (Round 1+2 review findings, applied during
convergence)
| Commit | What |
|---|---|
| `b1768f5db` return red aggregate when e2e-readiness resolver throws |
Round 1 finding (3 of 7 reviewers converged independently). When
`demosResolver(slug)` throws, the catch path was emitting a `__resolver`
red side row but `slugPresentInRegistry` defaulted to `true` and was
never flipped, so the fail-loud branch was skipped and the aggregate
fell through to green with `note: "no demos declared"`. Catch now
returns early with a red aggregate (`errorDesc: "resolver-error"`),
mirroring the missing-registry early-return shape. New C12b test pins
the aggregate state, since the existing C12 test only asserted the side
row. |
| `44037b9ce` align claude-sdk-python requirements.txt comment with
lazy-import implementation | Round 1 trivial - the comment block above
the `openai` dep claimed it was "imported at module top". The
`00e801209` fix moved the import to a lazy import; the comment was now
falsified by the diff. Updated to describe the actual two-layer
protection (lazy import as runtime safety net, requirements.txt as
authoritative dep declaration). |
| `070ecd051` empty in-band input.demos no longer suppresses
missing-registry red | Round 2 finding. The in-band fallback `if
(demos.length === 0 && Array.isArray(input.demos))` triggered even when
`input.demos === []`, forcing `slugPresentInRegistry = true` and
silently bypassing the fail-loud branch. A misconfigured probe YAML with
`demos: []` for a slug not in the registry would have rendered green.
Tightened to also require `input.demos.length > 0`. |
## Live-dashboard impact
After deploy:
- **13 cells** whose `d5:<slug>/<feature>` is already green flip to D5
immediately (purely from the depth-walk fix; no probe/agent change
required).
- **30+ cells** across `mastra` / `langroid` / `llamaindex` /
`langgraph-typescript` / `langgraph-fastapi` / `ms-agent-python` /
`ms-agent-dotnet` will surface as red `__missing-registry` rows on the
next probe tick (the previously-silent missing-registry fault becomes
loud and actionable).
- **9 cells** in `claude-sdk-python` flip green once the integration
container rebuilds with the openai dep.
- **15 cells** in `ag2` framework column should flip green once the
agent server boots cleanly (the `agent_config_agent` import crash was
preventing `agent_server.py` from loading at all).
- **1 cell** `google-adk` × `chat-slots` flips green on next deploy
(welcome screen now renders the chat composer).
## Deploy-stale cohort (NOT code-fixable, flagged for ops)
| Framework | D2 cells | Status |
|---|---|---|
| `built-in-agent` | 28 | **Needs Railway redeploy** - local `next
build` succeeds for all 53 routes; live
`https://showcase-built-in-agent-production.up.railway.app` returns HTTP
404 for every demo added in commits `e18ee1d25 e4afffd4f7abb1d07674e57483869cb5f51c7c44b71f6 53455bfeb`. Source healthy; deploy is
older than the demo-landing commits. |
| `claude-sdk-typescript` | 10 | **Needs Railway redeploy** - same
pattern: local build green, live URL returns 404 for newly-added demos
(`b9dbacd99 / 70522b1a6 / 417e3b397`). |
| `ms-agent-python` | 2 | Probable snapshot-stale - live HTML now
renders the canonical `data-testid="copilot-chat-textarea"` selector;
the readiness scan was probably running before the redeploy completed.
Should self-heal on next probe tick. |
Slack thread to Jordan with the `built-in-agent` deploy-stale evidence
already sent earlier in the run.
## Test plan
- [x] `pnpm --filter @copilotkit/showcase-harness exec vitest run
e2e-readiness` - 34/34 pass (32 pre-existing + 1 new C12b resolver-throw
aggregate-red + 1 new empty-in-band missing-registry)
- [x] `npx vitest run depth-utils` (in `showcase/shell-dashboard/`) -
30/30 pass (23 pre-existing + 7 new D5-bypass cohort)
- [x] `pnpm typecheck` on `showcase/harness` - clean
- [x] `npx tsc --noEmit` on `showcase/shell-dashboard` - touched files
(`depth-utils.ts`, `depth-utils.test.ts`) clean. Pre-existing TS errors
in `compute-tally-detail.test.tsx` are on `main`, not introduced by this
PR.
- [x] cr-loop converged on Round 3 (7 unbiased reviewers, 0 bucket (a)
findings, byte-identical verbatim prompts)
- [ ] CI green on PR HEAD
- [ ] After merge: ops triggers Railway redeploy of
`showcase-built-in-agent-production` and
`showcase-claude-sdk-typescript-production` - those 38 cells flip green
without any code change.
## Out of scope (deferred to follow-up PRs)
A 7-agent unbiased CR surfaced ~50 pre-existing concerns in files this
PR happens to touch but whose subject is distinct (bucket (d) per
cr-loop's classification rules). They have coherent, named subjects and
belong in their own PRs:
- **a2ui_dynamic.py agent observability audit** (claude-sdk-python):
broad `except Exception` leaks tracebacks to chat content; sync `openai`
call in async generator blocks the event loop; Anthropic-format messages
forwarded to OpenAI's `chat.completions.create` will 400 on the second
tool round; no max-iteration cap on the outer `while True` loop;
empty-string `ANTHROPIC_API_KEY` fallback masks misconfig;
`tool_call_id` falls back to empty string.
- **e2e-readiness probe robustness audit**: `Math.max(0, deadline -
Date.now())` returns `0` after deadline expiry, which Playwright treats
as "no timeout" instead of "fail immediately"; whitespace-only `route`
strings bypass the `config-invalid` guard; URL concatenation does not
normalize trailing/leading slashes; no abort check between `newPage()`
and `page.goto()`; `setTimeout` overflow on huge `E2E_DEMOS_TIMEOUT_MS`
env values; `__resolver` / `__missing-registry` sentinel namespacing
speculation.
- **e2e-readiness test hygiene**: the C7 "aborts mid-selector-loop
without walking all 6 selectors" test asserts behavior the
implementation no longer has (single compound selector replaced the
sequential loop); the assertion
`expect(selectorsTried.length).toBeLessThan(5)` is now trivially true.
- **depth-walk semantics audit**: D6 unreachable when `isD5Green` is
false (D6 was always gated on D5-green even before this PR; design
discussion on whether D6-green should imply D5-green - the same way
D5-green now implies D3/D4-pass).
- **agent_config_agent observability**: invalid frontend values
(`tone="snarky"`) silently coerce to defaults with no log; `logger`
instance imported but never used.
## Worktrees / cleanup
- Integration worktree at
`<repo>/.claude/worktrees/blitz-d2-to-d4-integration` retained alive
across cr-loop rounds per blitz protocol; will be removed after this PR
merges.
- 8 per-slot blitz worktrees and 2 ephemeral cr-fix worktrees were
created; 7 cleaned up successfully, 3 had Windows file-lock issues
(built-in-agent, claude-sdk-typescript, ms-agent-python) - git metadata
is gone, just stale dirs that need a manual `rm -rf`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
historical probe services
Running panel and historical run chips now show result-specific
icons: green=checkmark, yellow=warning, red=X. Previously all
completed services showed a checkmark regardless of result, making
it look like failing tests passed. Also fix inflight tally to count
yellow (degraded) results as failures alongside red.
overlays, and probe drilldown
Replace replaceState with pushState for user-initiated navigation
(tab switches, overlay toggles, preset changes, probe selection).
Add popstate listener to sync React state on back/forward. Encode
probe detail drilldown in URL hash (#ops:probe=<id>) so the back
button closes the detail panel and returns to the ops overview.
## Summary
- Persist per-service probe results from the tracker snapshot into
PocketBase `summary.services` when a run finishes
- Dashboard historical run rows with service data show a chevron —
clicking expands a detail row with service chips matching the inflight
panel pattern
- Existing runs (pre-deploy) render without the expand affordance; only
new runs populate service data
## Test plan
- [x] Verify new runs after harness deploy populate `summary.services`
in PocketBase
- [x] Verify historical run rows with service data show expand chevron
- [x] Verify clicking chevron expands/collapses the service chip grid
- [x] Verify old runs (no service data) render without chevron as before
Persist per-service probe results from the tracker snapshot into
PocketBase summary.services when a run finishes. Dashboard runs-list
rows with service data now show a chevron — clicking expands a detail
row with a service chip grid matching the inflight panel's pattern.
Existing runs (before this deploy) won't have service data and will
render without the expand affordance.
Two bugs in DocsRow's link construction surfaced after PR #4395 + #4419:
1. **No feature-default fallback for shell_docs_path.** The component
only read `integration.docs_links.features[fid].shell_docs_path`. If
no per-integration override existed, the link rendered as 'missing'
even when the feature-registry had a valid default. The og_docs_url
path already had this fallback; shell_docs_path didn't.
Fix: same `hasOverride ? override : feature_default` chain that
og_docs_url uses. Explicit `null` overrides still mean opt-out
(rendered with the 'opt-out' tooltip variant). Missing override
inherits the feature default.
2. **Obsolete /unselected/ prefix in URL construction.** The shell-docs
IA restructure (commit c11976819) retired the `unselected/`
directory. The dashboard was still building URLs as
`<framework>/unselected<path>`. Shell-docs's route handler tolerates
the legacy shape (it strips the prefix), but the URLs are
misleading. Dropped the SHELL_UNSELECTED_PATH constant and emit
canonical `<framework><path>` URLs.
After: 0 broken (404) shell-docs links across 720 cells (was 8). 577
cells now resolve cleanly to a docs page; the remaining 143 are
features that genuinely don't have a shell-docs page yet (some have
upstream docs.copilotkit.ai pages, some don't — see follow-up ticket).
The contiguous depth walk in deriveDepth previously short-circuited at
D2 whenever the D3 (e2e) row was missing or red, even when the D5
(e2e-deep) row was green. Per the harness model the e2e-deep driver
only emits a green d5:<slug>/<featureType> row after D3 + D4 probes
have passed, so a green D5 is sufficient evidence that D3/D4 implicitly
passed at some point.
After the D2 (agent) gate now passes, deriveDepth checks D5 first. If
D5 is green it sets achieved = 5 and proceeds to the D6 check; otherwise
it falls through to the existing D3 -> D4 -> D5 walk. D1 and D2 are
still hard gates -- D5 cannot bypass health/agent.
Adds 7 new tests covering: D3-missing+D5-green, D3-red+D5-green,
D3-missing+D5-missing (unchanged D2), D1-red+D5-green (must stay D0),
D2-red+D5-green (must stay D1), D6 via D5-bypass, and isRegression
when bypass lifts cell from D2 to D5 at max_depth=5.
Resolves the 13 live-dashboard cells (8 with e2e=? + 5 with e2e=red)
that should have shown D5 but were stuck at D2.
## Summary
- When a probe is running, the summary table now computes a live tally
from `inflight.services` instead of showing stale results from the last
completed run.
- Last Run and Duration columns also update to reflect the in-progress
run's start time and elapsed duration.
- Result shows "X/Y pass — running" while inflight, with amber tone for
in-progress and red if any failures detected.
## Test plan
- Trigger a probe run, watch the summary row update in real-time as
services complete.
- Once the run finishes, verify the row switches back to showing the
completed run's final results.
## Summary
- Previous 94%/6% color-mix was invisible (white vs near-white). Changed
to 50/50 mix between `--bg-surface` and `--bg-muted` for actually
visible alternating rows.
## Test plan
- Alternating rows should have a subtle but visible tint difference.
## Summary
- Adds subtle alternating row backgrounds to the feature matrix using
`color-mix(in srgb, var(--bg-surface) 94%, var(--bg-muted))` on odd
rows. Sticky feature-name column matches the stripe. Theme-aware via CSS
variables.
## Test plan
- Visual: alternating rows should have a barely-noticeable tint
difference.
- Sticky column background should match the row stripe when scrolling
horizontally.
## Summary
- Makes the fixed legend footer collapsible via a toggle button. Default
is open. On mobile the legend was taking up most of the screen, making
the table unusable.
## Test plan
- Click the "Legend" toggle — content collapses to a thin bar. Click
again to expand.
- Verify on mobile viewport — collapsed state leaves room for the table.
Per user preference, restore the 🚫 emoji as the unsupported marker in
both DepthChip and feature-grid. Keeps a slate-colored pill (vs the
dashed-gray unshipped chip and the red ✗ no-demo marker) so the three
states remain visually distinct.
The matrix view's links/depth overlays are rendered by feature-grid.tsx,
which had its own no-demo branch that displayed a hard ✗ in danger color
when integration.demos didn't contain an entry for the feature.
That branch fired even when the feature was architecturally unsupported
(in not_supported_features), because unsupported integrations don't add
the feature to their demos[] list. Result: 29 unsupported cells rendered
as a red ✗ titled 'No demo' — visually identical to the unshipped marker
the user is trying to distinguish from.
Now feature-grid checks integration.not_supported_features first; if the
feature is listed there, the cell renders the same slate N/A pill we use
in DepthChip (consistent with the unsupported branch added there in the
prior commit). The hard ✗ remains for genuine no-demo cases.
Also adds not_supported_features to the Integration TypeScript interface
in registry.ts (the field was already present in registry.json data;
the type just didn't expose it).
The 🚫 emoji falls back to a tofu/X-shaped glyph on systems without
color emoji support (notably Windows Chrome with default fonts), which
viewers misread as the unshipped X marker — making 29 unsupported cells
look indistinguishable from unshipped.
Switched to a slate-colored N/A pill with a distinct border so the
unsupported state reads clearly across all platforms regardless of emoji
font availability.
- generate-registry: cross-validate not_supported_features against features list
- catalog-types: widen manifestation union to include 'starter'
- claude-sdk-python: drop invalid 'declarative-schema' from generative_ui enum
- crewai-crews: remove duplicate animated_preview_url key in mcp-apps demo
- built-in-agent + pydantic-ai: REASONING_MODEL env-var fallback so deployers can swap if gpt-5.2/gpt-5 isn't available
Adds a fourth cell status, "unsupported", to the showcase dashboard for
features whose framework cannot architecturally support them (e.g.,
no graph-interrupt API, no MCP tool runtime). This is distinct from
"unshipped" (just unbuilt).
- Manifest schema gains optional `not_supported_features: string[]`.
- generate-registry.ts: determineCellStatus checks `not_supported_features`
first; CatalogCell/CatalogMetadata gain `unsupported`; max_depth=0 for
unsupported cells; parity tier is computed against the supportable subset
of the reference (unsupported features no longer drag a framework's tier
down for gaps it legitimately cannot fill).
- depth-utils.ts: deriveDepth returns D0 with no regression for
unsupported cells.
- DepthChip: renders a dashed gray border + 🚫 glyph with
"Not supported by this framework" tooltip; data-status attribute
distinguishes unshipped vs unsupported.
- CoverageBar gains diagonal-stripe gray segment for unsupported.
- StatsBar / AdaptiveStatsBar / CellsView: separate "Unsupported" chip.
- CellMatrix gaps filter excludes unsupported cells (not work to do).
- Tests: cover unsupported in DepthChip, depth-utils, cell-matrix, and
catalog generator metadata/max_depth assertions.
Command demos (npx copilotkit init) have no E2E/D5/D6 probes, so
health badges were perpetual gray noise. Skip HealthLayer when the
demo declares a command instead of a route.
Restructure page to viewport-filling flex layout so table column
headers stick below the overlay toggle bar on scroll. Halve cell
padding, reduce column min-width, center UWCT/health badges and
links. Remove duplicate CellStatus from CommandCell (ComposedCell
HealthLayer handles it). Split stats bar out of sticky container
so it scrolls away naturally.
Depth system was capped at D4 — D5/D6 PocketBase rows existed but
the chip never advanced past D4. Extends deriveDepth() through D6
with proper short-circuit gating, adds isD5Green() for multi-key
CATALOG_TO_D5_KEY resolution, and colors D5/D6 chips emerald to
distinguish from D3/D4 blue.
Dashboard: resolveCell() looked up D5 rows by catalog feature ID
(e.g. d5:slug/hitl-in-chat) but the harness writes them keyed by D5
feature type (e.g. d5:slug/hitl-text-input). Added CATALOG_TO_D5_KEY
mapping so the dashboard resolves the correct PB row for each cell.
CLI: buildDeepInputs() passed manifest registry IDs as `features` but
the D5 driver only maps them through demosToFeatureTypes() when they
arrive as `demos`. Changed to pass `demos` so the driver's mapping
from registry IDs to D5 feature types activates correctly.
Renders ComposedCell with real CellStatus, DocsRow, DepthChip, and Badge
components (only data layer mocked) to verify that overlay toggles
actually control visible content. Covers all 5 overlays individually,
key combinations (Health+Docs, Health-without-Docs), all 3 presets
(Catalog, Assessment, Parity Review), testing-kind D5/D6 hiding, and
the critical regression test proving Health-only no longer leaks docs
indicators after the CellStatus DocsRow removal fix.
Add shared formatTs() utility that converts ISO 8601 UTC strings to
the user's local timezone via toLocaleString(). Apply it to all raw
timestamp interpolation sites in tooltip/badge/drilldown rendering:
- live-status.ts formatTooltip() (4 sites)
- cell-pieces.tsx formatTransitionLine() (3 sites)
- level-strip.tsx resolveBadge() (1 site)
- cell-drilldown.tsx refactored to use shared formatTs()
Update existing live-status tests to expect formatted timestamps.
Add unit tests for formatTs() covering valid/invalid inputs.
CellStatus unconditionally rendered DocsRow for non-testing features,
causing docs indicators (docs-og, docs-shell) to appear in matrix cells
whenever the Health overlay was active — even when the Docs overlay was
toggled OFF. DocsLayer in ComposedCell already handles docs rendering
correctly, gated on overlays.has("docs"). Remove the redundant DocsRow
from CellStatus so docs indicators respect the overlay toggle.
Remove conditional early return that rendered different DOM structures
on server vs client (bare fragment vs div>button wrapper). All hooks
now run unconditionally; interaction is disabled via guards when items
array is empty. Fixes React errors #418 and #310.
TallyTrigger wraps each tally count (green/amber/red) in OverlayColumnHeader
with hover-open (200ms delay) and click-pin semantics. The popover lists
which features/signals fall in each bucket with colored dots and dimension tags.
Add companion function to computeColumnTally() that returns per-bucket
TallyItem arrays instead of numeric counts. Wire a tallyDetails memo
into FeatureGrid and pass tallyDetail prop to OverlayColumnHeader.
Includes 6 unit tests covering connection error, health routing,
e2e tone bucketing, and demo-less feature filtering.
Alem Tuzlak