Commit Graph

217 Commits

Author SHA1 Message Date
github-actions[bot] c0fb61e774 style: auto-fix formatting 2026-05-14 22:06:20 +00:00
Jordan Ritter a1b7cc7c4e feat(showcase/dashboard): add system health banners for auth and browser pool
DiscoveryAuthBanner renders above all tabs when discovery auth fails.
Two variants: serving-stale (probes running against cached data) and
no-cache (probes offline). Also surfaces browser pool degradation.
Runtime signal shape validation, auto-dismiss on recovery. 12 tests.
2026-05-14 15:05:15 -07:00
Jordan Ritter 25a76275e7 fix(showcase): red chipColor when tests exist but all fail
Previously achievedDepth=0 always produced gray regardless of whether
tests existed. Now: ceilingDepth=0 (no tests) = gray, ceilingDepth>0
with achievedDepth=0 (tests exist, all fail) = red. Tally dimension
derived from model instead of hardcoded "e2e".
2026-05-13 23:50:05 -07:00
Jordan Ritter 686286f771 fix(showcase): remove U/W/C/T badges and fix dashboard type hygiene
Remove misleading header badges that read integration-level probes
independent of per-feature cell data. Replace 5 duplicate local
Overlay types with canonical import. Remove dead connection prop.
Add exhaustive state handling in level-strip. Remove redundant
?? false in isSupported expressions.
2026-05-13 23:45:26 -07:00
Jordan Ritter 57d26314da fix(showcase): derive column header tallies from buildCellModel
Tallies now count by CellModel.chipColor instead of resolveCell rollup,
ensuring header numbers match what cells actually render. Gray cells
(no data) are excluded from counts.
2026-05-13 22:55:21 -07:00
Jordan Ritter f7fde425d9 fix(showcase): wire unified cell model into Coverage and Baseline tabs
Coverage tab uses buildCellModel + UnifiedCell. Baseline tab CellMatrix
migrated from deriveDepth to buildCellModel. Fixes three dashboard bugs:
D0 shown despite passing tests, yellow D4 at ceiling, no-entry icon
alongside test badges. ComposedCell and deriveDepth deprecated.
2026-05-13 22:11:04 -07:00
Jordan Ritter 475cd7c991 feat(showcase): add UnifiedCell with chipColor-driven DepthChip
DepthChip accepts pre-computed chipColor prop (green when achieved equals
ceiling). UnifiedCell is the single rendering codepath: unsupported cells
show only the no-entry icon, badges render only for existing test levels.
arePropsEqual synced with buildCellModel reads (e2e/chat/tools/d5 keys).
2026-05-13 22:10:58 -07:00
Jordan Ritter 4d57d7b869 feat(showcase): add CellModel type and buildCellModel() for unified cell rendering
Single source of truth for Coverage-tab cell state. Replaces fragmented
depth/badge resolution. Resolves D3/D4/D5 test existence and status
independently, computes contiguous ceiling depth and chip color relative
to ceiling (green at ceiling, gray for no data, amber/red below).
2026-05-13 22:10:51 -07:00
Jordan Ritter b6ad624c94 style: apply formatter to vue package and other unformatted files 2026-05-13 17:06:47 -07:00
Tyler Slaton 70e2fb13c8 refactor(showcase): rename byoc-* slugs to declarative-* + sort index by manifest features
User-facing renames so the showcase reads the way a cold visitor would
expect:

- `byoc-hashbrown` → `declarative-hashbrown` (and `byoc-json-render` →
  `declarative-json-render`). The display titles already said
  "Declarative UI: …"; only the URL slugs and folder paths still
  leaked the internal BYOC ("Bring Your Own Components") jargon.
  Renamed:
    /demos/byoc-hashbrown          → /demos/declarative-hashbrown
    /demos/byoc-json-render        → /demos/declarative-json-render
    /api/copilotkit-byoc-*         → /api/copilotkit-declarative-*
    src/app/demos/byoc-*           → src/app/demos/declarative-*
    qa/byoc-*.md                   → qa/declarative-*.md
    tests/e2e/byoc-*.spec.ts       → tests/e2e/declarative-*.spec.ts
  Internal Python module names + langgraph graph IDs stay legacy
  (`byoc_hashbrown_agent.py`, `byoc_hashbrown`) — those are not
  user-facing and renaming them is a separate cross-codebase pass.
- `a2ui-fixed-schema` slug intentionally unchanged.
- Tool Rendering trio parenthetical rename (Default → Catch-all →
  Custom progression reads clearly as "how much do I customize?"):
    Tool Rendering (Default)        — unchanged
    Tool Rendering (Custom default) → Tool Rendering (Catch-all)
    Tool Rendering (Specific)       → Tool Rendering (Custom)
- `tool-rendering-reasoning-chain` cell renamed from
  "Generative UI: Rendering multiple tools" to
  "Generative UI: Tool calls + reasoning" (the demo is about combining
  reasoning + tool rendering, not about quantity of tools).
- `Open Generative UI: Default` / `Open Generative UI: Custom`
  descriptions expanded so a visitor understands how Open Generative UI
  differs from Tool Rendering (agent composes UI from a registered
  library vs. attaching a renderer to a *named* backend tool).
- Showcase index now sorts demos within each tag by `manifest.features`
  order. Previously demos appeared in manifest declaration order, which
  ignored the team's curated "polished flagship → simplest start →
  variants" arc.

Cross-cutting registry / harness / dashboard updates that fall out of
the rename:

- `shared/feature-registry.json` adds the two new IDs alongside the
  legacy `byoc-*` (so the catalog stays valid; the other 17
  integrations still declare `byoc-*` in their manifests).
- `shared/constraints.yaml` adds the new IDs to the
  generative-ui-approach allow-list.
- `scripts/__tests__/generate-catalog.test.ts` updates the cell-count
  expectations (45 features × 18 integrations = 810; 792 after docs-
  only exclusion; 45 LGP cells = 38 wired + 1 stub + 6 unshipped).
- Harness probe `d5-byoc.ts` + `d5-byoc.test.ts` now route both slug
  families through `preNavigateRoute` and exercise the new branches.
- `d5-feature-mapping.ts` and `shell-dashboard/live-status.ts` mirror
  the dual-ID mapping so both legacy and renamed slugs roll up under
  the same `byoc` D5 featureType.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 15:16:02 -07:00
Tyler Slaton 80a7f9af0e feat(showcase): align demo names + add Show Deprecated toggle
Two related changes that bring the dashboard's gold-standard view in
line with the desired naming convention and surface deprecated rows
behind a toggle (instead of hiding them at catalog generation).

## Naming alignment

Applied 28 renames in feature-registry.json + 20 in LGP manifest per
the user-provided mapping. Highlights:

- "Pre-Built CopilotChat" -> "Pre-Built: CopilotChat"
- "Headless Chat (Simple/Complete)" -> "Headless UI: Simple/Complete"
- "Multi-modal / File Uploads" -> "Attachements" (intentional spelling)
- "Controlled Gen-UI (Display)" -> "Generative UI: useComponent"
- "In-Chat HITL (use*)" -> "Human In/in the Loop: In-chat / Interrupts"
- "Headless Interrupt" -> "Human in the Loop: Headless Interrupts"
- "Declarative Generative UI (A2UI - *)" -> "Declarative UI: */* A2UI"
- "Fully Open-Ended Generative UI" -> "Open Generative UI: Default"
- "Tool Rendering ..." -> "Generative UI: Tool Rendering (...)"
- "Tool Rendering + Reasoning Chain" -> "Generative UI: Rendering multiple tools"
- "Agentic Generative UI ..." -> "Generative UI: Agent State"
- "Frontend Tools (...)" -> "Frontend Tools: ..."
- "Shared State (...)" -> "Shared State: ..."
- "State Streaming" -> "Shared State: Streaming"
- "Readonly State (Agent Context)" -> "Shared State: Frontend Context"
- "BYOC Hashbrown <-> json-render" -- labels intentionally swapped per
  user instruction (demos were historically reversed; new labels
  reflect what they actually do).

LGP manifest demos[].name updated to match feature-registry names so
the dojo and dashboard surface the same human-readable label.

## Show Deprecated toggle (feature-grid.tsx)

Added a checkbox in the matrix header -- default OFF -- that filters
feature rows where `feature.deprecated === true`. Toggle ON shows all
deprecated features across all integrations (audit trail); toggle OFF
hides those rows entirely so the gold-standard view stays clean.

Reverted the catalog-side filter from PR #4744 (which dropped LGP
cells for deprecated features at catalog-generation time). Now the
catalog emits cells uniformly for all (integration x feature) pairs,
and visibility is controlled at the dashboard layer. Toggling on
shows complete cross-integration data without missing-cell artifacts.

Affects 4 features marked deprecated:true in feature-registry.json:
agentic-chat-reasoning, hitl, hitl-in-chat-booking,
reasoning-default-render.

LGP cell count: back to 43 (38 wired + 1 stub + 4 unshipped). The 4
unshipped rows are hidden by default; toggle to surface them.

Tests: 18/18 catalog tests + 1588/1588 harness vitest passing.
validate-fixture-tool-surface clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 20:06:39 -07:00
Tyler Slaton 256dcd5a61 test(shell-dashboard): refresh depth-utils tests post dual-claim drop
The previous "multi-key D5 mapping" tests used `shared-state-read-write`
as the example, but with the dual-claim drop that registry-id is now
single-key (only `shared-state-write`). Tests still passed logically
but the names were misleading.

- Rewrote both shared-state-read-write tests to assert the new
  single-key contract directly (no more `d5:lgp/shared-state-read`
  row in the live map — that row belongs to a different cell now).
- Added new beautiful-chat tests as the canonical multi-key example
  (5 per-pill literals) so the multi-key code path stays exercised.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 17:48:07 -07:00
Tyler Slaton be94bd7a6f feat(showcase): add 3 LGP D5 probes + driver retry-once
Closes the demo↔probe coverage gap for /demos/{interrupt-headless,
shared-state-read, tool-rendering-reasoning-chain} so every demo
under langgraph-python (the north-star integration) now has a D5
probe writing to its own PocketBase cell — not relying on cross-
demo umbrella records.

New probes (multi-turn, mirroring the agentic-chat structure):
  - d5-interrupt-headless: exercises useHeadlessInterrupt — chip
    prompt → backend interrupt(...) → app-surface popup → slot pick
    → resume → assistant confirmation. Distinct from gen-ui-interrupt
    (which uses inline useInterrupt).
  - d5-tool-rendering-reasoning-chain: combines reasoning-block slot
    + per-tool renderer (WeatherCard, FlightListCard) on the same
    chat surface. Catches a regression in either side.
  - d5-shared-state-read: recipe-editor demo (neutral default agent,
    no tools) — verifies recipe-card form mounts AND agent reads
    shared state across turns. Drops the dual-claim that
    d5-shared-state.ts had on `shared-state-read` (now write-only).

Driver retry-once (e2e-deep.ts):
  Probes that fail with a transient class (`goto-error` /
  `conversation-error`) AND took ≥2s on the first attempt now retry
  once before recording red. Persistent assertion-style failures
  (sub-2s) and intentional aborts/feature-timeouts skip retry —
  retrying a deterministic mismatch just burns clock and obscures
  the signal. Cuts ~10× the dashboard flap rate.

Plumbing:
  - D5FeatureType enum: +interrupt-headless, +tool-rendering-reasoning-chain.
  - REGISTRY_TO_D5 (harness) + CATALOG_TO_D5_KEY (dashboard) mirror
    the new mappings; d5-mapping-drift test enforces this.
  - LGP manifest features + demos entries + constraints allowlist.
  - feature-registry.json: +shared-state-read.
  - aimock d5-all.json: +2 shared-state-read fixtures (interrupt-
    headless + tool-rendering-reasoning-chain reuse existing fixtures
    that already match their chip prompts).

Tests: 1588/1588 harness vitest green. validate-fixture-tool-surface
clean (282 fixtures × 627 demos, no drift). Two pre-existing test
fixes folded in — d5-gen-ui-interrupt assertion mock updated to
match the current evaluate-poll resume signal; conversation-runner
preFill ordering test now asserts the actual deferred-cascade
contract instead of a stricter pre-preFill ban that the runner
never enforced.

Known follow-up (not in this PR): auth.spec.ts test #5 ("signing
back in re-mounts a fresh chat surface") fails on Railway — second
sign-in's "Hello again" never produces an assistant response. Looks
like a react-core/v2 ref-handling regression on <CopilotKit>
unmount/remount; deserves its own focused investigation.

Other integrations may flip red on the new probes — that's
expected. We're treating LGP as the template; cross-integration
parity follows in a separate wave.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 17:22:57 -07:00
Tyler Slaton 80a998d59a fix(showcase): CR Round 1 — resolve precedence + probe robustness
Bucket (a) findings from CR Round 1, fixed inline:

A1. shell-dashboard resolveD5Row precedence
    Multi-key D5 cells (beautiful-chat → 5 per-pill keys) returned
    the first non-null row as worst, only upgrading on red. A degraded
    row encountered after a green row was silently dropped → cell
    rendered green when it should have been amber. Replaced the
    "only red wins" check with a numeric rank table (red=3, degraded=2,
    green=1) so red > degraded > green holds regardless of iteration
    order. Added 5 multi-key fan-out tests covering: red-after-green,
    red-before-green, degraded-vs-green order independence, red-beats-
    degraded, all-green-stays-green.

    Symbols touched: resolveD5Row (live-status.ts:176), new
    D5_STATE_RANK constant. Call-site enumeration: resolveD5Row is
    called only by resolveCell at live-status.ts:371 — same input/
    output shape, no caller change needed.

A2. fixture _comment lies about aimock arg shape
    Both gen-ui-agent.json and shared-state-streaming.json's _comment
    claimed `arguments` MUST be JSON-stringified or aimock silently
    drops the call. aimock's `normalizeResponse` (verified in
    node_modules/@copilotkit/aimock/dist/fixture-loader.cjs:14-19)
    auto-stringifies object-valued arguments at load time, so both
    shapes work. Updated the comments to reflect reality and stop
    misleading future fixture authors.

A3. e2e-deep per-feature timeout race orphaned runFeature
    When the synthetic timer won the Promise.race, runFeature was
    abandoned but never told to tear down. Browser context stayed
    held until the global timeout eventually fired, while the outer
    Semaphore.release ran immediately — a NEW feature could acquire
    the slot while the orphan still held the context, silently
    exceeding FEATURE_CONCURRENCY's pool budget. Now: a per-feature
    AbortController forwards the parent abort signal to runFeature;
    when the timer wins, .abort() fires so runFeature's finally chain
    tears down its page/context. The setTimeout cleanup is in a
    try/finally so a thrown rejection (defensive — runFeature's
    contract says no) doesn't leak the timer. The parent-abort event
    listener is removed on cleanup to prevent listener accumulation
    over many feature iterations.

    Symbols touched: per-feature loop body in executeE2eDeepDriver
    (e2e-deep.ts:982). runFeature signature unchanged.

A5. d5-chat-css user-bubble inner selector substring too loose
    `[class*="bg-muted"]` matches `bg-muted-foreground` too. Real
    Tailwind output puts `bg-muted-foreground` on nested children of
    the user bubble; the probe could read computed styles off the
    wrong element and silently mis-validate. Switched to
    `[class~="bg-muted"]` (whole-token match in space-separated
    class lists), the standard CSS3 way to express "this exact class
    is present on the element."

A7. auth legacy fill/press swallowed errors silently
    Legacy-shape assertion's catch block dropped fill/press errors
    so a chat-input cascade mismatch (or disabled textarea after
    sign-out) produced a generic "error surface did not appear"
    timeout instead of the real cause. Now captures the error
    message and appends it to the eventual error string so the
    failure record names what actually broke.

A9. d5-feature-mapping header listed removed `hitl-steps`
    The header's "destinations" list still showed `hitl-steps : 1
    demo` even though my PR's narrative says it was removed in
    genuine-pass Phase 0 — falsifying a claim my own diff makes.
    Updated the header to reflect the current REGISTRY_TO_D5 shape:
    `hitl-text-input` covers the 3 in-chat HITL variants (including
    the legacy `hitl` alias) and mcp-apps/subagents are split.

A6 reclassified to bucket (b) — the 6s waste on chip-driven probes is
sub-10% of the new 5-min per-feature timeout and not load-bearing for
convergence. Documented in the round summary; can be addressed in a
follow-up via a `noSend` ConversationTurn option.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 19:47:22 -07:00
Tyler Slaton 2d78a0f409 fix(showcase): rename gen-ui-headless D5 type to headless-simple and rewrite probe
The headless-simple demo was refactored to a deliberately minimal
"two hooks, one shadcn shell" template — text-in/text-out only, no
gen-UI. The D5 type literal `gen-ui-headless` no longer described
what the probe tests, and the old probe (Profile-card useComponent
+ continent fallback) was asserting against UI that no longer exists.

Three coordinated changes:

1. Rename `gen-ui-headless` D5FeatureType to `headless-simple` so the
   slug matches the demo. Updated d5-registry, REGISTRY_TO_D5,
   CATALOG_TO_D5_KEY (dashboard), and dependent tests/comments.
   `headless-complete` keeps its existing literal because that demo
   still drives the full gen-UI surface.

2. Replace d5-gen-ui-headless.{ts,test.ts,fixture} with
   d5-headless-simple.ts + headless-simple.json fixture. New probe
   clicks the "Say hello in one short sentence." chip and asserts the
   `[data-testid="headless-message-assistant"]` bubble mounts with
   non-empty content.

3. Restore `data-message-role` attributes on the headless-simple
   UserBubble + AssistantBubble. The runner's chat-input cascade
   documents these as the headless-template contract; the refactor
   dropped them, breaking the runner's settle plateau detection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 16:48:43 -07:00
Tyler Slaton 551a6b20ec fix(showcase): align dashboard CATALOG_TO_D5_KEY with harness REGISTRY_TO_D5
Three orphan mappings in shell-dashboard read PB rows that nothing
emits (`hitl→hitl-steps`, `interrupt-headless`, `tool-rendering-reasoning-chain`),
and seven harness mappings have no dashboard counterpart so cells
that should advance to D5 stayed at D4. Aligned the dashboard map to
mirror REGISTRY_TO_D5 exactly and added a drift test in the harness
that asserts structural equality going forward.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 16:21:01 -07:00
Alem Tuzlak fd6f7d418e Merge remote-tracking branch 'origin/main' into fix/showcase-d5-beautiful-chat-followup
# Conflicts:
#	showcase/aimock/d5-all.json
2026-05-06 18:50:39 +02:00
Alem Tuzlak ff4d7efa2a refactor(showcase): split beautiful-chat D5 probe into per-pill family
The aggregated multi-turn probe from #4672 hit a CopilotKit v2 quirk on
/demos/beautiful-chat: only the FIRST useComponent tool call in a
conversation paints its component. Subsequent tool calls emit (the
agent's followup content arrives) but the component never mounts.
Reproduced cleanly without any frontend tool involvement —
pie-chart turn 1 paints 5 svg circles in seconds, bar-chart turn 2
emits "Bar chart rendered above..." but paints zero recharts elements.

The runner can't sidestep this from inside one conversation without a
page.reload() between turns, which the structural Page type doesn't
expose. Splitting into per-pill scripts means each probe gets its own
browser launch — fresh page state, fresh conversation, no useComponent
ordering pollution. CATALOG_TO_D5_KEY maps `beautiful-chat` to all
listed literals; isD5Green requires every key green for the cell to
advance to D5, and per-pill failure isolation surfaces in PB row names.

Coverage in this PR (5 pills):
  - beautiful-chat-toggle-theme   (frontend tool, html.dark flip)
  - beautiful-chat-pie-chart      (controlled gen-UI useComponent)
  - beautiful-chat-bar-chart      (controlled gen-UI useComponent)
  - beautiful-chat-search-flights (A2UI fixed-schema FlightCards)
  - beautiful-chat-schedule-meeting (HITL with slot-click resolution)

All 5 verified locally end-to-end (5/5 pass against the local stack).

Out of scope, intentionally (track in follow-up):
  - Excalidraw     — depends on mcp.excalidraw.com reachability
  - Calculator     — sandboxed iframe; dup of d5-gen-ui-open
  - Sales Dashboard — generate_a2ui → render_a2ui chain renders
                      Metric labels but Row-bound charts don't paint
                      recharts containers under aimock fixtures (live
                      pill against same fixture chain shows the
                      inverse symptom). Suggests aimock's
                      non-progressive arg streaming differs from a
                      live LLM in a way the A2UI binder is sensitive
                      to. Needs separate aimock/binder investigation.
  - Task Manager   — manage_todos dispatches and agent emits closing
                      content, but StateStreamingMiddleware's
                      state.todos propagation doesn't populate the
                      App pane TodoList through aimock — same suspected
                      root cause as Sales Dashboard.

Architecture details:
  - _beautiful-chat-shared.ts factors DOM helpers + per-pill
    assertions, mirroring _hitl-shared.ts's pattern for an extended
    Page type with click() + a runtime guard
  - Each fixture uses unique D5-prefixed userMessage substrings; the
    multi-stage Schedule Meeting flow uses hasToolResult false→true
    for round disambiguation (no toolCallId leakage since each probe
    runs in its own fresh page session)
2026-05-06 18:40:47 +02:00
Alem Tuzlak 48cd3b2c93 feat(showcase/voice): D5 mapping + sample-button bypasses /transcribe (#4674)
## Summary

- **Dashboard mapping fix.** `CATALOG_TO_D5_KEY` in
`showcase/shell-dashboard/src/lib/live-status.ts` was missing `voice →
["voice"]`, so `computeMaxPossible` capped the langgraph-python voice
cell at D4 even when the d5-voice probe row was green. The harness
`REGISTRY_TO_D5` already had the entry; only the dashboard mirror was
out of sync.
- **Sample-button decoupled from `/transcribe`.** The "Play sample"
button used to fetch `sample.wav` and POST it to the runtime's
transcription endpoint, which made the sample button and the mic
indistinguishable under aimock (both returned the canned transcription).
Reworked it into a synchronous static-text injector — sample button is
now a deterministic test/demo affordance, and the mic is the only path
that exercises real Whisper transcription. Synced across all 18
voice-enabled integrations. Phrase stays `"What is the weather in
Tokyo?"` so aimock's `weather in Tokyo` substring fixture still matches.
- **Probe-test parity.** Added the missing `d5-voice.test.ts` companion
(every other `d5-*.ts` script has one) — 9 tests covering registration,
`buildTurns`, `preFill` (sample-button click + textarea-poll path), and
the weather/Tokyo assertion.
- **QA + e2e cleanup** for langgraph-python: dropped the
no-longer-applicable "Transcribing…" mid-flight assertion and the `block
/demo-audio/sample.wav` error-state subsection. Other 16 integrations'
qa/e2e files follow in a parity sync PR.

## Test plan

- [x] `nx test @copilotkit/showcase-harness -- --run d5-voice` → 9/9
pass
- [x] `npm test` in `showcase/shell-dashboard` → 509/510 pass (1
skipped, 0 failed)
- [x] `nx build @copilotkit/showcase-harness` → clean
- [x] Local boot: `langgraph-cli dev` (port 8123) + `next dev` (port
3000) + dashboard (port 3002) — voice page at `/demos/voice` renders,
"Play sample" injects the canned phrase instantly, send → agent returns
weather, mic → real Whisper transcription with `OPENAI_API_KEY` set
- [ ] Reviewer: confirm the langgraph-python voice cell on the live
dashboard advances to D5 once the next d5-voice probe tick lands a green
row
2026-05-06 18:25:33 +02:00
Alem Tuzlak 728ed61ce8 feat(showcase/voice): D5 mapping + sample-button bypasses /transcribe
The langgraph-python voice cell sat at D4 even when its d5-voice probe
row was green. Root cause: the dashboard's CATALOG_TO_D5_KEY mirror in
showcase/shell-dashboard/src/lib/live-status.ts was missing voice ->
["voice"], so computeMaxPossible capped voice at D4 regardless of probe
state. The harness REGISTRY_TO_D5 already had the entry; only the
dashboard mirror was out of sync.

Separately, the "Play sample" button used to fetch sample.wav and POST
it to /transcribe. With aimock that meant both the sample button AND
the mic returned the same canned response, which made it impossible to
demo the mic path locally without conflating the two affordances.
Reworked the button into a synchronous static-text injector
(onTranscribed(sampleText)) so:

- Sample button = deterministic test/demo affordance, no runtime calls.
- Mic = real Whisper transcription via /transcribe.

Synced across all 18 voice-enabled integrations. Phrase stays "What is
the weather in Tokyo?" so aimock's "weather in Tokyo" substring fixture
still matches.

Also adds the missing d5-voice.test.ts companion (every other d5-* probe
script has one) and trims the langgraph-python qa/voice.md + e2e steps
that depended on the now-removed async behavior.
2026-05-06 18:11:24 +02:00
Alem Tuzlak 53d7999c05 Merge remote-tracking branch 'origin/main' into feat/showcase-d5-headless-chat
# Conflicts:
#	showcase/aimock/d5-all.json
2026-05-06 17:04:00 +02:00
Alem Tuzlak 3eb53a8621 feat(showcase): D5 probe for headless-complete + extend headless-simple (langgraph-python)
Promotes /demos/headless-complete to its own D5 feature type so the
dashboard cell can reach D5 instead of riding on the headless-simple
probe (which was navigating to /demos/headless-simple regardless of
which catalog feature triggered it).

- New gen-ui-headless-complete D5 feature type + script that clicks
  each suggestion chip via preFill and asserts the right surface
  renders: WeatherCard (get_weather), StockCard (get_stock_price),
  HighlightNote (frontend useComponent), Excalidraw best-effort, and
  the canonical "Asia is the largest continent" text reply.
- Existing gen-ui-headless script now drives both turns by chip
  click (Profile card + Largest continent) instead of typing.
- Fixtures pin narration legs with both userMessage AND toolCallId
  and order them before the bare userMessage toolCall fixture —
  aimock's toolCallId matcher reads the LAST tool message in the
  request, but in a multi-turn probe that "last tool" stays on a
  previous turn's id until a new tool runs, which would otherwise
  hijack a later turn's prompt with a stale narration.
- headless-complete UserBubble + AssistantBubble now carry
  data-message-role so the harness conversation runner can detect
  message arrivals (mirrors the headless-simple convention).
- Mappings updated in lockstep:
    - REGISTRY_TO_D5:  headless-complete -> ["gen-ui-headless-complete"]
    - CATALOG_TO_D5_KEY (dashboard): same.
2026-05-06 16:21:43 +02:00
Alem Tuzlak e11c1d0810 feat(showcase): D5 conversation probe for beautiful-chat (langgraph-python)
Beautiful Chat was capped at D4 in the dashboard because it had no
dedicated D5 probe and was deliberately excluded from CATALOG_TO_D5_KEY
(commit 974494ecb stripped the freeloading "agentic-chat" alias). PR
#4668 fixed the A2UI surface rendering and added e2e tests, but those
land at the D3 tier — D5 is a separate probe with its own driver.

Changes:

- New d5-beautiful-chat probe asserts the A2UI fixed-schema FlightCard
  surface renders with literal United/Delta/$349/$289 fingerprints from
  the search_flights tool. 60s budget on first card, 5s on siblings.
- New harness/fixtures/d5/beautiful-chat.json with two-stage fixture
  (hasToolResult false→true) mirroring the gen-ui-headless pattern.
  Fixture spliced into the bundled aimock/d5-all.json.
- New "beautiful-chat" D5FeatureType literal in the registry's union +
  runtime mirror.
- d5-feature-mapping.ts: replace "beautiful-chat": ["agentic-chat"]
  alias with ["beautiful-chat"] so the probe targets its own dedicated
  PB key instead of freeloading agentic-chat's green status.
- live-status.ts CATALOG_TO_D5_KEY: re-add "beautiful-chat":
  ["beautiful-chat"] so computeMaxPossible lifts the D4 cap to D5.
2026-05-06 14:35:07 +02:00
Alem Tuzlak 6ea83510b4 fix(showcase/dashboard): cap maxPossible at D5 and align live-cell + RefDepth chips
Two follow-ups to the parity/cell-matrix fix:

1. The live cell view (`composed-cell.tsx`, `feature-grid.tsx` →
   `ref-depth-column.tsx`) had the same regression-vs-ceiling collision —
   they passed `regression={isRegression}` to DepthChip and `regression`
   short-circuits the chip to red regardless of `maxDepth`. Switched
   them to pass `maxDepth={maxPossible}` only, mirroring the matrix fix.
   `RefDepthCellProps` also flips `regression?` → `maxDepth?` to stop
   propagating the dead prop.

2. `computeMaxPossible` returned 6 whenever a D5 mapping existed, on the
   assumption D6 was structurally reachable. There is no per-feature D6
   registry, so D6 is a stretch goal in practice and every D5 cell was
   rendered amber as "below ceiling". Cap at 5; D6-green cells still
   render green because `depthColorClass` treats `depth >= maxDepth`
   as at-ceiling. Tests updated accordingly.

`page.tsx` healthStats lost its `isRegression` short-circuit so it now
mirrors the same graduated logic the chips use.
2026-05-05 15:36:36 +02:00
Alem Tuzlak 8ed0c8dbb3 fix(showcase/dashboard): pass maxDepth to DepthChip so D5 cells aren't all red
The matrix chips were hard-coded red at D5 because parity-matrix and
cell-matrix passed `regression={depth.isRegression}` to DepthChip but
never `maxDepth`. DepthChip short-circuits to red when `regression` is
true, regardless of depth. Since `isRegression` was redefined as
`achieved < maxPossible` and `computeMaxPossible` returns 6 whenever a
D5 mapping exists, every D5 cell without a green D6 probe (the common
case — D6 probes are rare) ended up red.

Pass `maxDepth={depth.maxPossible}` and drop the `regression` prop. This
re-engages the chip's intended graduated coloring: green at ceiling,
amber 1-2 below, red 3+ below. The `isRegression` field is still used
by the cell-matrix `filter="regressions"` row filter — that's unchanged.
2026-05-05 14:31:34 +02:00
Alem Tuzlak 8bc2e531cd fix(showcase/dashboard): add maxPossible to parity-matrix DepthResult literals
Two object literals in parity-matrix.tsx ParityCategorySection were missing
the required maxPossible field on DepthResult, causing 'next build' to fail
with TS2322 and blocking the shell-dashboard production rebuild. Setting
maxPossible: 0 matches achieved: 0 for unwired cells (no probe → no ceiling),
which is consistent with how DepthChip renders an unshipped cell.

Cherry-picked from 1db2fbc7b on fix/dashboard-polish (Jordan Ritter); landing
standalone so the showcase deploy can rebuild from main without waiting on
the larger dashboard-polish branch.
2026-05-05 12:12:51 +02:00
Tyler Slaton 04f77586f3 style: fix formatting failures on main
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 13:46:32 -07:00
Jordan Ritter 22cef3b073 Remove outer black border from showcase-dashboard favicon
Keep the black fill inside the shield shape, only make the area
outside the shield transparent. Revert og-image.png to original.
2026-05-01 23:30:00 -07:00
Jordan Ritter b1b7bb5cb8 Add Baseline tab, Coverage perf + depth fixes, dark mode toggle
Baseline tab (new):
- 33-feature x 25-partner correctness matrix from Notion Partner Hub
- E3 cell treatment: status emoji + colored letter badges (C/A/I/▶/D/T/✱)
- View/Edit toggle with commit/cancel accumulator (batch saves to PB)
- PocketBase baseline collection with public read/write, SSE live updates
- Stats bar, collapsible categories, fixed legend, 200ms CSS tooltips
- Full-row hover highlight, zebra stripes matching Coverage tab

Coverage tab fixes:
- Depth chip: green = at max achievable (not hardcoded per level)
- maxPossible computed from probe existence (CATALOG_TO_D5_KEY)
- D5 false positives: removed shared key aliases
- Stats bar derives green/amber/red from same logic as depth chips
- Hide badges for non-existent tests (was strikethrough)
- Render perf: memoize featuresByCategory, React.memo CategorySection
- Load perf: getFullList single call (was 7 sequential round-trips)

Both tabs:
- Dark/light/system theme toggle (upper right)
- Column order: Coverage first (1-18), Baseline extras (19-25)
- LangGraph naming (orchestration layer, not model router)
- Row hover highlight including sticky column
2026-05-01 22:51:23 -07:00
Jordan Ritter 92f5a0ec83 fix(showcase): only show docs in CLI Start row when Docs toggle is on
DocsLayer rendered unconditionally for docs-only features whenever
any overlay was active. Now respects the Docs toggle — docs-only
cells show Links (command) when Links is on, Docs when Docs is on,
empty otherwise.
2026-05-01 18:29:33 -07:00
Jordan Ritter dc517e505b fix(showcase): order drilldown dimensions highest to lowest
Was: API (D3), CV (D5), RT (D4), Health, Smoke
Now: CV (D5), RT (D4), API (D3), Health, Smoke
2026-05-01 00:08:06 -07:00
Jordan Ritter d0bb0eca2d fix(showcase): restore depth 6 in DepthChipProps type union
AchievedDepth includes 6 — removing it from the chip prop broke the
build. Keep 6 in the type (renders as emerald, same as D5).
2026-04-30 23:59:25 -07:00
Jordan Ritter 660aae0d8d fix(showcase): depth chip colors — D3/D4 amber, D1/D2 red, remove D6
D3/D4 was blue (accent), now amber/yellow to signal "not yet D5".
D1/D2 was amber, now red to signal "needs work".
Removed D6 references — D6 no longer exists.
2026-04-30 23:46:02 -07:00
Jordan Ritter 59f8a54c80 fix(showcase): render CLI Start command in docs-only cells
The docs-only early return only rendered DocsLayer, skipping LinksLayer
which is responsible for rendering CommandCell when a demo has a
command field. The npx starter command disappeared from the matrix.
2026-04-30 17:18:39 -07:00
Jordan Ritter 73833a368b fix(showcase): show docs links for docs-only features under any active overlay
The CLI Start Command feature (kind: "docs-only") was rendering empty
cells in the dashboard grid because the docs row only displayed when the
"docs" overlay was explicitly toggled on. Since docs are the only content
for docs-only features, show the docs row whenever any content-producing
overlay is active (links, depth, health, or docs), not just docs alone.
2026-04-30 17:04:57 -07:00
Jordan Ritter d439b61c68 Revert "perf(showcase-dashboard): yield during fetch + parallelize initial pages (#4504)"
This reverts commit d17ea911e0, reversing
changes made to a0770ea0cf.
2026-04-30 16:31:37 -07:00
Jordan Ritter d17ea911e0 perf(showcase-dashboard): yield during fetch + parallelize initial pages (#4504)
## Summary

Follow-up to #4502. With matrix cells memoized and SSE deltas coalesced,
the dashboard is much smoother — but the **initial dashboard load**
still freezes for a few seconds because:

1. **Initial PB fetch is 10 sequential round-trips.** `fetchInitial`
walks `getList(page=1, 200) → getList(page=2, 200) → …` up to 10 pages,
each awaiting the previous. Network wall time stacks.
2. **First commit with real data is unavoidably a full-matrix
re-render.** The empty-map → populated-map transition invalidates
per-key memo checks on every cell — that's 720 cell renders in one
synchronous React commit, blocking the main thread.

## Changes

- **Parallelize the initial fetch.** Pull page 1 sequentially to learn
`totalItems`, then fire pages 2..N concurrently via `Promise.all`. Wall
time drops from `sum(rtt_per_page)` to roughly `max(rtt_per_page)`
modulo network parallelism. PocketBase reads are independent so this is
safe.
- **`startTransition` around the initial `setRows(initial)`.** The first
big commit is unavoidable, but marking it as a transition lets React 19
yield to user input mid-walk instead of blocking for the entire render.
`setStatus` stays urgent so the "connecting → live" indicator still
flips immediately.
- **`startTransition` around the SSE flush.** Bursts that the 16ms
coalescer can't fully absorb (large reconnect replays, simultaneous
probe completions) still need to yield rather than block.

## Test plan

- [x] `useLiveStatus.test.tsx` 15/15 pass with parallelized fetch (the
mock handles arbitrary page numbers).
- [x] `composed-cell.test.tsx` 9/10 pass (1 pre-existing failure, same
as #4502).
- [x] No new TypeScript errors in `useLiveStatus.ts`.
- [ ] **Manual perf trace before/after on the live dashboard.**
Expected: initial-load wall time drops sharply (sequential → parallel
pages), and any remaining heavy commit no longer blocks the main thread
(transition lets React yield).
- [ ] **Functional sanity** — confirm rows still arrive correctly when
pages return out of order, and the "connecting → live" status transition
still fires before the heavy render.
2026-04-30 16:02:43 -07:00
Jordan Ritter 8e36bb8b2d fix(showcase): reorder legend — L1-L4 first, add D3, remove D0-D4 2026-04-30 14:50:39 -07:00
Jordan Ritter 0c4ad13460 fix(showcase): exclude docs-only features from catalog metadata counts (#4538)
## Summary

- docs-only features (e.g. cli-start) exist purely for documentation
coverage tracking and have no route, depth probes, or health signals
- The catalog metadata was counting their 18 stub cells in the headline
wired/stub/unshipped/unsupported breakdown, inflating the total and
making the stats bar misleading
- Exclude docs-only cells from the headline counts; add a new
`docs_only` field to track them separately

**Before:** total_cells=720 wired=673 stub=18 unsupported=29
**After:** total_cells=702 wired=673 stub=0 unsupported=29 docs_only=18

## Test plan

- [x] `showcase/scripts` test suite: all 574 tests pass (including 13
catalog tests)
- [x] `showcase/shell-dashboard` test suite: same 3 pre-existing
failures, no regressions
- [x] TypeScript check clean (no new errors)
- [x] Invariant holds: wired + stub + unshipped + unsupported +
docs_only == cells.length
2026-04-30 14:39:50 -07:00
Jordan Ritter 79c10dbce2 fix(showcase): exclude docs-only features from catalog metadata counts
docs-only features (e.g. cli-start) exist purely for documentation
coverage tracking -- they have no route, no depth probes, and no
health signals. The catalog metadata was counting their 18 stub cells
in the headline wired/stub/unshipped/unsupported breakdown, inflating
the total and making the stats bar misleading.

Exclude docs-only cells from the headline counts. A new docs_only
field tracks the excluded count separately so the invariant
(wired + stub + unshipped + unsupported + docs_only == cells.length)
holds.

Before: total_cells=720 wired=673 stub=18 unsupported=29
After:  total_cells=702 wired=673 stub=0  unsupported=29 docs_only=18
2026-04-30 14:38:18 -07:00
Jordan Ritter 9ec8b90516 fix(showcase): remove D6 from depth stats, tighten grid spacing 2026-04-30 14:23:58 -07:00
Jordan Ritter 7556b0d244 fix(showcase): improve cell drilldown dialog width, signal readability, and deduplication
Widen dialog from w-72 to w-[480px], extract key signal fields
(errorDesc, error, failureSummary, backendUrl) as readable key-value
pairs instead of buried JSON, make raw signal collapsible, remove
duplicate tooltip text that repeated badge color/status info.
2026-04-30 14:15:46 -07:00
Jordan Ritter accf9b5050 fix(showcase): hide depth chip for docs-only features, add depth distribution to stats bar
Suppress RefDepthCell for docs-only features in the parity overlay,
showing '--' instead of a depth chip since docs-only features have no
probe coverage. Add depth distribution counts (D6..D1) to the
AdaptiveStatsBar when the depth overlay is active, giving operators
at-a-glance visibility into how many wired cells are at each depth.
2026-04-30 14:02:57 -07:00
Jordan Ritter 1d817577f2 fix(showcase): add favicon and og-image to shell-dashboard 2026-04-30 13:53:57 -07:00
Jordan Ritter 03aa2f3756 feat(showcase/dashboard): add docs-only feature kind
Features with kind "docs-only" show only the docs row in the
dashboard -- no Dx badges, no status badges, no links or depth
layers. The row label is muted and tagged like "testing" rows.

- feature-registry.json: cli-start changed from primary to docs-only
- registry.ts: FeatureKind union extended with "docs-only"
- cell-pieces.tsx: CellStatus returns null for docs-only (hides all
  badges)
- composed-cell.tsx: docs-only features render only DocsLayer, skip
  links/depth/health
- feature-grid.tsx: docs-only rows use muted italic styling with a
  "docs-only" tag
- Tests added for all three behavioral changes
2026-04-30 13:49:05 -07:00
Jordan Ritter 1541cbc96f fix(showcase/dashboard): unsupported cells show indicator instead of numeric depth
deriveDepth() was computing D2 for not_supported_features because D1/D2
are integration-scoped. Add unsupported boolean to DepthResult, guard in
deriveDepth, update consumer components. 24/24 depth-utils tests pass.
2026-04-30 13:34:57 -07:00
Tyler Slaton 26e245c009 chore: run pnpm format
Signed-off-by: Tyler Slaton <tyler@copilotkit.ai>
2026-04-30 12:32:31 -07:00
Jordan Ritter d7a9a2efca feat(showcase): add D2/API badge to dashboard per-cell status row
The cell status row rendered RT (D4/e2e) and CV (D5) badges but was
missing the D2/API badge despite the legend already documenting it.

- Add `d2` field to `CellState` sourced from `agent:<slug>` rows
- Render API LiveBadge before RT in CellStatus (D2 < D4 ordering)
- Add API (Agent) to CellDrilldown DIMENSIONS array
- Fix stale FP/D6 references in tests left over from #4513
2026-04-30 10:49:06 -07:00
Jordan Ritter 60e8da1a75 fix(showcase): remove FP/D6 badge from cells and drilldown 2026-04-30 10:17:59 -07:00
Jordan Ritter 5d81f6c381 fix(showcase): reorder legend D2 before D4/D5, remove stale D6 comment 2026-04-30 10:07:49 -07:00