The previous fixture regression (HTML+CSS only, no jsFunctions) slipped
past CI because the e2e suite only asserted "iframe mounts with non-empty
srcdoc" — which passes whether or not the iframe is interactive. Adds
two layers of guard so the same regression cannot land silently:
1. showcase/scripts/__tests__/open-gen-ui-advanced-fixtures.test.ts
(vitest, runs in showcase_validate on every PR): asserts each of the
three interactive fixture entries in d5-all.json ships jsFunctions
referencing the matching host bridge (evaluateExpression / notifyHost).
Catches "someone removed jsFunctions" at PR-time with no
infrastructure dependencies.
2. showcase/integrations/langgraph-python/tests/e2e/open-gen-ui-advanced.spec.ts
(playwright, runs in test_e2e-showcase-on-demand): adds three
round-trip tests that drive the in-iframe controls and assert the
host-side handler ran by capturing its console.log + verifying the
iframe output element reflects the host response. Catches "the
renderer fails to inject jsFunctions into the sandbox" too.
The e2e tests also switch the existing smoke tests off pill-click and
onto a textarea-driven fill+Enter path, following the same precedent as
commit 15db0bbf3 (gen-ui-headless-complete) — chip mounts diverge
between EmptyState and SuggestionBar surfaces, and Playwright's pill
click races React hydration. Using [data-testid="copilot-chat-textarea"]
with an explicit click + waitForLoadState("networkidle") makes the
suite reliable end-to-end (7/7 passing locally against the aimock-driven
stack).
The "Draw a flowchart" suggestion pill in the mcp-apps demo sent
"Use Excalidraw to draw a simple flowchart with three steps." which
had no matching create_view fixture in d5-all.json. aimock walked
through to feature-parity.json's `{userMessage: "steps"}` substring
fixture and returned a generic "Here is my plan..." content blurb
with no MCP tool call, so the runtime never invoked create_view, the
MCP middleware never fetched the UI resource, and the sandboxed
iframe never mounted.
Add a fixture pair in d5-all.json (and its harness mirror) keyed on
"draw a simple flowchart": turn 1 emits create_view with a three-
step Start -> Process -> End flowchart, turn 2 emits the narration
after the tool result. The distinctive substring beats the generic
feature-parity catch-alls under first-match-wins.
Adds a regression test that loads the same fixture files in the same
order as docker-compose.local.yml and asserts via aimock's matchFixture
that each mcp-apps pill routes to its create_view fixture on turn 1 and
its narration fixture on turn 2.
User-facing renames so the showcase reads the way a cold visitor would
expect:
- `byoc-hashbrown` → `declarative-hashbrown` (and `byoc-json-render` →
`declarative-json-render`). The display titles already said
"Declarative UI: …"; only the URL slugs and folder paths still
leaked the internal BYOC ("Bring Your Own Components") jargon.
Renamed:
/demos/byoc-hashbrown → /demos/declarative-hashbrown
/demos/byoc-json-render → /demos/declarative-json-render
/api/copilotkit-byoc-* → /api/copilotkit-declarative-*
src/app/demos/byoc-* → src/app/demos/declarative-*
qa/byoc-*.md → qa/declarative-*.md
tests/e2e/byoc-*.spec.ts → tests/e2e/declarative-*.spec.ts
Internal Python module names + langgraph graph IDs stay legacy
(`byoc_hashbrown_agent.py`, `byoc_hashbrown`) — those are not
user-facing and renaming them is a separate cross-codebase pass.
- `a2ui-fixed-schema` slug intentionally unchanged.
- Tool Rendering trio parenthetical rename (Default → Catch-all →
Custom progression reads clearly as "how much do I customize?"):
Tool Rendering (Default) — unchanged
Tool Rendering (Custom default) → Tool Rendering (Catch-all)
Tool Rendering (Specific) → Tool Rendering (Custom)
- `tool-rendering-reasoning-chain` cell renamed from
"Generative UI: Rendering multiple tools" to
"Generative UI: Tool calls + reasoning" (the demo is about combining
reasoning + tool rendering, not about quantity of tools).
- `Open Generative UI: Default` / `Open Generative UI: Custom`
descriptions expanded so a visitor understands how Open Generative UI
differs from Tool Rendering (agent composes UI from a registered
library vs. attaching a renderer to a *named* backend tool).
- Showcase index now sorts demos within each tag by `manifest.features`
order. Previously demos appeared in manifest declaration order, which
ignored the team's curated "polished flagship → simplest start →
variants" arc.
Cross-cutting registry / harness / dashboard updates that fall out of
the rename:
- `shared/feature-registry.json` adds the two new IDs alongside the
legacy `byoc-*` (so the catalog stays valid; the other 17
integrations still declare `byoc-*` in their manifests).
- `shared/constraints.yaml` adds the new IDs to the
generative-ui-approach allow-list.
- `scripts/__tests__/generate-catalog.test.ts` updates the cell-count
expectations (45 features × 18 integrations = 810; 792 after docs-
only exclusion; 45 LGP cells = 38 wired + 1 stub + 6 unshipped).
- Harness probe `d5-byoc.ts` + `d5-byoc.test.ts` now route both slug
families through `preNavigateRoute` and exercise the new branches.
- `d5-feature-mapping.ts` and `shell-dashboard/live-status.ts` mirror
the dual-ID mapping so both legacy and renamed slugs roll up under
the same `byoc` D5 featureType.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related changes that bring the dashboard's gold-standard view in
line with the desired naming convention and surface deprecated rows
behind a toggle (instead of hiding them at catalog generation).
## Naming alignment
Applied 28 renames in feature-registry.json + 20 in LGP manifest per
the user-provided mapping. Highlights:
- "Pre-Built CopilotChat" -> "Pre-Built: CopilotChat"
- "Headless Chat (Simple/Complete)" -> "Headless UI: Simple/Complete"
- "Multi-modal / File Uploads" -> "Attachements" (intentional spelling)
- "Controlled Gen-UI (Display)" -> "Generative UI: useComponent"
- "In-Chat HITL (use*)" -> "Human In/in the Loop: In-chat / Interrupts"
- "Headless Interrupt" -> "Human in the Loop: Headless Interrupts"
- "Declarative Generative UI (A2UI - *)" -> "Declarative UI: */* A2UI"
- "Fully Open-Ended Generative UI" -> "Open Generative UI: Default"
- "Tool Rendering ..." -> "Generative UI: Tool Rendering (...)"
- "Tool Rendering + Reasoning Chain" -> "Generative UI: Rendering multiple tools"
- "Agentic Generative UI ..." -> "Generative UI: Agent State"
- "Frontend Tools (...)" -> "Frontend Tools: ..."
- "Shared State (...)" -> "Shared State: ..."
- "State Streaming" -> "Shared State: Streaming"
- "Readonly State (Agent Context)" -> "Shared State: Frontend Context"
- "BYOC Hashbrown <-> json-render" -- labels intentionally swapped per
user instruction (demos were historically reversed; new labels
reflect what they actually do).
LGP manifest demos[].name updated to match feature-registry names so
the dojo and dashboard surface the same human-readable label.
## Show Deprecated toggle (feature-grid.tsx)
Added a checkbox in the matrix header -- default OFF -- that filters
feature rows where `feature.deprecated === true`. Toggle ON shows all
deprecated features across all integrations (audit trail); toggle OFF
hides those rows entirely so the gold-standard view stays clean.
Reverted the catalog-side filter from PR #4744 (which dropped LGP
cells for deprecated features at catalog-generation time). Now the
catalog emits cells uniformly for all (integration x feature) pairs,
and visibility is controlled at the dashboard layer. Toggling on
shows complete cross-integration data without missing-cell artifacts.
Affects 4 features marked deprecated:true in feature-registry.json:
agentic-chat-reasoning, hitl, hitl-in-chat-booking,
reasoning-default-render.
LGP cell count: back to 43 (38 wired + 1 stub + 4 unshipped). The 4
unshipped rows are hidden by default; toggle to surface them.
Tests: 18/18 catalog tests + 1588/1588 harness vitest passing.
validate-fixture-tool-surface clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three fixes that follow up on PR #4743 to bring LGP closer to fully-green
on the dashboard:
1. tool-rendering-reasoning-chain probe: was failing with
`expected [data-testid="reasoning-block"] to mount within 30000ms`.
Root cause: the demo's `<ReasoningBlock>` slot only mounts when a
reasoning-role message lands in the transcript, which requires
aimock to emit REASONING_MESSAGE_* events, which in turn requires
the fixture's first-leg response to carry a `reasoning` field. The
weather/Tokyo and SFO/JFK first-leg fixtures were missing it.
Mirrors the convention documented in reasoning-display.json:2.
Patched both source (harness/fixtures/d5/) and bundle (aimock/d5-all.json).
2. gen-ui-interrupt source fixture: the source fixture file was missing
the resume-leg toolCallId entries that already existed in the bundle.
Cosmetic mirror so re-bundling stays consistent. Same chip prompts +
same toolCallIds as interrupt-headless.json (both probes share the
same agent and aimock fixture set; the difference is the FRONTEND
rendering — useInterrupt inline vs useHeadlessInterrupt separate-pane).
3. Dashboard gold-standard filter: 4 deprecated/legacy features
(agentic-chat-reasoning, hitl, hitl-in-chat-booking,
reasoning-default-render) used to render as X-marked rows in the
LGP gold-standard dashboard view because LGP intentionally does
NOT implement them — they were consolidated into the modern shape
(reasoning-custom + reasoning-default; hitl-in-chat with
useHumanInTheLoop). Other 17 integrations still serve those legacy
demos, so we don't yank the features from feature-registry.json
entirely. Instead: marked them `deprecated: true` and updated
generate-registry.ts to skip emitting cells when a deprecated
feature is unshipped for an integration. LGP cells: 43 → 39 (the
4 deprecated rows disappear). Other integrations: unchanged
(audit trail preserved). Catalog total: 774 → 770.
Tests:
- 1588/1588 harness vitest passing
- 19/19 generate-catalog + generate-registry tests passing
(counts updated for the 4 dropped LGP cells + new deprecated-
feature filter test)
- validate-fixture-tool-surface clean (282 fixtures × 627 demos)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three highlight paths in langgraph-python's manifest pointed at files
that don't exist after the PR #4694 reorganization:
- hitl-in-chat → src/agents/hitl_in_chat.py (actually hitl_in_chat_agent.py)
- chat-slots → custom-welcome-screen.tsx (file doesn't exist; use slot-wrappers.tsx)
- mcp-apps → copilotkit-mcp-apps/route.ts (actually .../[[...slug]]/route.ts)
The bundler walks every highlight at build time; one missing path aborts
the whole CI step. Fix all three.
Test snapshot counts in generate-catalog and generate-registry hardcoded
40 features / 720 cells / 702 total. With the two new feature IDs added
to the registry (reasoning-default + reasoning-custom), counts shift to
42 / 756 / 738; the LGP-specific cell distribution moved from
39 wired + 1 stub + 0 unshipped to 35 wired + 1 stub + 6 unshipped, and
the registry-side LGP feature/demo count drops to 36 (PR #4694 trimmed
4 items from the manifest's features list).
docs-only features (e.g. cli-start) exist purely for documentation
coverage tracking -- they have no route, no depth probes, and no
health signals. The catalog metadata was counting their 18 stub cells
in the headline wired/stub/unshipped/unsupported breakdown, inflating
the total and making the stats bar misleading.
Exclude docs-only cells from the headline counts. A new docs_only
field tracks the excluded count separately so the invariant
(wired + stub + unshipped + unsupported + docs_only == cells.length)
holds.
Before: total_cells=720 wired=673 stub=18 unsupported=29
After: total_cells=702 wired=673 stub=0 unsupported=29 docs_only=18
- claude-sdk-typescript manifest: G3's headless-complete and a2ui-fixed-schema entries pointed at -agent.ts paths but the actual files are -prompt.ts. Bundle-demo-content blew up on the missing files.
- generate-catalog test: update reference and crewai-crews wired-count assertions for the post-blitz state. Multiple integrations now tie at 39 wired; auto-detection picks the alphabetical winner (langgraph-fastapi) over langgraph-python. crewai wired count moved to 37.
Adds a fourth cell status, "unsupported", to the showcase dashboard for
features whose framework cannot architecturally support them (e.g.,
no graph-interrupt API, no MCP tool runtime). This is distinct from
"unshipped" (just unbuilt).
- Manifest schema gains optional `not_supported_features: string[]`.
- generate-registry.ts: determineCellStatus checks `not_supported_features`
first; CatalogCell/CatalogMetadata gain `unsupported`; max_depth=0 for
unsupported cells; parity tier is computed against the supportable subset
of the reference (unsupported features no longer drag a framework's tier
down for gaps it legitimately cannot fill).
- depth-utils.ts: deriveDepth returns D0 with no regression for
unsupported cells.
- DepthChip: renders a dashed gray border + 🚫 glyph with
"Not supported by this framework" tooltip; data-status attribute
distinguishes unshipped vs unsupported.
- CoverageBar gains diagonal-stripe gray segment for unsupported.
- StatsBar / AdaptiveStatsBar / CellsView: separate "Unsupported" chip.
- CellMatrix gaps filter excludes unsupported cells (not work to do).
- Tests: cover unsupported in DepthChip, depth-utils, cell-matrix, and
catalog generator metadata/max_depth assertions.
remove dead starter scripts
Update probe configs, fixtures, and test drivers to reference
showcase/integrations/ instead of showcase/packages/.
Remove obsolete generate-starters test, starter-consistency
test, validate-workflow-starters, and rebuild-coord.sh.
Delete the pre-generated starters/ directory tree (previously
synced from packages/ by generate-starters.ts). Add
extract-starter.ts which produces a starter tarball on demand
from any integration. Move shared starter template files to
showcase/shared/starter-template/.
1. Add missing @json-render/core and @json-render/react deps to
google-adk package.json (Docker build failed on byoc-json-render
demo imports)
2. Update catalog test expectations to match actual generated counts
(wired 482, crewai-crews 28, LGP 37 wired + 2 unshipped)
3. Regenerate google-adk starter from template to eliminate drift
detected by the starter consistency check
## Summary
Replace the hardcoded 200-line INTEGRATIONS array in
`integration-smoke.spec.ts` with a 6-line derivation from
`registry.json`. New demos automatically appear in smoke tests when
manifests are updated.
- Delete hardcoded Integration interface + 17-entry array
- Derive slug, name, backendUrl, deployed, hasToolRendering, demos from
registry
- Add unit tests verifying derivation correctness + regression guard
against re-hardcoding
## Test plan
- [x] New vitest tests pass (7/7)
- [x] Existing showcase build pipeline tests pass (1183/1183 across 26
files)
- [ ] CI green
Add missing feature IDs (hitl, hitl-in-chat-booking) to langgraph-python
manifest so it reclaims reference status from langgraph-fastapi. Update
catalog test expectations and validate-pins fail-baseline.json hash/count
after PR #4287 dependency changes.
Replace the 200-line hardcoded INTEGRATIONS array in integration-smoke.spec.ts
with a 6-line derivation from registry.json. New demos automatically appear in
smoke tests when manifests are updated — no manual maintenance needed.
Cross-joins 38 features x 17 integrations + 17 starters = 663 cells.
Each cell carries status (wired/stub/unshipped), auto-derived parity
tier, max depth, and human-readable display names from feature-registry
and manifests. Reference integration auto-detected by max wired count.
## Summary
- **Gitignore all generated `src/data/*.json` across the 4 shell apps**
— these are regenerated by every build path (Docker, CI, `npm run
build`, `npm run dev`) and don't need to be tracked. Removes 11 blobs
totaling ~28K lines of generated content.
- **Strip `generated_at` timestamps** from all 5 generator scripts and
all consumer interfaces/types — these were the root cause of constant
git noise (every build bumped the timestamp even when content was
identical).
- **Make shell-dashboard independent** — imports now use `@/data/`
instead of cross-importing from `../../../shell/src/data/`.
`probe-docs.ts` writes directly to shell-dashboard. Dockerfile no longer
copies the entire shell package.
- **Fix build scripts** — shell-dojo's `build` now runs generators
before `next build`; shell's `dev` now runs all one-shot generators on
startup (not just demo-content in watch mode).
- **Document generated data files** in `showcase/README.md` with a table
covering all 6 file types, their generators, and which shell apps
consume them.
## Test plan
- [ ] CI passes (scripts, shell builds, dashboard builds)
- [ ] `npm run dev` in each shell app generates fresh data files on
startup
- [ ] `npm run build` in shell-dojo completes (was previously bare `next
build`)
- [ ] shell-dashboard Docker build succeeds without copying shell/
- [ ] No `generated_at` fields in any generated JSON output
- [ ] Generated JSON files no longer show up in `git status` after
build/dev
Every generator embedded `generated_at: new Date().toISOString()` in its
output, causing constant git noise on every build/dev run even when
actual content was unchanged. Remove the field from all 4 generator
scripts, all consumer interfaces (Registry, BundledContent,
BundledStarters, DocsStatusBundle), inline type casts, and test
assertions.
Also: add shell-dashboard as a generate-registry output directory (it
was cross-importing from shell); move probe-docs output to
shell-dashboard/src/data/ (sole consumer); update test beforeAll to
generate files instead of restoring from git HEAD (prep for gitignore).
Add health-path verification tests that assert getAgentHealthPath()
returns the correct path for every framework, derived from reading the
actual agent server source code. Tests verify:
- Fixture map covers all 17 FRAMEWORKS entries
- getAgentHealthPath(fw) matches the fixture for each framework
- Generated entrypoint.sh watchdog probes the correct URL
- langgraph starters probe /ok, all others probe /health
- Frontend health route uses the correct agent probe path
Also documents langgraph /ok verification: langgraph_cli Python and
@langchain/langgraph-cli TS both serve /ok as the only built-in
health endpoint. /health is NOT served. Keeping /ok is correct.
Declare open-gen-ui and open-gen-ui-advanced in langgraph-python
manifest (code existed, was never registered). Add both to
constrained-explicit allowlist, fill shell_docs_path for 5 demos,
add hitl-in-app override, drop stale chat-customization-css fallback.
Regenerate registry.json, demo-content.json, constraints.json,
and docs-status.json across shell / shell-dojo / shell-docs.
Bump feature/demo count assertion 30→32 in generate-registry test.
Extend check-binaries.sh whitelist for sister-shell demo-content.
Extract drift-comparison logic out of validate-pins.ts into
validate-pins-core.ts (pure module, CLI re-exports + remains thin
orchestrator) and extract the formatter out of
redirect-decommission-report.ts into redirect-decommission-core.ts.
Add vitest coverage for both cores with baseline fixtures so
showcase-ops ProbeDrivers can reuse the same logic without forking.
Drop legacy generate-status.ts — superseded by showcase-ops live
status feed.
Post-#4029 (c92dde419) the langgraph-python manifest dropped from 32 to 30
features/demos when open-gen-ui was scrubbed from the constraint schemas
and manifest. The Registry Generator test wasn't updated and has been
failing on main since — blocking every subsequent PR's CI.
Updates the hardcoded 32 → 30 to match the current manifest reality so
CI can go green again. Pre-existing drift, unrelated to the aimock
fixture work in the parent commit but bundled here to unblock this PR.
Substring-match fixtures (pie chart, bar chart, schedule, trip, etc.)
cross-fired across demos with different tool surfaces and returned tool
names the target agent never registered, causing demos to render nothing
in prod when aimock handles traffic.
Fixture changes (showcase/aimock/feature-parity.json):
- Replace generic pie-chart / bar-chart matches with per-suggestion
specific phrases so gen-ui-tool-based gets render_pie_chart /
render_bar_chart directly and beautiful-chat gets pieChart / barChart
with real data (skipping the query_data two-step that caused the
infinite loop on re-matching prompts).
- Narrow schedule+meeting to the Beautiful Chat 30-minute prompt
returning scheduleTime.
- Narrow flight+fly to flights-from-SFO-to-JFK.
- Narrow background to sunset-themed-gradient.
- Remove trip, sales, pipeline, todo: substring-false-firing across
unrelated demos; interrupt/A2UI demos fall through to real LLM.
Guardrail (showcase/scripts/validate-fixture-tool-surface.ts):
- Pure validate() cross-references every fixture's tool-call name
against the tool surface of each demo whose suggestion prompt contains
the fixture's match substring. Loud failure when the fixture returns a
tool the demo's agent does not register.
- CLI walks packages/ collecting suggestions from page.tsx + hooks/,
frontend tools from useComponent / useHumanInTheLoop / useFrontendTool
/ useRenderTool / useDefaultRenderTool, and backend tools via route.ts
agentId->graphId map + langgraph.json graph->file + @tool decorators.
- 7 vitest cases written TDD-first covering the drift detection,
content-only fixtures, case-insensitivity, and multi-tool responses.
- Current state: 33 fixtures x 191 demos, no drift. Counterfactual
(reverting the pie-chart fix) correctly flags gen-ui-tool-based and
declarative-gen-ui.
Also fixes a separate runtime bug in the langgraph-python package
Dockerfile: WORKDIR /app left /app owned by root; the app user could
not create the .langgraph_api cache dir LangGraph's in-memory runtime
needs, so the agent crashed on boot. Added a non-recursive chown
app:app /app (preserves the original perf intent of the explicit
--chown on COPY, which avoided a recursive chown).
github-actions[bot]