The validator was not considering the aimock match.toolName field when
checking for drift. When a fixture has toolName set, aimock only fires
it for agents that register that tool -- skip demos that don't have it.
Delete the pre-generated starters/ directory tree (previously
synced from packages/ by generate-starters.ts). Add
extract-starter.ts which produces a starter tarball on demand
from any integration. Move shared starter template files to
showcase/shared/starter-template/.
Substring-match fixtures (pie chart, bar chart, schedule, trip, etc.)
cross-fired across demos with different tool surfaces and returned tool
names the target agent never registered, causing demos to render nothing
in prod when aimock handles traffic.
Fixture changes (showcase/aimock/feature-parity.json):
- Replace generic pie-chart / bar-chart matches with per-suggestion
specific phrases so gen-ui-tool-based gets render_pie_chart /
render_bar_chart directly and beautiful-chat gets pieChart / barChart
with real data (skipping the query_data two-step that caused the
infinite loop on re-matching prompts).
- Narrow schedule+meeting to the Beautiful Chat 30-minute prompt
returning scheduleTime.
- Narrow flight+fly to flights-from-SFO-to-JFK.
- Narrow background to sunset-themed-gradient.
- Remove trip, sales, pipeline, todo: substring-false-firing across
unrelated demos; interrupt/A2UI demos fall through to real LLM.
Guardrail (showcase/scripts/validate-fixture-tool-surface.ts):
- Pure validate() cross-references every fixture's tool-call name
against the tool surface of each demo whose suggestion prompt contains
the fixture's match substring. Loud failure when the fixture returns a
tool the demo's agent does not register.
- CLI walks packages/ collecting suggestions from page.tsx + hooks/,
frontend tools from useComponent / useHumanInTheLoop / useFrontendTool
/ useRenderTool / useDefaultRenderTool, and backend tools via route.ts
agentId->graphId map + langgraph.json graph->file + @tool decorators.
- 7 vitest cases written TDD-first covering the drift detection,
content-only fixtures, case-insensitivity, and multi-tool responses.
- Current state: 33 fixtures x 191 demos, no drift. Counterfactual
(reverting the pie-chart fix) correctly flags gen-ui-tool-based and
declarative-gen-ui.
Also fixes a separate runtime bug in the langgraph-python package
Dockerfile: WORKDIR /app left /app owned by root; the app user could
not create the .langgraph_api cache dir LangGraph's in-memory runtime
needs, so the agent crashed on boot. Added a non-recursive chown
app:app /app (preserves the original perf intent of the explicit
--chown on COPY, which avoided a recursive chown).
github-actions[bot]