Cross-checks each showcase package's manifest-declared demos against
the spec (tests/e2e/) and qa/ directory contents, plus
examples/integrations provenance via SLUG_TO_EXAMPLES / FALLBACK_MAP.
Key design:
- Discriminated Anomaly union with nine variants
(count-mismatch, not-deployed, missing-examples, missing-manifest,
malformed-manifest, unreadable-dir, unreadable-manifest,
unreadable-examples, mapped-candidate-not-directory). bucketFor uses
an exhaustive switch with a never guard so a new variant cannot
silently escape routing.
- CountState tagged union separates known-count, legitimate-missing,
and unreadable cases so an EACCES on tests/e2e/ cannot be
misclassified as a real zero count.
- ExamplesSourceResult carries structured unreadableForSlug /
nonDirectoryForSlug flags; classification never substring-matches
the human-readable warning text.
- SHOWCASE_AUDIT_ROOT env var is validated with statSync + distinct
error messages for ENOENT vs ENOTDIR vs EACCES.
- Text and --json output modes. Exit-code taxonomy: 0 ok, 1 anomalies,
2 invalid-input, 3 unreadable, 4 internal, 5 strict-warnings.
- Deep-freezes AuditReport.packages and anomalies before return.
Tests cover every Anomaly variant, every exit code (in-process + CLI
subprocess), buildReport bucket exhaustiveness, EACCES routing via
path-filtered fs spies, TOCTOU ENOENT races, and the --columns filter
surface.
Rework the Controlled Generative UI demo to be minimal, self-contained,
and render charts via `useComponent`.
Frontend (`src/app/demos/gen-ui-tool-based/`):
- `page.tsx` -- `CopilotKit` + full-screen `CopilotChat` (no sidebar),
two `useComponent` registrations (`render_bar_chart`,
`render_pie_chart`), three chart-oriented suggestions.
- `bar-chart.tsx`, `pie-chart.tsx` -- ported from
`examples/integrations/langgraph-python`. Each file reads top-to-
bottom as imports -> schema -> props type -> component; colors and
animation helpers are inlined inside the component. No shared
chart-config module.
- Haiku card + schema removed.
Agent (`src/agents/main.py`):
- Rewritten using `create_agent` + `CopilotKitMiddleware()` so the
LangGraph middleware injects the frontend `render_*_chart` tools into
the model request at runtime.
- Deleted all backend-tool modules (`tools.py`, `todos.py`,
`a2ui_dynamic_schema.py`, `a2ui_fixed_schema.py`). System prompt
trimmed to a data-viz assistant.
Peripheral cleanups:
- `demos/agentic-chat/page.tsx` -- `change_background` tool + its
suggestion removed; it had nothing to do with chat.
- New `demos/frontend-tools/` (In-App Actions) hosts
`change_background` as its own demo with a `frontend_tools` agent
name registered in `api/copilotkit/route.ts`.
- Stub `gen-ui-tool-based/agent.py` removed.
- `generate-starters.test.ts` no longer requires every python package
to have backend tool imports -- a package with only frontend tools
(like this controlled-gen-ui demo) is now valid.
- Whitelist generated shell data (`demo-content.json`,
`search-index.json`, `starter-content.json`) in check-binaries hook
-- they're generated artifacts like `package-lock.json`.
- Regenerated `demo-content.json` + `registry.json`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add --validate-on-load to all aimock invocations (4 workflows/scripts
+ 13 integration docker-compose files)
- Replace hardcoded 2-file fixture list with dynamic discovery across
showcase/, examples/integrations/*/, scripts/doc-tests/ (16 fixtures)
- Add sanity check to prevent silent zero-test pass when discovery fails
- Extend showcase_validate.yml path filter to trigger on
examples/integrations/**/fixtures/** and scripts/doc-tests/fixtures/**
- Import and use ValidationResult type for callback parameters
- Fix scripts/doc-tests/fixtures/default.json to use { fixtures: [...] }
envelope shape
Follow-up to #3971. aimock supports fixture schema validation at startup via
--validate-on-load, but it's opt-in. The showcase Dockerfile did not pass
the flag, so fixtures with unrecognized response keys (e.g. "text" instead
of "content") loaded silently and only failed at request time with HTTP 500.
That's what crashed crewai-crews on startup.
Changes:
- showcase/aimock/Dockerfile: pass --validate-on-load so broken fixtures
fail the container boot, not individual requests.
- showcase/scripts/__tests__/aimock-fixtures.test.ts: new vitest spec that
loads feature-parity.json and smoke.json via @copilotkit/aimock's
loadFixtureFile + validateFixtures and asserts zero errors. Runs as part
of the existing showcase-validate CI workflow.
- showcase/scripts/package.json: add @copilotkit/aimock dependency for the
validator import.
Verified red-green: with the pre-#3971 broken "text" fixtures, validateFixtures
flags 5 errors; post-#3971 it returns zero. Docker red-green: container with
an intentionally broken fixture fails to start with "Validation failed: 1
error(s)" and non-zero exit.
## Summary
Addresses all findings from 7-agent MSAL code review of the starter
crash fix PRs (#3933, #3941, #3943).
**Agent server (claude-sdk-typescript):**
- Remove 6 unnecessary `as any` casts — use TypeScript discriminated
union narrowing on `msg.role`
- Add `ANTHROPIC_API_KEY` validation on startup (exit if missing)
- Add `console.warn` logging to empty catch blocks (tool args + schema
parse failures)
- Sanitize error messages sent to client — no raw `err.message` in SSE
stream
- Fix misleading comment about system/developer role handling
- Set explicit 2MB JSON body limit on Express
- Remove unused `assistantMsgId` variable
**Generation script:**
- Remove dead `_agentDir` parameter from `rewritePythonImports`
- Add warning when `PIN_OVERRIDES` dep not found in framework
dependencies
**Tests (3 new):**
- Context-aware import rewriting for files inside `tools/` directory
- `from agents.X import` relative import rewriting
- `PIN_OVERRIDES` integration test for mastra version pinning
## Test plan
- [x] 254 generate-starters tests pass (3 new)
- [x] Drift check passes
- [x] Pre-commit hooks pass
Agent server (claude-sdk-typescript):
- Remove 6 unnecessary as-any casts, use TS discriminated union narrowing
- Add ANTHROPIC_API_KEY validation on startup (exit if missing)
- Add logging to empty catch blocks (tool args + schema parse)
- Sanitize error messages sent to client (no raw err.message in SSE)
- Fix misleading comment about system/developer role handling
- Set explicit 2MB JSON body limit
- Remove unused assistantMsgId variable
Generation script:
- Remove dead _agentDir parameter from rewritePythonImports
- Add warning when PIN_OVERRIDES dep not found in dependencies
Tests:
- Add context-aware import rewriting test (files inside tools/)
- Add from-agents.X relative import rewriting test
- Add PIN_OVERRIDES integration test for mastra version pinning
The generate-starters script was flattening src/main/{java,resources} into
agent/{java,resources}, breaking Maven builds. Add a post-copy step that
moves them to agent/src/main/{java,resources} as Maven expects.
The langgraph-python starter crashes on Railway because of a directory
structure mismatch: agents were in agent/ with langgraph.json inside
agent/, causing langgraph_cli to fail resolving module paths.
Match the demo package's working layout:
- Move agents from agent/ to src/agents/ (same as demo)
- Put langgraph.json at project root (same as demo)
- Point entrypoint at root langgraph.json instead of agent/langgraph.json
- Templatize Dockerfile.python with AGENT_DIR and DOCKER_EXTRA_COPY vars
- Skip langgraph.json path rewriting when agentDir matches agentSourceDir
- Update tests to use framework agentDir instead of hardcoding "agent"
Regenerate all 17 starters.
Jordan Ritter