Commit Graph

650 Commits

Author SHA1 Message Date
Jordan Ritter bc35c2e8ad feat(showcase): validation tooling suite (Bundle 3 consolidation) (#4018)
## Showcase validation tooling (Bundle 3)

Ships three CLI validators, a shared parsing lib, and two CI workflows
that enforce consistency across the 17 showcase packages and detect
drift before it lands on main. Consolidates four earlier tooling PRs
(#3985, #3987, #3995, #3996).

## What's in the box

### `showcase/scripts/` — three validators

| Tool | Purpose | Exit codes |
|------|---------|-----------|
| `audit.ts` | Cross-checks manifest-declared demos against
`tests/e2e/*.spec.ts` and `qa/*.md`, plus `examples/integrations/`
provenance via `SLUG_TO_EXAMPLES` / `FALLBACK_MAP` | 0 ok, 1 anomalies,
2 invalid-input, 3 unreadable, 4 internal, 5 strict-warnings |
| `validate-pins.ts` | Framework-dep pin-drift between
`showcase/packages/*/` and their dojo `examples/integrations/*/`
counterparts. Parses package.json, requirements.txt, pyproject.toml
(Poetry + PEP 621) | 0 ok, 1 drift, 2 internal, 3 unreadable |
| `validate-parity.ts` | Enforces demo ↔ spec ↔ qa coverage per package
with a monotonic demo-count floor | 0 ok, 1 warnings, 2 invalid-input, 3
unreadable, 4 internal, 5 must-failure |

### `showcase/scripts/lib/` — shared primitives

- **`slug-map.ts`** — single-source-of-truth `ENTRIES` for the showcase
slug taxonomy;
`BORN_IN_SHOWCASE`/`SLUG_MAP`/`SLUG_TO_EXAMPLES`/`FALLBACK_MAP` derived
and frozen at module load. `SlugEntry` is a discriminated union that
makes illegal states (born-in-showcase with non-empty examples)
unrepresentable. `freezeSet`/`freezeMap` helpers install throwing
replacements via `Object.defineProperty({writable:false,
configurable:false})` so `Set.add` / `Map.set` truly fail at runtime.
- **`manifest.ts`** — `parseManifest` returns a tagged `ParsedManifest`
union (`ok` | `missing` | `malformed{subkind: "syntax"|"shape"}` |
`unreadable`) with a never-throws content contract. Uses `statSync` +
errno inspection (not `existsSync`, which conflates ENOENT with EACCES).
`DemoId` is a branded string minted only via `createDemoId`.

### `.github/workflows/` — CI enforcement

- **`showcase_validate.yml`** — runs on PR and push-to-main. Enforces
the e2e-spec floor, runs the validators, and drives the pin-drift
ratchet.
- **`showcase_drift-report.yml`** — weekly Monday 10:00 UTC +
workflow_dispatch. Computes `set_status` (OK / SET DRIFTED / COUNT
DRIFTED) and posts to Slack.

Both workflows run on `depot-ubuntu-24.04-4` (Startup plan, unlimited)
for persistent pnpm/npm cache across runs.

## The pin-drift ratchet

`validate-pins.ts` currently finds **111 existing pin-drift failures**
across 12 showcase packages. Rather than block the PR on those, we
baseline them in `showcase/scripts/fail-baseline.json` and ratchet:

- `validatePinsFailCount` must not increase; CI tells you to ratchet
down when it decreases.
- `validatePinsFailHash` is SHA-256 of the sorted-uniqued `[FAIL]` set.
When the count is equal but the hash differs, a fail healed AND a new
one regressed — CI prints the diff and fails.
- `baselineDemoCount` (9) is the single source of truth for the e2e-spec
floor; consumed by both the workflow and `validate-parity.ts` with sync
enforced by a dedicated regression test.

Tracked in #4047. The 111 failures are mostly showcase packages pinning
`@copilotkit/*` to the `next` dist-tag while dojo pins concrete
versions; direction of fix (align showcase → dojo vs. bump dojo →
showcase) is a separate versioning decision outside this PR.

## Correctness posture

- **977 tests**, 13 files, covering every `Anomaly` / `PackageIssue` /
`ParsedManifest` variant in-process and via subprocess CLI for every
exit code. EACCES/ENOTDIR/TOCTOU paths are exercised via chmod probes
(with `it.skipIf` fallback when CI runs as root) and path-filtered
`vi.spyOn` fall-throughs.
- **`fs.statSync` + errno everywhere** — `fs.existsSync` silently
collapses ENOENT with EACCES and is a known anti-pattern in validation
tooling; the codebase uses structured errno discrimination throughout.
- **Tagged discriminated unions with exhaustive `switch` + `never`
guards** — `bucketFor` in `audit.ts`, `deriveMessage` in
`validate-parity.ts`. Adding a new variant without wiring every site is
a compile error.
- **Partial-report preservation** — when an infra error hits
mid-slug-loop, `UnreadableInputError.partialReport` carries
already-collected drift findings so the top-level catch prints them
before exiting 3. One bad package never orphans signal for the rest.
- **Per-slug isolation** — in `validate-parity.ts runParityImpl`, each
slug's audit is wrapped; a crash surfaces as a `crashed` `PackageIssue`
and forces `EXIT_INTERNAL` without aborting siblings.
- **Pipefail + scoped `|| true`** — every workflow step uses `set -euo
pipefail` with grep's no-match tolerance wrapped in `{ grep || true; }`
so producer failures (sort, shasum, cut) still surface.

## Diff

+16,455 / −18 across 34 files (26 source + 5 fixture trees + 2 workflows
+ 1 baseline).

Commits grouped by purpose:

1. `chore(showcase/scripts)`: vitest config + test deps
2. `feat(showcase/scripts)`: shared slug-map and manifest parsing lib
3. `feat(showcase/scripts)`: audit.ts coverage auditor
4. `feat(showcase/scripts)`: validate-pins.ts pin-drift validator
5. `feat(showcase/scripts)`: validate-parity.ts demo/spec/qa parity
validator
6. `ci(showcase)`: validation + weekly drift-report workflows (Depot
runners)

## Test plan

- [x] `pnpm vitest run` in `showcase/scripts/` — 977/977 green
- [x] Exit-code taxonomy verified end-to-end via subprocess tests for
every documented code
- [x] EACCES/ENOENT/ENOTDIR routing verified in all three validators
- [x] Partial-report preservation verified in both in-process and
subprocess paths
- [x] Per-slug crash isolation verified (one broken slug does not orphan
siblings)
- [x] Baseline sync contract (`BASELINE_DEMO_COUNT` ↔
`fail-baseline.json.baselineDemoCount`) pinned by test
- [ ] First CI run on Depot to confirm cold-cache timing (expected 5–8m
vs. 18–20m on ubuntu-latest)

Refs: [Full Action
Inventory](https://www.notion.so/3443aa38185281b5a1dfc6e0890264e1),
#4047
2026-04-17 23:07:14 -07:00
Jordan Ritter 5acbbcd633 feat(showcase/scripts): add validate-parity.ts demo/spec/qa parity validator
Audits each showcase package's manifest-declared demos for matching
spec (tests/e2e/*.spec.ts) and qa/*.md coverage, and enforces a
monotonic demo-count baseline via fail-baseline.json.

Key design:
- PackageIssue tagged union (13 variants) cleanly separates MUST
  errors from warnings; deriveMessage is the single renderer so new
  variants cannot emit mismatched prose.
- ProbeResult tagged union (missing | ok | unreadable) driven by
  statSync + errno inspection, distinguishing ENOENT from EACCES and
  surfacing ENOTDIR as a misconfiguration rather than a silent miss.
- runParityImpl isolates each slug's audit in try/catch; a crash in
  one slug surfaces as a crashed PackageIssue variant and forces
  EXIT_INTERNAL without aborting siblings.
- runParity never throws for content errors. InvalidBaselineError
  covers coerceBaseline failures; unknown errors route to
  EXIT_INTERNAL via formatErrorChain (walks .cause with cycle
  guard + depth cap).
- parseMainArgs rejects unrecognised flags and duplicate --baseline
  with EXIT_INVALID_INPUT (2), mirroring audit.ts parseArgs
  discipline.
- BASELINE_DEMO_COUNT default must match
  fail-baseline.json.baselineDemoCount; enforced by
  __tests__/baseline-sync.test.ts.
- Exit codes: 0 ok, 1 should-warnings-only, 2 invalid-input, 3
  unreadable, 4 internal, 5 must-failure.

Tests cover every PackageIssue variant, every exit code
(in-process + subprocess), per-slug crash isolation, EACCES routing,
ENOTDIR classification, cascade suppression when tests/e2e or qa
dirs are unreadable, and baseline coercion edge cases (leading
zeros, negative, float, hex, non-numeric).
2026-04-17 22:34:40 -07:00
Jordan Ritter cb0bf1c9b5 feat(showcase/scripts): add validate-pins.ts pin-drift validator
Compares framework dependency pins across showcase/packages/*/ and
the corresponding dojo examples/integrations/* trees, flagging drift
between the two and rejecting non-exact specs on the showcase side.

Key design:
- Parses package.json, requirements.txt, and pyproject.toml
  (including Poetry's [tool.poetry.dependencies] and PEP 621
  [project.dependencies] / optional-dependencies). Separate jsDeps
  and pythonDeps maps prevent cross-ecosystem name collisions.
- isExactSpec enforces exact-version pins per ecosystem (npm: no
  operators, workspace refs, or ranges; Python: ==X / ===X / ~=X
  with PEP 440 body). Symmetric rejection of bare MAJOR-only forms.
- parseRequirementsTxt and parsePyprojectToml thin wrappers throw
  when the detailed form produced skipped[] or dropped[] entries,
  preventing silent data loss in simpler callers.
- canonicalizeDepMap canonicalises names per PEP 503 and surfaces
  same-file collisions with differing specs as warnings.
- First-writer-wins at both file and package levels.
- UnreadableInputError carries an optional partialReport so an
  infra failure mid-slug-loop preserves already-collected drift
  findings for other slugs.
- Exit codes: 0 ok, 1 drift, 2 internal, 3 unreadable.

fail-baseline.json is the single source of truth for the CI ratchet
(validatePinsFailCount + validatePinsFailHash) and the demo-count
floor (baselineDemoCount, cross-checked against validate-parity.ts
in a dedicated sync test).

Test coverage spans every parser variant, EACCES routing via chmod
probe + fs spies, exit-code taxonomy subprocess tests, partial-report
preservation on mid-loop infra throws, and Poetry/PEP 503 edge cases
via committed fixture files under __tests__/fixtures/pins/.
2026-04-17 22:34:18 -07:00
Jordan Ritter bf603a6e4f feat(showcase/scripts): add audit.ts coverage auditor
Cross-checks each showcase package's manifest-declared demos against
the spec (tests/e2e/) and qa/ directory contents, plus
examples/integrations provenance via SLUG_TO_EXAMPLES / FALLBACK_MAP.

Key design:
- Discriminated Anomaly union with nine variants
  (count-mismatch, not-deployed, missing-examples, missing-manifest,
  malformed-manifest, unreadable-dir, unreadable-manifest,
  unreadable-examples, mapped-candidate-not-directory). bucketFor uses
  an exhaustive switch with a never guard so a new variant cannot
  silently escape routing.
- CountState tagged union separates known-count, legitimate-missing,
  and unreadable cases so an EACCES on tests/e2e/ cannot be
  misclassified as a real zero count.
- ExamplesSourceResult carries structured unreadableForSlug /
  nonDirectoryForSlug flags; classification never substring-matches
  the human-readable warning text.
- SHOWCASE_AUDIT_ROOT env var is validated with statSync + distinct
  error messages for ENOENT vs ENOTDIR vs EACCES.
- Text and --json output modes. Exit-code taxonomy: 0 ok, 1 anomalies,
  2 invalid-input, 3 unreadable, 4 internal, 5 strict-warnings.
- Deep-freezes AuditReport.packages and anomalies before return.

Tests cover every Anomaly variant, every exit code (in-process + CLI
subprocess), buildReport bucket exhaustiveness, EACCES routing via
path-filtered fs spies, TOCTOU ENOENT races, and the --columns filter
surface.
2026-04-17 22:33:58 -07:00
Jordan Ritter 4373629763 feat(showcase/scripts): add shared slug-map and manifest parsing lib
Two foundational modules consumed by all three validators:

- lib/slug-map.ts: single source of truth for the showcase slug
  taxonomy. ENTRIES array is the sole declaration; BORN_IN_SHOWCASE,
  SLUG_MAP, SLUG_TO_EXAMPLES, and FALLBACK_MAP are derived at module
  load and frozen via freezeSet/freezeMap/freezeMap2D helpers
  (defineProperty-based to block Set.add / Map.set at runtime).
  SlugEntry is a tagged union: born-in-showcase variants have empty
  examples and no fallback; non-born variants carry a non-empty
  tuple. Each slug passes isShowcaseSlug at module load.

- lib/manifest.ts: parseManifest returns a tagged ParsedManifest
  union (ok | missing | malformed | unreadable) with never-throws
  content contract. Uses statSync + errno inspection rather than
  existsSync to distinguish ENOENT from EACCES/ENOTDIR. DemoId is a
  branded string minted only through createDemoId. Empty-string
  dirSlug is rejected as a caller bug; undefined opts out of the
  slug-match check. Deep-freezes the returned Manifest.
2026-04-17 22:33:39 -07:00
Jordan Ritter 707cca1397 chore(showcase/scripts): add vitest config and test dependencies
Configure file-level isolation (fileParallelism: false) to prevent
cross-test env-var contamination when the three validators mutate
process.env.VALIDATE_PARITY_REPO_ROOT / VALIDATE_PINS_REPO_ROOT /
SHOWCASE_AUDIT_ROOT for fixture tmpdirs. Adds scripts test deps to
showcase/scripts/package.json.
2026-04-17 22:33:22 -07:00
Jordan Ritter c8276ee923 feat(showcase): internal feature-matrix shell + canonical demo/code routes (#4039)
## Summary

Adds a new internal-facing showcase app — `showcase/shell-internal` —
that renders a **feature × integration grid**. Each cell links to one of
two new **canonical standalone routes** on the main `shell` app, or
shows a red ✗ when the feature isn't supported.

## What's new

### 1. Two canonical standalone routes in `showcase/shell`

These give every (integration × feature) pair a single, embeddable URL
for each artifact — useful for docs, marketing, and tooling.

- **`/integrations/[slug]/[demo]/preview`** — iframe-only hosted demo,
no chrome
- **`/integrations/[slug]/[demo]/code`** — code viewer only. Supports
URL params for future refinements:
  - `?file=<filename>` — which file tab to show
  - `?lines=10-20` or `?lines=10-20,35` — highlight specific line ranges

Example:
`/integrations/langgraph-python/agentic-chat/code?file=page.tsx&lines=15-22`

### 2. `showcase/shell-internal` — a new Next.js app on port 3002

- Single grid page: **rows = features**, **columns =
integrations/frameworks** (transpose of shell's existing `/matrix` page,
which has integrations as rows)
- Each cell has **two mini-links** — green `▶ demo` and blue `</> code`
— pointing at the canonical `shell` routes, or a red `✗` if not
supported
- Reads `showcase/shell/src/data/registry.json` directly via relative
import — single source of truth, no duplicate data
- `NEXT_PUBLIC_SHELL_URL` env var (default `http://localhost:3000`) to
point the cells at a deployed `shell` in non-local environments

### 3. Small fix: drop `--turbopack` from shell's dev script

`showcase/shell`'s Next.js 15.4.10 turbopack panics (`"Next.js package
not found"`) on this repo's multi-lockfile layout. Switching to webpack
dev resolves it; production builds (which don't use turbopack) are
unaffected.

## Why two apps instead of one

`shell-internal` could have hosted the demo and code pages itself, but
keeping them in `shell`:

- Makes the canonical URLs reusable outside internal ops (docs,
marketing, linking into product)
- Avoids duplicating the demo-rendering and code-viewer plumbing across
two apps

Internal shell stays a pure overview.

## Test plan

- [ ] `cd showcase/shell && npm run dev` — confirm shell starts on :3000
(webpack, no turbopack panic)
- [ ] `cd showcase/shell-internal && npm install && npm run dev` —
confirm shell-internal starts on :3002
- [ ] Open http://localhost:3002 — verify the feature × integration grid
renders
- [ ] Click a `▶ demo` cell — verify it opens
`http://localhost:3000/integrations/<slug>/<feature>/preview` with only
the iframe demo
- [ ] Click a `</> code` cell — verify it opens
`http://localhost:3000/integrations/<slug>/<feature>/code` with the code
viewer
- [ ] In the code route, try `?file=<name>` and `?lines=10-20` URL
params — verify file switches and lines highlight
- [ ] Verify red ✗ shows for unsupported (integration × feature)
combinations

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-17 19:18:59 -07:00
Atai Barkai 7bec77a13a feat(showcase/langgraph-python): Controlled Gen UI with bar + pie charts
Rework the Controlled Generative UI demo to be minimal, self-contained,
and render charts via `useComponent`.

Frontend (`src/app/demos/gen-ui-tool-based/`):
- `page.tsx` -- `CopilotKit` + full-screen `CopilotChat` (no sidebar),
  two `useComponent` registrations (`render_bar_chart`,
  `render_pie_chart`), three chart-oriented suggestions.
- `bar-chart.tsx`, `pie-chart.tsx` -- ported from
  `examples/integrations/langgraph-python`. Each file reads top-to-
  bottom as imports -> schema -> props type -> component; colors and
  animation helpers are inlined inside the component. No shared
  chart-config module.
- Haiku card + schema removed.

Agent (`src/agents/main.py`):
- Rewritten using `create_agent` + `CopilotKitMiddleware()` so the
  LangGraph middleware injects the frontend `render_*_chart` tools into
  the model request at runtime.
- Deleted all backend-tool modules (`tools.py`, `todos.py`,
  `a2ui_dynamic_schema.py`, `a2ui_fixed_schema.py`). System prompt
  trimmed to a data-viz assistant.

Peripheral cleanups:
- `demos/agentic-chat/page.tsx` -- `change_background` tool + its
  suggestion removed; it had nothing to do with chat.
- New `demos/frontend-tools/` (In-App Actions) hosts
  `change_background` as its own demo with a `frontend_tools` agent
  name registered in `api/copilotkit/route.ts`.
- Stub `gen-ui-tool-based/agent.py` removed.
- `generate-starters.test.ts` no longer requires every python package
  to have backend tool imports -- a package with only frontend tools
  (like this controlled-gen-ui demo) is now valid.
- Whitelist generated shell data (`demo-content.json`,
  `search-index.json`, `starter-content.json`) in check-binaries hook
  -- they're generated artifacts like `package-lock.json`.
- Regenerated `demo-content.json` + `registry.json`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 17:40:04 -07:00
Atai Barkai 099c105272 feat(showcase): Docker-based local dev for Railway parity
Adds a single command to spin up the exact image Railway deploys,
for any of the 17 showcase packages, with a single shared .env:

  ./showcase/scripts/dev-local.sh up [<slug> ...]   # all if empty
  ./showcase/scripts/dev-local.sh down|build|logs|ps|ports

Pieces:
- `docker-compose.local.yml` with a service per package. Ports come
  from `shared/local-ports.json` (langgraph-python -> 3100, ...).
- `.env.example` as a commit-safe template. Real `.env` is gitignored
  and fed to every container via `env_file`, so keys (OPENAI_API_KEY,
  etc.) live in one place.
- `dev-local.sh` wraps `docker compose` and handles the
  `shared_python/` / `shared_typescript/` staging step that CI does
  before `docker build` (see showcase_deploy.yml).
- Staged `shared_*` dirs added to .gitignore.

Shell wiring:
- `shell/next.config.ts` reads `shared/local-ports.json` when
  `SHOWCASE_LOCAL=1` is set and injects it as a public env.
- `/integrations/[slug]/[demo]/preview` uses that map to iframe
  `http://localhost:<port>` instead of `integration.backend_url`.
  Per-slug; any slug not running locally falls back to Railway.
  Unset SHOWCASE_LOCAL -> prod behavior, unchanged.

Full workflow + prerequisites (Colima / Docker Desktop) documented in
showcase/README.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 17:37:37 -07:00
Atai Barkai 14537d8f3b feat(showcase): reorganize feature matrix + auto-rebundle demo content
Feature registry reorganization:
- Move `tool-rendering` to generative-ui (was agent-capabilities);
  move `interactivity` category above `agent-state`.
- Split `hitl` into `hitl-in-chat` (generative-ui) + `hitl-in-app`
  (interactivity). All 17 manifests updated: feature + demo id renamed
  `hitl` -> `hitl-in-chat`; demo routes stay `/demos/hitl` so deployed
  backends are unaffected.
- Rename `Tool-Based Generative UI` -> `Controlled Generative UI`;
  drop duplicate `controlled-gen-ui` registry entry.
- Add generative-ui rows: `declarative-gen-ui`, `open-gen-ui`,
  `a2ui` (moved from a2ui category), `mcp-apps` (moved from platform).
- Rename `Frontend Tools` -> `Frontend Tools (In-app actions)`.
- Add `frontend-tools` feature to langgraph-python manifest + register
  `frontend_tools` agent name in api/copilotkit/route.ts (noise-free
  `change_background` demo split out from agentic-chat).

Bundle script improvements:
- `bundle-demo-content.ts`: resolve demo directory from `demo.route`
  instead of `demo.id`. Decouples feature id renames from on-disk
  directory names.
- `--watch` mode using native `fs.watch` over `packages/` with a
  debounced re-bundle on edits under demos/, agents/, agent/, mastra/,
  or README.md.
- `shell/package.json` dev script runs the bundler in watch mode
  alongside `next dev` via `npx concurrently -k`.

Tests updated for the rename and new counts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 17:36:00 -07:00
Jordan Ritter a7fcd2da59 fix(docs-sync): emit review_items_file output, fix auto_push fast path, tighten gates 2026-04-17 17:28:20 -07:00
Jordan Ritter ae5fc2cc0f fix(docs-sync): manifest path, add-order, exit-code, marker, stripTrailingEol, dead search 2026-04-17 17:28:20 -07:00
Jordan Ritter 157cf7d4a0 fix(docs-sync): harden shell injection, add needs-review Slack, fix silent re-resolution + PR collision 2026-04-17 17:28:20 -07:00
Jordan Ritter 5bb7c19cab ci(docs-sync): auto-open PR instead of warn-and-skip on conflict 2026-04-17 17:28:20 -07:00
Jordan Ritter 88a0a6e594 fix(showcase-scripts): keep /ok probe for langgraph starters in generator
langgraph-python, langgraph-fastapi, and langgraph-typescript starters run
langgraph_cli dev which exposes /ok (not /health). The template now probes
/health, so the generator must rewrite this back to /ok for langgraph-* only,
otherwise drift-check fails and regenerating would break those starters'
health routes.
2026-04-17 16:29:14 -07:00
Jordan Ritter 27f886e59c fix(showcase): complete open-gen-ui scrub — 5→4 count, schema enum, fixture, full regen 2026-04-16 18:30:02 -07:00
Jordan Ritter 513d72eb54 fix: address R2 — add --validate-on-load to template generator + docstrings 2026-04-16 13:14:22 -07:00
Jordan Ritter d1928cdb67 fix: address CR findings on aimock validate-on-load hardening
- Add --validate-on-load to all aimock invocations (4 workflows/scripts
  + 13 integration docker-compose files)
- Replace hardcoded 2-file fixture list with dynamic discovery across
  showcase/, examples/integrations/*/, scripts/doc-tests/ (16 fixtures)
- Add sanity check to prevent silent zero-test pass when discovery fails
- Extend showcase_validate.yml path filter to trigger on
  examples/integrations/**/fixtures/** and scripts/doc-tests/fixtures/**
- Import and use ValidationResult type for callback parameters
- Fix scripts/doc-tests/fixtures/default.json to use { fixtures: [...] }
  envelope shape
2026-04-16 13:00:01 -07:00
Jordan Ritter 3f62cd45e6 fix: validate aimock fixtures at load time to prevent runtime 500s
Follow-up to #3971. aimock supports fixture schema validation at startup via
--validate-on-load, but it's opt-in. The showcase Dockerfile did not pass
the flag, so fixtures with unrecognized response keys (e.g. "text" instead
of "content") loaded silently and only failed at request time with HTTP 500.
That's what crashed crewai-crews on startup.

Changes:
- showcase/aimock/Dockerfile: pass --validate-on-load so broken fixtures
  fail the container boot, not individual requests.
- showcase/scripts/__tests__/aimock-fixtures.test.ts: new vitest spec that
  loads feature-parity.json and smoke.json via @copilotkit/aimock's
  loadFixtureFile + validateFixtures and asserts zero errors. Runs as part
  of the existing showcase-validate CI workflow.
- showcase/scripts/package.json: add @copilotkit/aimock dependency for the
  validator import.

Verified red-green: with the pre-#3971 broken "text" fixtures, validateFixtures
flags 5 errors; post-#3971 it returns zero. Docker red-green: container with
an intentionally broken fixture fails to start with "Validation failed: 1
error(s)" and non-zero exit.
2026-04-16 12:32:24 -07:00
Jordan Ritter e68d546b11 fix: CR findings — type safety, error handling, test coverage (#3945)
## Summary

Addresses all findings from 7-agent MSAL code review of the starter
crash fix PRs (#3933, #3941, #3943).

**Agent server (claude-sdk-typescript):**
- Remove 6 unnecessary `as any` casts — use TypeScript discriminated
union narrowing on `msg.role`
- Add `ANTHROPIC_API_KEY` validation on startup (exit if missing)
- Add `console.warn` logging to empty catch blocks (tool args + schema
parse failures)
- Sanitize error messages sent to client — no raw `err.message` in SSE
stream
- Fix misleading comment about system/developer role handling
- Set explicit 2MB JSON body limit on Express
- Remove unused `assistantMsgId` variable

**Generation script:**
- Remove dead `_agentDir` parameter from `rewritePythonImports`
- Add warning when `PIN_OVERRIDES` dep not found in framework
dependencies

**Tests (3 new):**
- Context-aware import rewriting for files inside `tools/` directory
- `from agents.X import` relative import rewriting
- `PIN_OVERRIDES` integration test for mastra version pinning

## Test plan

- [x] 254 generate-starters tests pass (3 new)
- [x] Drift check passes
- [x] Pre-commit hooks pass
2026-04-15 21:15:58 -07:00
Jordan Ritter c651f09669 fix: address CR Round 3 findings — type correctness, dead code, 4 new tests
Agent server:
- Use ContentBlockParam[] (input type) not ContentBlock[] (response type),
  remove citations: null workaround
- Type emit helper as BaseEvent, remove as-any cast
- Remove unused toolCallArgs accumulator
- Remove (c: any) annotation on context map
- Simplify Anthropic constructor (reads ANTHROPIC_API_KEY from env)
- Remove always-true health endpoint field

Generation script:
- Fix os usage regex from \bos[.\s] to \bos\b (catches os), os], etc.)
- Fix PIN_OVERRIDES comment (caret ranges, not concrete versions)
- Replace rewritePythonImportsInDir with forEachPyFile
- Hoist PIN_OVERRIDES to module scope

Tests (4 new, 258 total):
- import os preservation when used elsewhere in file
- import os removal when only used in sys.path.insert
- forEachPyFile skips data/ directories
- PIN_OVERRIDES warn path (no phantom deps)
2026-04-15 17:43:20 -07:00
Jordan Ritter 89b1ab0e22 fix: address CR Round 2 findings — recursive scans, regex consistency, dead code
Generation script:
- Make tools.py rename and langgraph import rewrite recursive via
  forEachPyFile helper (was only processing top-level .py files)
- Fix from-src.agents regex to use [\w.]+ for dotted sub-paths,
  matching the from-agents regex
- Simplify isInsideToolsDir branch to computed prefix variable
- Safe import-os removal: keep if os is used elsewhere in file
- Fix extractUvicornModule JSDoc example to match actual output

Agent server (claude-sdk-typescript):
- Remove dead ternary branches after API key guard
- Remove unreachable ?? "" fallback
2026-04-15 17:28:16 -07:00
Jordan Ritter 2e97925f9b fix: address CR findings — type safety, error handling, test coverage
Agent server (claude-sdk-typescript):
- Remove 6 unnecessary as-any casts, use TS discriminated union narrowing
- Add ANTHROPIC_API_KEY validation on startup (exit if missing)
- Add logging to empty catch blocks (tool args + schema parse)
- Sanitize error messages sent to client (no raw err.message in SSE)
- Fix misleading comment about system/developer role handling
- Set explicit 2MB JSON body limit
- Remove unused assistantMsgId variable

Generation script:
- Remove dead _agentDir parameter from rewritePythonImports
- Add warning when PIN_OVERRIDES dep not found in dependencies

Tests:
- Add context-aware import rewriting test (files inside tools/)
- Add from-agents.X relative import rewriting test
- Add PIN_OVERRIDES integration test for mastra version pinning
2026-04-15 17:11:03 -07:00
Jordan Ritter 39347a7ada feat: remove open-genui rendering from showcase starters (#3944)
## Summary

- Remove the Open GenUI rendering strategy from all 17 showcase starters
- Clean up all supporting files: feature registries, constraints, e2e
tests, Python middleware, docs
- Regenerate all starters from updated template

### What was removed

The "open-genui" render mode allowed agents to generate arbitrary HTML
rendered in a sandboxed iframe. This is being extricated from starters
into a separate project. The core SDK `OpenGenerativeUIRenderer`
component is **not** affected — only the showcase starter UI option.

### Files changed

- **Template**: Deleted `open-genui/` renderer, updated types.ts,
page.tsx, suggestions hook
- **Registry**: Removed `open-gen-ui` and `byoc-opengenui` features from
all registries (shell, shell-dojolike, shared)
- **Constraints**: Removed "open" constraint block and open-gen-ui
references
- **E2E tests**: Removed Open GenUI test cases from starter-e2e,
screenshots, renderer-selector specs
- **Python middleware**: Removed OPEN_GENUI_INSTRUCTION and open-genui
handling from render_mode.py
- **Starters**: Regenerated all 17 via generate-starters.ts
- **Docs**: Updated QA-COVERAGE.md pill count from 5 to 4

### Local verification

- Docker build of langgraph-python starter: success
- Rendered page shows exactly 4 render modes (Tool-Based, A2UI,
json-render, HashBrown)
- Zero "open-genui" references in rendered HTML
- 521 validation tests pass

## Test plan

- [x] Docker build succeeds locally
- [x] Rendered starter shows 4 modes, no Open GenUI
- [x] Zero grep matches for open-genui across showcase/
- [x] 521 vitest validation tests pass
- [ ] CI green

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-15 17:03:13 -07:00
github-actions[bot] 31c7380b86 style: auto-fix formatting 2026-04-15 23:52:57 +00:00
Jordan Ritter fe7129fbcf fix: context-aware Python import rewriting for files inside tools/ dir
Files inside tools/ need "from tools import X" → "from . import X"
(current package), not "from .tools import X" (sub-module). The
previous rewrite rule was correct for files outside tools/ but caused
ModuleNotFoundError for crewai-crews custom_tool.py.
2026-04-15 16:46:14 -07:00
Jordan Ritter f8bbb1390c feat: complete open-genui removal from tests, registries, middleware, and docs 2026-04-15 16:44:28 -07:00
Jordan Ritter eecb252263 fix: resolve remaining starter crashes — ESM resolution, recursive chown, mastra version pins
1. Remove nested agent/package.json in TypeScript Dockerfile template to
   prevent ESM module resolution failure (claude-sdk-typescript)
2. Use chown -R for /app in all Dockerfile templates so runtime-created
   dirs (e.g. /app/src/mastra/public) are writable by non-root user
3. Pin mastra ecosystem deps to stable versions (1.6.0/1.25.0) in
   generation script — floating beta tags caused version drift between
   mastra CLI and @mastra/core
2026-04-15 15:57:17 -07:00
Jordan Ritter cffecf1fc1 fix: resolve 3 starter crash root causes — Docker home dir, path depth, import rewriting
1. EACCES on /home/app: Dockerfile templates now create home dir for
   non-root user before USER app (langgraph-ts, claude-sdk-ts, mastra)
2. IndexError Path.parents[4]: search_flights.py schema fallback now
   guards against shallow Docker paths (agno, claude-sdk-py, ms-agent-py)
3. Wrong import path: rewritePythonImports() now converts
   from agents.X to relative imports in starter agent dirs (langroid, crewai)
2026-04-15 09:27:08 -07:00
Jordan Ritter 34be9fead9 fix: restructure spring-ai starter to use Maven standard directory layout
The generate-starters script was flattening src/main/{java,resources} into
agent/{java,resources}, breaking Maven builds. Add a post-copy step that
moves them to agent/src/main/{java,resources} as Maven expects.
2026-04-14 22:33:53 -07:00
github-actions[bot] fa8190d6b8 style: auto-fix formatting 2026-04-15 00:07:54 +00:00
Jordan Ritter 580a3e9214 fix: langgraph-python starter agent — permissions, imports, and tools.py naming collision
Root causes (verified locally with docker build + run):
1. PermissionError: non-root user can't write .langgraph_api dir — fix: chown -R app:app /app
2. ImportError: relative imports fail in langgraph_cli context — fix: absolute imports
3. ValueError: tools.py and tools/ directory collision — fix: rename to tool_wrappers.py

Tested: docker run returns {"status":"ok","agent":"ok"}
2026-04-14 17:06:02 -07:00
Jordan Ritter 9952fb1d82 fix: update test expectations for agent_server:app devScripts 2026-04-14 15:45:21 -07:00
Jordan Ritter 59c073c610 fix: add agent_server.py to all Python starters, matching demo packages
Python starters were trying to run uvicorn with agent.<module>:app but
those modules don't export a FastAPI app. The demo packages use
agent_server.py as the FastAPI wrapper, so starters need it too.

Changes:
- Copy agent_server.py from each demo package into starter root,
  rewriting "from agents." to "from agent." for the starter layout
- Update all non-langgraph Python devScripts to use agent_server:app
- Update Dockerfile.python to COPY agent_server.py for non-langgraph
- Update getEntrypointBlock() generic Python to use agent_server:app
- Make langgraph-fastapi use langgraph_cli dev like langgraph-python
  (it was incorrectly configured as a uvicorn-based starter)
- Regenerate all starters
2026-04-14 15:41:34 -07:00
Jordan Ritter 695c945d3e fix: align langgraph-python starter with working demo package layout
The langgraph-python starter crashes on Railway because of a directory
structure mismatch: agents were in agent/ with langgraph.json inside
agent/, causing langgraph_cli to fail resolving module paths.

Match the demo package's working layout:
- Move agents from agent/ to src/agents/ (same as demo)
- Put langgraph.json at project root (same as demo)
- Point entrypoint at root langgraph.json instead of agent/langgraph.json
- Templatize Dockerfile.python with AGENT_DIR and DOCKER_EXTRA_COPY vars
- Skip langgraph.json path rewriting when agentDir matches agentSourceDir
- Update tests to use framework agentDir instead of hardcoding "agent"

Regenerate all 17 starters.
2026-04-14 14:37:37 -07:00
Jordan Ritter 6a8b754689 fix: resolve langgraph-python starter crash from wrong graph path and fragile imports
The langgraph.json in the starter was copied verbatim from the demo package
with path ./src/agents/main.py:graph, but the starter flattens agents into
the agent/ dir so the correct path is ./main.py:graph.

Also wraps A2UI agent imports in try/except so the starter starts cleanly
even if copilotkit's ToolRuntime patch isn't available — these are optional
features that shouldn't prevent the core agent from running.

Fixes the generate-starters.ts script to rewrite langgraph.json graph paths
when copying from packages to starters, preventing this drift in the future.
2026-04-14 14:12:03 -07:00
Jordan Ritter 55932c5441 fix: correct starter Dockerfiles for spring-ai (mvnw), ms-agent-dotnet (adduser), mastra (agent dir)
spring-ai: Use maven:3-eclipse-temurin-21 image with mvn instead of
missing ./mvnw wrapper.

ms-agent-dotnet: Replace addgroup/adduser (not available on aspnet:9.0)
with groupadd/useradd. Applied same fix to all four Dockerfile templates
for consistency.

mastra: Make Dockerfile.typescript template use {{AGENT_DIR}} variable
so mastra correctly gets src/mastra/ instead of hardcoded agent/.
Updated generate-starters.ts to substitute variables in Dockerfiles.
2026-04-14 13:48:11 -07:00
Jordan Ritter a48ae18fbf fix: update QA sync parent page to new Showcase QA Instructions under QA Root 2026-04-14 13:31:04 -07:00
Jordan Ritter d8d889aebf chore: remove dead demo-wrapper/error-boundary files, regenerate shell data
Remove orphaned demo-wrapper.tsx (1 file) and error-boundary.tsx (17 files)
that still imported from the deleted @copilotkit/showcase-shared.
Regenerate registry.json, demo-content.json, and starter-content.json.
2026-04-14 12:57:32 -07:00
Jordan Ritter 5ac07d4d9b feat: CI, shell, and aimock integration for showcase starters
CI:
- Add 17 starter services to showcase deploy workflow with Railway IDs
- Add drift detection workflow (triggers on starters, packages, scripts, shared)
- Remove shared_frontend copy step for starter builds

Shell:
- Update clone command to npx degit with clipboard fallback
- Update starter content bundler for full component tree + .java support
- Add clone_command to manifest schema and registry types

Aimock:
- Expand feature-parity.json from 18 to 37 fixture rules
- Add docker-compose.packages.yml for CI aimock sidecar (strict mode)
- Add run-e2e-with-aimock.sh convenience script
2026-04-14 12:52:52 -07:00
Jordan Ritter 7012a7e07b feat: add starter template, generation script, and comprehensive test suite
Canonical template at showcase/starters/template/ with Sales Dashboard
hero page, 5 GenUI rendering strategies, per-language Dockerfiles
(non-root user, agent health checks), and self-contained agent backends.

Generation script at showcase/scripts/generate-starters.ts:
- Python sys.path.insert removal + relative import rewriting
- TypeScript shared-tools import rewriting
- Deterministic output (sorted devDependencies, stable file ordering)
- --check mode for CI drift detection
- Strict error handling (throws on missing required inputs)

537 tests: generation unit tests (transform functions, entrypoint blocks),
cross-starter consistency checks (270), Playwright e2e interaction tests
(renderer switching, deal creation, suggestion verification).
2026-04-14 12:51:44 -07:00
github-actions[bot] 92a84f004c style: auto-fix formatting 2026-04-10 13:30:24 -07:00
Jordan Ritter 80ea96fefe feat: Switch preview captures from GIF to MP4 with GitHub Release storage 2026-04-10 13:30:24 -07:00
Jordan Ritter 4bcb7e17d5 feat: update showcase generator to produce test files 2026-04-09 08:33:41 -07:00
Jordan Ritter 204dd29a97 fix: standardize health checks on /health port 8000 2026-04-08 21:22:35 -07:00
Jordan Ritter 6d05499ecd feat: add automated docs sync from main with transform pipeline 2026-04-08 19:59:31 -07:00
Jordan Ritter 31f046d0ac feat: add SEO redirect infrastructure with PostHog tracking 2026-04-08 19:59:30 -07:00
Jordan Ritter 101a783c0c fix: replace TODO placeholders and fix markdown rendering in demo content 2026-04-08 19:59:30 -07:00
Jordan Ritter 759ffeb29a feat: add animated preview GIFs with per-demo capture automation 2026-04-08 19:59:30 -07:00
Jordan Ritter 479e62cb7f feat: add unified showcase platform with CI, Docker starters, and 13 deployed integrations 2026-04-08 19:59:29 -07:00