Replace the regex-over-YAML options-block scanner with a real YAML
parse + typed navigation down on.workflow_dispatch.inputs.service.
options. The previous regex depended on a brittle terminator
(`^\S | \n\s*\w+:\s*\n\s{6,}\w+:|\nconcurrency:`) that
would silently break under trivial reformats — adding a top-level key
after `on:` or reordering `concurrency:` would have produced a
false-positive pass.
ALL_SERVICES stays regex-scanned because its matrix is embedded as a
bash heredoc inside a run: step, not as a YAML sub-structure. We do
locate the step via YAML (jobs.detect-changes.steps[*] with run body
referencing ALL_SERVICES) and only then scan `"dispatch_name":"X"`
occurrences inside that step's run body. Cannot JSON.parse the matrix
directly — it interpolates ${{ github.sha }} expressions that aren't
valid JSON pre-execution.
NIT 2: drop the dual `import.meta.url === \`file://${argv[1]}\``
branch and keep only the canonical `fileURLToPath(import.meta.url)`
comparison. The file:// form was belt-and-suspenders for a
pre-fileURLToPath era of Node; Node 20+ handles the modern form
uniformly.
Verified:
- All 12 validator tests pass (substring-spoof, missing-from-both,
template-excluded, empty-starters, missing-workflow).
- Real showcase_deploy.yml returns OK: all 17 starter(s) registered.
Every peer validator under showcase/scripts/ already has a matching
__tests__ suite — this one shipped without. Model the new suite on
validate-parity.test.ts: per-test tmpdir fixtures + CLI subprocess
runs gated by VALIDATE_WORKFLOW_STARTERS_REPO_ROOT.
Extend the existing env-var override to re-home STARTERS_DIR as well
(previously it only redirected .github/). One env var, one root,
matches the pattern used by the other validators.
Coverage:
- happy path (all slugs registered, exit 0)
- slug missing from workflow_dispatch options only (exit 1)
- slug missing from ALL_SERVICES matrix only (exit 1)
- slug missing from both (exit 1, both sources named)
- empty showcase/starters/ (exit 3, refuses trivial pass)
- template/ excluded (not flagged as missing)
- substring-spoof: starter-ag2 missing vs starter-ag2-extended present
— regression guard for the word-boundary anchor
- showcase_deploy.yml absent (exit 3)
All 12 specs pass against the existing regex-based checks. The
substring-spoof case in particular pins the word-boundary contract so
a later refactor can't silently lose it.
The SERVICES=(...) array in showcase_smoke-monitor.yml's 'Check image
drift' step was a hardcoded copy of the 17 starter slugs already
declared by showcase/starters/*/ directory names. Every new starter
required a manual edit in three places; the parity validator caught
drift after the fact but couldn't prevent it.
This commit:
- Adds a sparse actions/checkout step for showcase/starters/ only.
- Replaces the literal starter-* entries with a filesystem enumeration
(for dir in showcase/starters/*/; do ... done), skipping template/.
- Fails loudly if the enumeration produces zero starters, so a broken
checkout can't silently under-check drift.
- Updates validate-workflow-starters.ts to drop the smoke-monitor check
(drift is now structurally impossible) while keeping the two remaining
literal-list checks against showcase_deploy.yml (workflow_dispatch
options must be literal pre-checkout; ALL_SERVICES matrix carries
per-starter deploy metadata like railway_id).
Non-starter services stay literal — they don't live under
showcase/starters/ and are provisioned differently.
The scripts under showcase/scripts/ run via tsx at runtime and had no
tsconfig.json, which left LSP falling back to no-config defaults and
flagging bogus errors ("Cannot find module 'fs' / 'path' / 'url'",
"Cannot find name 'process'"). Adds a minimal tsconfig that matches
tsx's runtime semantics: bundler resolution (so existing extensionless
relative imports keep working), ES2022 + DOM libs, Node types, strict
discriminated-union narrowing. Excludes __tests__/ to avoid surfacing
pre-existing type issues outside this PR's scope.
The starter slug list is duplicated across at least three places:
- .github/workflows/showcase_deploy.yml workflow_dispatch options
- .github/workflows/showcase_deploy.yml ALL_SERVICES matrix entries
- .github/workflows/showcase_smoke-monitor.yml SERVICES bash array
Adding a new starter under `showcase/starters/` but forgetting any of
these leaves the service deployable in theory but invisible to the
dispatch UI and/or drift detection — exactly the failure mode this PR
is trying to close.
Add `showcase/scripts/validate-workflow-starters.ts`. It enumerates
every directory under `showcase/starters/` (excluding `template/`)
and confirms `starter-<slug>` is present in each of the three
workflow locations, emitting a precise "missing from: <source>"
diagnostic per gap.
Wire it into showcase_validate.yml right after `validate-parity` so
it gates every PR + main push touching `showcase/**` or the relevant
workflow files. Also extend that workflow's `on.paths` filter to
include `showcase_deploy.yml` and `showcase_smoke-monitor.yml` so
edits to those files trigger the parity check.
Wires up a single-command path to run the full integration smoke suite
against a local Docker stack instead of Railway. Useful when Railway is
degraded (OOM, rate limits) or when testing changes that haven't been
deployed yet.
Additions:
- showcase/docker-compose.local.yml: add `aimock` as 18th service so
integration containers can reach http://aimock:4010 on the compose
network, mirroring the Railway setup where they call showcase-aimock.
- showcase/tests/e2e/integration-smoke.spec.ts: `LOCAL_PORTS=1` env
rewrites each integration's Railway URL to http://localhost:<port>
via showcase/shared/local-ports.json. Starters are skipped under this
flag because they aren't in local-ports.json.
- showcase/scripts/smoke-local.sh: orchestrates build → up → wait → run
Playwright → tear down. Supports --level=L1/L2/L3/L4, --keep, --no-build.
- showcase/tests/package.json: `pnpm smoke:local[:L1|:keep|:nobuild]`
scripts delegate to the helper.
- showcase/.env.example: document optional OPENAI_BASE_URL +
ANTHROPIC_BASE_URL (route through local aimock) and package-specific
GitHubToken + GOOGLE_API_KEY (ms-agent-dotnet, google-adk).
Verified locally: `pnpm smoke:local:L1` → 17/17 L1 green against the
local stack.
The STARTERS filter in integration-smoke.spec.ts gated on
`i.starter?.deployed === true`, but the manifest→registry bundler
(showcase/scripts/bundle-demo-content.ts + generate-registry.ts)
does not carry that field forward from the source YAML manifests.
At registry regen time (commit 14537d8f3), all `starter.deployed`
values were dropped.
Result: the filter matched zero entries, `Deployed Starters`
yielded no tests, and the starter-deployed-smoke CI workflow has
failed on every run (>15 consecutive reds starting 2026-04-18) with
Playwright erroring `No tests found.` against
`--grep "@starter-health|@starter-agent|@starter-chat"`.
Switch to integration-level `i.deployed`, which is the single
deployment flag the registry actually carries and which the
INTEGRATIONS array in the same file already uses for its own
gating. This keeps both filters anchored to the same source of
truth.
Verified locally: `--grep "@starter-health|@starter-agent|@starter-chat" --list`
now discovers 17 tests (one per deployed integration).
Real pin-fix PRs have landed on `main` since the baseline was last
refreshed, reducing the FAIL set from 111 to 109 and invalidating the
stored hash. The ratchet gate now rejects every PR (including ones
that don't touch pins) with a "ratchet down" instruction.
Refresh the baseline to reflect the actual state of `origin/main`:
validatePinsFailCount: 111 -> 109
validatePinsFailHash: 77b586b7 -> d03716b5
Values computed by running the canonical pipeline from
`.github/workflows/showcase_validate.yml` ("Run validate-pins (ratchet)"
step) against a clean `origin/main` worktree:
pnpm exec tsx showcase/scripts/validate-pins.ts 2> stderr
grep ^Summary stdout -> FAIL=109
grep '^\[FAIL\]' stderr | LC_ALL=C sort -u | shasum -a 256
-> d03716b5...f597e81d
This is a pure ratchet-down to match reality, not a policy change.
No validator behavior, workflow, or pin change is included. Actual
pin drift cleanup (109 -> 0) continues as a separate effort.
Unblocks #4068 and any other PR stalled on the same ratchet.
To support exhaustive E2E testing via multiple variants per feature ×
framework, extend the status model with an optional `variants[]` array
per demo (each variant has the same demo/code/E2E/Smoke/QA/health
breakdown) and mount five different visual treatments so we can compare
side-by-side before committing to one:
- `/variants-stack` — each variant rendered as its own mini-row
in the cell; tall cells, all info visible.
- `/variants-tabs` — tabs at the cell top, click to switch
variant; cell stays compact.
- `/variants-aggregate` — pass/total rollups per signal +
"N variants ▾" expand button to drill down.
- `/variants-grid` — mini-matrix: rows = variants, cols =
demo/code/E2E/Smoke/QA/health.
- `/variants-strip` — one colored chip per variant per signal;
hover chip for variant name, click for URL.
Refactor: the grid chrome moves to `components/feature-grid.tsx`
(accepts a `renderCell` callback). Main `/` keeps the existing
single-variant layout via `components/cell-single.tsx`. Shared badge /
links helpers live in `components/badges.tsx` and
`components/variant-pieces.tsx`.
Mock variant data is seeded on 4 demos (langgraph-python's
agentic-chat, gen-ui-tool-based, hitl-in-chat; langgraph-typescript's
agentic-chat) so each option shows variants in context alongside
ordinary cells.
Variant-specific deep links append `?variant=<name>` to the shell
preview / code / hosted URLs — the shell routes can pick that up
later to highlight variant-specific files or payloads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New `scripts/generate-status.ts` — probes every
`integration.backend_url + demo.route` in parallel (~166 URLs) and
writes health status per demo to `shell/src/data/status.json`. E2E,
Smoke, and QA stay mock with explicit `TODO(wire-*)` comments; real
readers for those ingest from `showcase_aimock-e2e.yml`,
`showcase_smoke-monitor.yml`, and `showcase_qa-sync.yml` later.
`GENERATE_STATUS_MOCK_HEALTH=1` offline override for dev.
- Health badge in the feature-matrix cell is now clickable — opens the
hosted URL (`integration.backend_url + demo.route`) in a new tab,
with a tooltip noting the last probe time + status.
- Ran the probe once against Railway: 10/22 langgraph-python demos up
(pre-merge features), 12/22 down (new demos on this branch, not yet
deployed). Other 16 integrations fully live.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each grid cell now renders a two-row readiness rollup:
demo · code
E2E ✓ Smoke ✓ QA 3d ● up
Signals:
- E2E / Smoke — pass/fail + freshness (green <6h, amber older, red fail
or no suite, gray when bundle itself is stale)
- QA — days since human sign-off (green <7d, amber <30d, red otherwise
or never)
- Health — live probe dot (up/down/unknown)
Cells with no demo show a single centered ✗ (unsupported) or `—`
(supported but no demo yet). Client-side staleness check: if
`status.json.generated_at` is older than 24h, every signal degrades
to a gray `?` and the header shows a stale-bundle warning — so a
failed cron visibly announces itself instead of silently serving
green badges.
Data layer (`src/lib/status.ts`) is a single source of truth for the
badge color/label logic. The cell component is pure presentation.
Status data is currently mock (deterministic per slug+demo) so the
visuals can be reviewed; the CI/Notion/health-probe pipeline that
writes the real `status.json` is out of scope for this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
Three showcase test suites leak working-tree drift (workflow YAMLs +
shell data JSONs) on every run. This fixes the leaks and adds a shared
`FileSnapshotRestorer` + `restoreFromGitHead` harness so the suites are
idempotent.
Also moves `showcase/scripts/vitest.config.ts` from the thread pool to
the fork pool, which is required under Node 20 for the test subprocess
churn in `validate-pins` + the three generator-invoking suites.
## What this PR does NOT fix — vitest 3.2.4 RPC timeout on Node 20
(upstream)
`unit (20.x)` still reports `Timeout calling "onTaskUpdate"` ->
ELIFECYCLE **after** all 14/14 test files and 1011/1011 tests pass.
Known upstream bug: https://github.com/vitest-dev/vitest/issues/6129.
The birpc timeout is hardcoded at 60 s (`DEFAULT_TIMEOUT = 6e4` in
vitest's bundled `index.B521nVV-.js`) and is NOT exposed to
`vitest.config.ts`. No `poolOptions.forks.*` / `teardownTimeout` /
`hookTimeout` knob influences it:
- `singleFork: true` made the run STRICTLY WORSE — only 1/14 files
completed (validate-pins consumes the whole 60 s budget on its own, run
24602985507).
- `fork-per-file` (default) gives each file a fresh RPC channel, and
every file passes — but the final pool-teardown RPC still races on Node
20 and surfaces as a process-level exit 1.
Under vitest 3.2.4 the fix requires either (a) upgrading to vitest 4.x
(out of scope — monorepo-wide upgrade), or (b) a pnpm patch against the
bundled `DEFAULT_TIMEOUT` constant (cross-cutting change; declined
here). The observable reality: this PR makes the showcase-scripts suites
idempotent and green; the remaining `unit (20.x)` redness is a known
vitest flake orthogonal to what this PR is trying to fix.
## CI-killer error (verbatim)
```
Error: Package directory already exists: /home/runner/work/CopilotKit/CopilotKit/showcase/packages/test-integration-tmp
⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯
Error: [vitest-worker]: Timeout calling "onTaskUpdate"
ELIFECYCLE Test failed.
Failed tasks:
- @copilotkit/showcase-scripts:test
```
## Root cause (fixed here)
Three test suites invoke real generator scripts that write to tracked
files OUTSIDE any tmp dir, leaking drift on every `nx run-many -t test`:
- `create-integration.test.ts` scaffolds
`showcase/packages/test-integration-tmp/` AND mutates three CI workflow
YAMLs (`showcase_deploy.yml`, `showcase_drift-detection.yml`,
`starter-smoke.yml`).
- `generate-registry.test.ts` rewrites
`showcase/shell/src/data/registry.json` + `constraints.json`.
- `bundle-demo-content.test.ts` rewrites
`showcase/shell/src/data/demo-content.json`.
## Fix (9 commits, by area of concern)
1. **`test(showcase/scripts): add shared test-cleanup snapshot/restore
helper`** — new `__tests__/test-cleanup.ts` + `__tests__/paths.ts` +
direct unit coverage in `__tests__/test-cleanup.test.ts`:
- `FileSnapshotRestorer` — snapshots file content as `Buffer`
(byte-exact, preserves non-utf8), restores only files that drifted,
writes atomically via temp-file + rename, recreates parent dirs on write
ENOENT. Temp filenames use `crypto.randomBytes(8).toString("hex")` so
concurrent writes can't collide and the `snapshot()` sweep can
unambiguously identify stragglers. On `snapshot()`, sweeps
`.<basename>.<16-hex>.tmp` stragglers **scoped to the snapshotted
basenames only**.
- `restoreFromGitHead(repoRoot, paths)` — partitions the input via `git
ls-files --error-unmatch` BEFORE any destructive op. **Narrow catch**:
only genuine exit-1 pathspec errors are treated as untracked; ENOENT /
EACCES / non-exit-1 failures re-raise with captured stderr. Uses
`execFileSync` (no shell), forces `LC_ALL=C` / `LANG=C`, scrubs all
`GIT_*` environment overrides, frozen exec options.
- `test-cleanup.test.ts` itself strips `GIT_*` from child env when it
creates tmp repos — pre-commit hooks (lefthook) run with `GIT_DIR` /
`GIT_INDEX_FILE` set, which would otherwise cause tmp-repo `git commit`
calls to write to the HOST working-tree HEAD.
2. **`fix(showcase/test-integration): clean up test-integration-tmp
between runs`** — `create-integration.test.ts` wires
`FileSnapshotRestorer` + `restoreFromGitHead` into the suite, wraps
`rmSync` in `try/finally` so workflow restoration always runs, and
migrates `execSync(string)` -> `execFileSync("npx", [...args])` via a
shared `runGenerator()` helper.
3. **`fix(showcase/test-integration): stop generate-registry +
bundle-content leaks`** — same pattern applied to
`generate-registry.test.ts` and `bundle-demo-content.test.ts`; drops a
redundant bundler pre-run in the latter.
4. **`fix(showcase/scripts): switch vitest to forks pool for Node 20
stability`** — `vitest.config.ts`: thread -> fork pool.
5. **`docs(showcase/scripts): tidy test-cleanup comments and JSDoc`** —
documentation cleanup.
6. **`fix(showcase/scripts): pin vitest to a single fork + bump teardown
timeouts`** — SUPERSEDED by commit 9 below (left in history for
auditability).
7. **`fix(showcase/scripts): add post-heal drifted-baseline guard`** —
the PR had claimed a drifted-baseline guard on CI for
`restoreFromGitHead`, but no post-heal `git diff --quiet` was actually
running. Adds the missing check: on CI, any tracked path still drifted
post-heal throws `drifted-baseline guard: post-heal diff failed`; off-CI
warns. Red-green unit coverage via a counter-based git shim that
selectively fails the N-th `diff --quiet` (so the post-heal diff is
targeted independently of the off-CI pre-checkout diff).
8. **`fix(showcase/scripts): decouple generate-registry test 2 from test
1 output`** — `sorts integrations by sort_order` was reading
`registry.json` without invoking the generator, so `afterEach(restore)`
between tests meant it was exercising the committed baseline rather than
live output. Adds a `runGenerator()` call at the top.
9. **`fix(showcase/scripts): revert singleFork — fork-per-file is
strictly better`** — empirical data from run 24602985507 proved
`singleFork: true` was worse than fork-per-file (1/14 vs 14/14 files
completing before RPC timeout). Reverts the
`poolOptions.forks.singleFork` change; keeps the 30 s `teardownTimeout`
/ `hookTimeout` bumps. Comments now accurately reflect that the RPC
timeout is upstream-hardcoded in birpc and NOT tunable via vitest
config.
## Proof of idempotence
```
pnpm nx run @copilotkit/showcase-scripts:test --skip-nx-cache
Run 1: Test Files 14 passed (14), Tests 1011 passed (1011)
Run 2: Test Files 14 passed (14), Tests 1011 passed (1011)
git status after each: only the intentional test file edits.
```
Red-green verified for the HIGH CR-findings:
- narrow `partitionTrackedPaths` catch: unit test with empty PATH
reproduces ENOENT; pre-fix hid it as "untracked", post-fix throws.
- basename-scoped tmp sweep: unit test places both target-basename and
unrelated `.something-else.<hex>.tmp`; pre-fix swept both, post-fix
sweeps only target.
- post-heal drifted-baseline guard: counter-based git shim fails the 2nd
`diff --quiet`; pre-fix tests pass (guard absent), post-fix tests throw
on CI / warn off-CI with the advertised message.
## Test plan
- [x] `pnpm nx run @copilotkit/showcase-scripts:test` passes 1011/1011
twice in a row (locally, Node 25)
- [x] Red-green: disabling `restore()` fails the regression + safety-net
tests
- [x] Red-green: disabling the post-heal drift guard fails the new guard
tests
- [x] Working tree clean after full run
- [x] `prettier --check` + `oxlint` clean on touched files
- [x] `GIT_*` scrub in test harness prevents pre-commit-hook-induced
pollution of real HEAD
## CI status
- **`unit (22.x)`**: pass
- **`unit (24.x)`**: pass
- **`unit (20.x)`**: all 14/14 files + 1011/1011 tests pass; post-suite
`onTaskUpdate` RPC timeout emits exit 1. Upstream bug
https://github.com/vitest-dev/vitest/issues/6129; not fixable at
`vitest.config.ts` level on vitest 3.2.4.
## Caveats
- Local verification ran on Node v25.8.0. No Node 20 binary on this dev
host; Node 20 CI was the final gate.
- The residual `unit (20.x)` failure is orthogonal to this PR. To
resolve it we would need to upgrade vitest to 4.x (monorepo-wide change)
or apply a pnpm patch to bump `DEFAULT_TIMEOUT` in vitest's bundled
`birpc`. Both are tracked separately.
Problem: shell's /code viewer was showing every `.py` file under
`src/agents/` for every demo of a package. Visually contaminating:
opening gen-ui-tool-based (Controlled Gen-UI Display) showed
a2ui_dynamic, a2ui_fixed, mcp_apps_agent, open_gen_ui_agent,
reasoning_agent, interrupt_agent, and tool_rendering_agent in the
file picker even though none of them are relevant to that demo.
Root cause: `bundle-demo-content.ts` ran `discoverBackendFiles()` once
per package and attached the same union of all agent files to every
demo. This was fine when all demos shared one graph, but since we
split demos into dedicated graphs the bundle stopped matching reality.
Fix:
- `manifest.schema.json`: add optional `backend_files` field per demo
(string array, paths relative to the package root).
- `bundle-demo-content.ts`: when `demo.backend_files` is present, bundle
exactly those. Otherwise fall back to the legacy full-package scan
so packages that haven't adopted the field still work as before.
- `langgraph-python/manifest.yaml`: populate `backend_files` for every
demo. Each demo bundles `src/agent_server.py` plus only the agent
file its graph routes to (main.py for shared-graph demos;
reasoning_agent.py / interrupt_agent.py / a2ui_dynamic.py /
a2ui_fixed.py / mcp_apps_agent.py / open_gen_ui_agent.py /
tool_rendering_agent.py for demos with dedicated graphs).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The LangGraph agent server was started without --host, defaulting to
'localhost' which resolves IPv6-first (::1) on modern Node. The Next.js
frontend's /api/health probe hits 'localhost:8123' which can resolve to
IPv4 127.0.0.1 — unbound — producing a 503 with agent:"error".
Add --host 0.0.0.0 so the agent listens on both IPv4 and IPv6 loopback.
Verified locally: /api/health returns 200 with agent:"ok".
parseManifest (lib/manifest.ts):
- Shape errors for demos[i].route: number / null / object / empty string
- Shape error for route not starting with /demos/
- Happy path persists frozen demo.route on the parsed entry
- Backward-compat: demos[i] without route is accepted with undefined route
validate-parity:
- Negative-case regression: missing-demo-dir's expectedDir is derived from
demo.route (not demo.id) when route is present; mismatched id + route
was silently hiding drift.
- Fallback-case: demo with no route resolves expectedDir from demo.id
- routeToDirName unit tests: undefined / bare /demos/ / normal tail segment
TDD verified: mutating the /demos/ prefix guard failed the relevant test
(RED), restoring the guard passed it (GREEN). Mutating expectedDir to
demo.id-only failed the negative-case test (RED), restoring passed
(GREEN).
demo.id is the CATALOG identifier (matched to qa/spec filenames and
shell registry entries). demo.route is the URL + filesystem path
(/demos/<dir> → src/app/demos/<dir>/). They are deliberately separate
— a manifest with id: hitl-in-chat and route: /demos/hitl lives at
src/app/demos/hitl/.
validate-parity.ts previously resolved the demo directory from
demo.id, producing a spurious missing-demo-dir MUST for every such
split. Fix:
- lib/manifest.ts: add optional route field to ManifestDemo; if present,
parser requires it to be a non-empty string beginning with "/demos/".
- validate-parity.ts: introduce routeToDirName helper (matches
bundle-demo-content.ts idiom); loop over demos resolving expected
dir from route and falling back to id. missing-demo-dir PackageIssue
now carries both demoId and expectedDir so deriveMessage can flag
route-resolved paths distinctly.
- __tests__/validate-parity.test.ts: red-green regression test — a
package with id: hitl-in-chat, route: /demos/hitl, and dir
src/app/demos/hitl/ must PASS (no missing-demo-dir error).
Review pass across all 18 langgraph-python demo columns. For each
demo, verified strict minimality (no cross-contamination from other
features) and extracted any inline view JSX >= 15 lines into dedicated
sibling files. All 18 demos validated end-to-end via Playwright
screenshot + visual review.
View components extracted to dedicated files:
- `a2ui-fixed-schema/flight-card.tsx` + `catalog.ts`
- `agentic-chat-reasoning/reasoning-block.tsx`
- `chat-slots/custom-welcome-screen.tsx`, `custom-assistant-message.tsx`,
`custom-disclaimer.tsx`
- `gen-ui-agent/InlineAgentStateCard.tsx`
- `gen-ui-interrupt/InterruptCard.tsx`
- `headless-complete/message-list.tsx`, `user-bubble.tsx`,
`assistant-bubble.tsx`, `input-bar.tsx`, `typing-indicator.tsx`
- `hitl-in-chat/approval-card.tsx`
- `open-gen-ui/sandbox-functions.ts`, `suggestions.ts`
- `tool-rendering/weather-card.tsx`
Backend fixes:
- `mcp_apps_agent.py`: tighten system prompt to prevent the LLM from
hallucinating unrelated framework names in its reply.
- `open_gen_ui_agent.py`: short-circuit subsequent runs when a
`ToolMessage` is already in state. `generateSandboxedUi` is
registered with `followUp: true` by the provider, which caused the
agent to re-emit the tool call in a loop after each sandbox handler
response.
`agentic-chat/README.md`: removed stale references to tools that no
longer exist in the minimized demo.
Regenerated `shell/src/data/demo-content.json`.
Each demo's page.tsx is now pure composition; presentational code lives
in dedicated sibling files. Zero cross-contamination between demos
(audited via grep for forbidden-feature imports).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Parallel review subagents each need a fresh
showcase-langgraph-python build to screenshot. Running
`dev-local.sh up langgraph-python` N times in parallel serializes on
Docker and wastes time rebuilding identical source.
`scripts/agent-notes/rebuild-coord.sh` aligns rebuild attempts to a
30-second clock tick with 0-5s jitter. First agent on a tick acquires
a `mkdir`-based lock, rebuilds, and writes a per-tick done marker at
`/tmp/showcase-rebuild.done-<tick>`. Peers on the same tick skip their
own build and wait up to 30s for that marker. If no marker appears,
they retry the next tick (up to 3 ticks).
Agents call `bash scripts/agent-notes/rebuild-coord.sh` instead of
`./scripts/dev-local.sh up langgraph-python` when they need the live
container refreshed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Move `agentic-chat-reasoning` from `chat-ui` category to
`generative-ui`, positioned after `tool-rendering`.
- Rename display "Agentic Chat (Reasoning)" -> "Reasoning".
- Update langgraph-python manifest demo entry's name, description,
and tag to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These 7 feature ids had zero manifest references, zero demo dirs, and
zero matching packages backing them in the showcase. They added noise
to the feature matrix without carrying substance:
- mobile-react-native
- mobile-swiftui
- mobile-android
- web-svelte
- web-vue
- web-tanstack — conceptually misplaced anyway; TanStack Start is a
React meta-framework, not a React alternative
- web-angular — `packages/angular/` exists as a published SDK, but the
showcase has zero integration for it. Angular support is really its
own matrix/story; removing this placeholder row until there's a real
showcase integration.
Also drops the `mobile` and `web-frameworks` categories now that they
have no members. Regenerates registry.json + constraints.json.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Generated via `generate-registry.ts` and `bundle-demo-content.ts` from
the new manifest + feature-registry + demo sources.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Minimal demo pages covering the Chat & UI category:
- `agentic-chat/page.tsx` — stripped to pure chat (CopilotKit +
CopilotChat, single "Write a sonnet" suggestion). No tool calls.
- `chat-slots/page.tsx` — three slot overrides: custom welcome screen,
custom disclaimer (input sub-slot), and a custom assistant message
bubble so the slot is visible during active chat.
- `headless-simple/page.tsx` — bare-bones custom chat built on
`useAgent` (no CopilotChat) + a single `useComponent` for inline
card rendering.
- `headless-complete/page.tsx` — fuller custom chat: scrollable
message list, user/assistant bubbles, auto-scroll, disabled state
while running, inline tool-call rendering, cancel button.
- `agentic-chat-reasoning/page.tsx` — custom `messageView.reasoningMessage`
slot renders an amber "Reasoning" block when the agent emits
`role: "reasoning"` messages.
- `prebuilt-chat/page.tsx` — minimal `<CopilotChat />`.
- `prebuilt-sidebar/page.tsx` — dummy main content + `<CopilotSidebar />`
(defaultOpen).
- `prebuilt-popup/page.tsx` — dummy main content + `<CopilotPopup />`
(defaultOpen).
Each outer function wraps provider + layout; inner `Chat` holds only
the registrations for that demo's feature. No cross-contamination
(no a2ui/HITL/interrupt/tool-rendering in demos that don't demonstrate
them).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- manifest.yaml: add 12 new demo entries (chat-slots, headless-simple,
headless-complete, agentic-chat-reasoning, hitl-in-chat,
gen-ui-interrupt, declarative-gen-ui, a2ui-fixed-schema, mcp-apps,
open-gen-ui, prebuilt-chat/sidebar/popup).
- langgraph.json: register dedicated graphs for demos that need a
non-default backend (tool_rendering, interrupt_agent, reasoning_agent,
a2ui_dynamic, a2ui_fixed, mcp_apps, open_gen_ui).
- route.ts: register each new agent name; demos with dedicated graphs
are mapped explicitly outside the shared-graph loop. A2UI runtime
middleware scoped to `declarative-gen-ui` + `a2ui-fixed-schema` so
non-A2UI demos keep their `useComponent` renderers intact. MCP Apps
middleware wraps only the `mcp-apps` agent and synthesizes an
`ACTIVITY_SNAPSHOT` event when the backend `show_mcp_app` tool fires.
- New `/api/copilotkit-ogui/route.ts`: dedicated isolated runtime for
the `open-gen-ui` demo. `openGenerativeUI: {...}` advertises the
`openGenerativeUIEnabled` flag globally on the probe response, which
causes the CopilotKit provider's setTools effect to wipe per-demo
`useFrontendTool`/`useComponent` registrations; isolating to its own
endpoint keeps the default runtime clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds rows for the demos being built in this branch:
- Split `chat-prebuilt` into `prebuilt-chat`, `prebuilt-sidebar`,
`prebuilt-popup` (CopilotChat / CopilotSidebar / CopilotPopup).
- Reorder `generative-ui` rows and add: `hitl-in-chat` (In-Chat HITL),
`gen-ui-interrupt`, `declarative-gen-ui` (Dynamic Schema),
`a2ui-fixed-schema` (new), `mcp-apps`, `open-gen-ui`, `gen-ui-agent`,
`tool-rendering`.
- `constraints.yaml`: expand `constrained-explicit` allowlist so the
langgraph-python manifest validates against the new feature ids.
- Bump expected langgraph-python feature/demo count in
`generate-registry.test.ts` from 10 -> 22 to match the manifest.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Run 24602985507 (singleFork: true) was strictly worse than the prior
fork-per-file run (24602657301):
singleFork (24602985507): 1/14 files completed, 64.64s, then timeout
fork-per-file (24602657301): 14/14 files passed, 169.56s, then teardown timeout
The RPC timeout under Node 20 is a hardcoded 60 s DEFAULT_TIMEOUT in
birpc (DEFAULT_TIMEOUT = 6e4 in vitest's index.B521nVV-.js, not tunable
via config). validate-pins.test.ts alone takes ~60 s because of its
134 `npx tsx` subprocess invocations, so singleFork blows its RPC
budget before a single file finishes. Fork-per-file gives each file
its own fresh 60 s RPC channel, which is why the prior run got all
14/14 files to complete before the final teardown hiccup.
Revert `poolOptions.forks.singleFork: true`. Keep the teardownTimeout
/ hookTimeout bumps to 30 s — they don't affect the RPC timeout but
do protect against slow per-hook teardowns under the same load. Note
in comments that the RPC timeout is upstream-hardcoded and tracked at
https://github.com/vitest-dev/vitest/issues/6129.
`sorts integrations by sort_order` read registry.json without invoking
the generator, making it dependent on whichever run last left disk
state in place. afterEach(dataRestorer.restore()) restores the data
files to HEAD between tests, so test 2 was reading the committed
baseline registry — not the live generator output — and would have
silently agreed with whatever state happened to be on main.
Add a `runGenerator()` call at the top of test 2 mirroring the
`runBundlerAndRead()` pattern that tests 2-5 of bundle-demo-content
already use. Test now exercises the actual generator's sort
behavior under live conditions.
The PR description advertised a drifted-baseline guard on CI for
restoreFromGitHead, but the implementation never actually ran a
post-heal `git diff --quiet HEAD -- <tracked>` check — the only
diff was the off-CI pre-checkout guard against clobbering developer
edits. A racing external mutator (or a parallel test suite we haven't
accounted for) could rewrite a tracked file between our `git checkout
HEAD --` and our subsequent snapshot, and we'd silently bake the drift
into the baseline and the afterEach restore loop would maintain it
forever.
This commit adds the missing post-heal check immediately after the
`git checkout HEAD --`. On CI it throws with `drifted-baseline guard:
post-heal diff failed` citing every offending path; off-CI it warns so
developers iterating on a dirty tree aren't blocked.
Also:
- Rephrases the `isBenignPathspec` catch comment to describe the
realistic case (belt-and-braces against a rm race, not the normal
flow) now that partitionTrackedPaths pre-filters untracked paths.
The branch is intentionally kept — cheap to tolerate, and it
guards against a race that's hard to rule out in a shared worktree.
- Red-green unit coverage: new tests use a counter-based git shim
that fails the N-th `diff --quiet` invocation, so we can target
the post-heal diff independently of the off-CI pre-checkout diff.
- CI throws on drift with the advertised message
- off-CI warns (does not throw) with the same message
- no false-positive on a clean tracked path
Verified red without the guard, green with it.
Run 24602657301 showed unit (20.x) still emitting
`Timeout calling "onTaskUpdate"` during @copilotkit/showcase-scripts:test
teardown — 1008/1008 tests passing, then ELIFECYCLE. The previous fix
switched the pool from threads to forks but left the default fork-per-file
behavior, so every file teardown tore down and reestablished the
parent↔child RPC channel. Under Node 20 the channel occasionally fails
to reestablish within the 10s default teardownTimeout — a known vitest
issue (https://github.com/vitest-dev/vitest/issues/6129).
Two defenses:
- poolOptions.forks.singleFork: true — one long-lived fork across all
test files (combined with the existing fileParallelism: false, files
still run sequentially). The RPC channel stays warm for the whole
run instead of being torn down and reestablished between every file,
which eliminates the per-file teardown as a surface for the race.
- teardownTimeout / hookTimeout bumped from the 10s default to 30s,
matching testTimeout. A slow teardown can't be the bottleneck that
kills the run.
This is the most conservative setting short of dropping to a single-
thread pool entirely. Node 22 / 24 unaffected.
Three small CR8 fixes bundled by area (crewai-crews Python):
* aimock_toggle.py: prod-guard WARNING now uses a grep-friendly prefix
`aimock_toggle: DISABLED (prod-guard: VAR=value)`. Rationale in-file:
`ENV=production` is a common shell-profile default that can silently
disable the toggle in dev/CI without the operator realizing; a
consistent greppable prefix lets them find unexpected refusals without
reading every WARNING line. Strengthened the refusal test to assert
exactly one WARNING is emitted with that prefix (regression guard
against future iteration of _PROD_ENV_VARS firing past the first match).
* agent_server.py: added a `CORSMiddleware` comment documenting why
`allow_origins=["*"]` is intentional for this LOCAL DEMO STARTER — the
agent binds to localhost:8000 (or :8123 inside a generated starter
container) and is only reached by the Next.js frontend during
development. Pointed at `.env.example` / a future `CORS_ORIGIN` env var
as the path for real deployments. No behavior change.
* test_aimock_toggle.py: `test_default_arg_path_mutates_os_environ` now
`monkeypatch.delenv`s every entry in `_PROD_ENV_VARS` before asserting
`enabled is True`. A CI runner or dev shell that exports e.g.
`ENV=production` or `RAILWAY_ENVIRONMENT=production` would otherwise
silently flip this test from "toggle enabled" to "prod-guard refused"
and the enabled-True assertion would fail for the wrong reason.
monkeypatch restores on teardown so other tests are unaffected.
Regenerated crewai-crews starter to mirror the package changes.
The R6 "scope NODE_ENV to Next.js invocation only" fix landed for the
Python Dockerfile template but missed the .NET / Java / TypeScript
templates — they still set `ENV NODE_ENV=production` at the image level,
which leaks into every child process. That's surprising for the TS
templates specifically (mastra dev, langgraph-cli dev, tsx) because those
frameworks gate HMR / source maps / telemetry on `NODE_ENV` and should
NOT see "production" when they're running as the agent backend behind
Next.js.
Fix: remove image-level `ENV NODE_ENV=production` from all three non-Python
Dockerfile templates. The entrypoint template already binds
`env NODE_ENV=production npx next start` so Next.js still sees the value —
only the child agent processes get their environment back. Added a lang-
specific comment to each template explaining the intent and calling out
the affected child processes.
Regenerated the 5 affected starter Dockerfiles
(claude-sdk-typescript, langgraph-typescript, mastra, ms-agent-dotnet,
spring-ai) via `tsx generate-starters.ts`; diffs are byte-identical to the
template change.
Propagates the crewai-crews aimock_toggle.py (adds RAILWAY_ENVIRONMENT_NAME
to `_PROD_ENV_VARS` + updated header comment) and .env.example (full
prod-guard var enumeration) from packages/ to starters/ via
generate-starters.ts. Only crewai-crews ships aimock_toggle.py today so
only that starter changes; other starters re-generate byte-identical.
Verified: `tsx generate-starters.ts --check` OK across all 17 starters.
- Add RAILWAY_ENVIRONMENT_NAME to `_PROD_ENV_VARS`. Railway exposes BOTH
the legacy `RAILWAY_ENVIRONMENT` and the newer canonical
`RAILWAY_ENVIRONMENT_NAME`; both can hold "production" on a prod service,
so both must be guarded. Missing this one was a production-aimock-leak
footgun for anyone hosting on newer Railway service templates.
- Update the parametrized prod-guard tests (both "refuses" + "silent
no-op" variants) to cover RAILWAY_ENVIRONMENT_NAME so new vars added to
`_PROD_ENV_VARS` can't drift from the test matrix.
- Sync .env.example comment block to enumerate ALL 8 prod-guard vars
(NODE_ENV, ENV, ENVIRONMENT, APP_ENV, DEPLOY_ENV, PYTHON_ENV,
RAILWAY_ENVIRONMENT, RAILWAY_ENVIRONMENT_NAME). The old comment said
"NODE_ENV or ENV" which was misleading — operators reading the .env
scaffold had no way to discover the full guard surface.
- Update outdated "Local dev on macOS ships Python 3.9" header comment
in aimock_toggle.py. CI matrix is 3.10/3.12 and requirements.txt
enforces 3.10+ via marker-gated typing_extensions.
Tests: 94 pass (was 88, +6 from RAILWAY_ENVIRONMENT_NAME parametrize entries
across both refuse + silent-no-op variants).
- Rewrite restoreFromGitHead JSDoc to match actual behavior for tracked
vs untracked paths (including the on-CI throw / off-CI warn for an
entirely-untracked input list).
- Rephrase internal review-round markers ("CR4/CR5 HIGH/MEDIUM") in
test-cleanup.ts and test-cleanup.test.ts to describe the behavior or
invariant being guarded instead of the review history.
- Point the WINDOWS comment at the sibling test files that actually
invoke npx (this module itself doesn't).
- Drop the "belt-and-braces" afterAll disclaimer from
bundle-demo-content.test.ts and generate-registry.test.ts — the
afterAll(restore) pattern is self-explanatory.
unit (20.x) CI was failing with 'Timeout calling onTaskUpdate' during
the showcase-scripts test run — an unhandled vitest error that causes
ELIFECYCLE after otherwise-green tests. Reproducible on Node 20 only;
22.x and 24.x are green on the same code.
Root cause: vitest's default thread-based worker pool times out on the
parent-worker RPC channel when a test file spawns many subprocesses
(validate-pins.test.ts runs 134 tests each invoking a subprocess;
create-integration / generate-registry / bundle-demo-content each
spawn npx tsx via execFileSync). Under Node 20 the stdio / signal
traffic from these children contends with the worker-thread RPC
channel and surfaces as an unhandled timeout mid-suite.
Switch to pool: 'forks' — the fork pool uses node IPC for the RPC
rather than worker-thread messageports and is robust under the same
load. fileParallelism: false keeps files sequential so shared-env /
tmp-dir mutations don't race, but each file now gets its own fresh
fork so one file's subprocess churn can't stall the RPC for
subsequent files. Node 22/24 unaffected either way.
Also pipes stdio explicitly on every git subprocess in
test-cleanup.test.ts — inherited stdio on a fork vitest worker
interleaves with the worker's own stdout/stderr and is another input
to the RPC contention under Node 20.
generate-registry.ts writes to shell/src/data/registry.json and
shell/src/data/constraints.json. bundle-demo-content.ts writes to
shell/src/data/demo-content.json. All three files are tracked, and
without restoration every run leaks regenerated JSON into the working
tree.
This commit:
- replaces execSync-with-path-interpolation invocations with
execFileSync argv form via runGenerator() / runBundler() helpers;
eliminates any shell-parser involvement (clean hygiene even when
the interpolated constant happens to be safe today).
- snapshots the three data files in beforeAll, restores them in
afterEach + afterAll, adds a regression-guard test that proves the
hooks actually heal drift (sentinel append + bit-exact in-memory
comparison), and a terminal safety-net bit-for-bit check.
- drops a redundant bundler pre-run in beforeAll (test 1 exercises
the bundler itself), and replaces byte-length sentinel checks with
content-level Buffer.concat assertions so the regression guard
survives any hypothetical fs shim that updates stat but not bytes.
create-integration generates an integration package in
showcase/test-integration-tmp and mutates existing workflow YAMLs in
.github/workflows/ (showcase_deploy.yml, showcase_drift-detection.yml,
starter-smoke.yml) to register the new slug. Without restoration, both
the generated tmp package and the workflow-YAML drift leak into the
working tree on every run, and the workflow-YAML drift in particular
breaks pnpm run check on subsequent test invocations.
This commit wires FileSnapshotRestorer + restoreFromGitHead into the
test so:
- tracked workflow files are restored to HEAD before each test run
- the tmp output dir is deleted eagerly in afterEach (independent of
the generator — no reliance on its internal cleanup)
- a regression-guard test proves the snapshot/restore hooks actually
heal drift via a sentinel append + bit-exact assertion against
the in-memory snapshot (not a re-read of disk, which would
silently agree with a buggy restore()).
- a terminal safety net re-asserts every snapshotted file is
byte-identical to its baseline at the end of the suite.
Introduces FileSnapshotRestorer + restoreFromGitHead helpers used by the
showcase test suites to snapshot tracked files in beforeAll and restore
them in afterEach / afterAll. Several of our test scripts invoke real
generators (create-integration, generate-registry, bundle-demo-content)
that write to tracked files outside any tmp dir: .github/workflows/ and
showcase/shell/src/data/*.json. Without explicit restoration these writes
leak into the working tree on every nx run-many -t test and, under Node
20 + vitest worker pools, the accumulated drift races the worker-RPC
channel surfacing as 'Timeout calling onTaskUpdate' -> ELIFECYCLE on CI.
Highlights:
- FileSnapshotRestorer captures bytes at snapshot time, rewrites only
drifted files via atomic temp+rename, and sweeps leftover
.<basename>.<hex>.tmp stragglers scoped to the snapshotted basenames
(no more whole-directory unlink).
- restoreFromGitHead uses execFileSync with a frozen env (GIT_*
scrubbed, PATH/HOME preserved) to heal a working tree left dirty by
a crashed prior run before we snapshot.
- On CI, a baseline that drifts after the pre-snapshot heal is a hard
error (git binary missing, sandbox, etc.); off-CI it warns instead
of blocking local iteration.
- Narrow catch in the git path partitioner: only genuine 'not in
index' pathspec errors are treated as untracked; ENOENT / EACCES /
non-exit-1 failures re-raise so sandbox and missing-binary cases
don't get silently swallowed and lock in a drifted baseline.
- test-cleanup.test.ts itself strips GIT_* from child env when it
creates tmp repos — pre-commit hooks (lefthook) run with GIT_DIR /
GIT_INDEX_FILE set, which would cause tmp-repo 'git commit' calls
to ignore cwd and write to the HOST working-tree HEAD. Without the
scrub, running 'git commit' itself silently accumulates 'initial' /
'init' commits on the real repo.
Also pulls scripts-dir / repo-root / data-dir constants into a shared
paths.ts so future layout changes flip in one place.
Downstream of the template refactor: Dockerfile + entrypoint.sh across
all 17 starters regenerated to pick up the generic NODE_ENV comment,
the `env NODE_ENV=production` invocation form, and the langgraph-only
gating of `/app/.langgraph_api`. Non-langgraph starters lose the
spurious mkdir layer they never used.
- Reword the NODE_ENV=production comment in Dockerfile + entrypoint
templates to generic language. The previous comment mentioned
`aimock_toggle.py refuses to apply` — that text got rendered
verbatim into .NET / TypeScript / Java starters via the shared
template, misleading readers who don't have a Python toggle. New
comment describes the general behavior (image-level NODE_ENV leaks
into every child process, most of which don't interpret it like
Next.js does).
- Use `env NODE_ENV=production npx next start` instead of bare
`NODE_ENV=production npx next start` in entrypoint.sh. `env` prefix
is the syntactically robust form across shells.
- Gate the `RUN mkdir -p /app/.langgraph_api` layer to langgraph-*
starters only via a new `{{LANGGRAPH_MKDIR}}` template variable.
Creating that directory in crewai / agno / pydantic-ai / claude-sdk
/ etc. was copy-paste residue — langgraph_cli is the only thing
that ever writes there.
- Apply the same NODE_ENV rewording to the crewai-crews SOURCE
Dockerfile + entrypoint (the templates' downstream consumers
regenerate via generate-starters, but the source package is hand-
maintained).
- Extend `_PROD_ENV_VARS` from {NODE_ENV, ENV} to also include
ENVIRONMENT, APP_ENV, DEPLOY_ENV, PYTHON_ENV, RAILWAY_ENVIRONMENT.
Railway is this project's primary hosting target (per CLAUDE.md:
"Always Railway for hosting") so its env var was a prominent gap.
The other additions cover common PaaS / framework conventions —
keep the guard broad (cheap) vs narrow (production-aimock-leak
footgun if someone deploys via a convention we don't recognize).
- Parametrize the prod-guard tests symmetrically across all prod vars.
Both the "refuses with warning" and "silent when URL unset" contracts
now run on NODE_ENV / ENV / ENVIRONMENT / APP_ENV / DEPLOY_ENV /
PYTHON_ENV / RAILWAY_ENVIRONMENT so a future var add that forgets
the silent-path fix would fail here.
- Collapse `_classify`'s "empty" + "whitespace" buckets into a single
"blank" label. The two never produced different behavior at any
use site; keeping them distinct was noise in the state machine.
Diagnostic strings still use "empty or whitespace-only" for the
operator-facing message (the input distinction doesn't matter; the
fix-your-env-value message is identical for both).
Test run: 88 passed (was 78 before; +10 from new prod-var parametrize).
Regenerate all 17 starters via `tsx generate-starters.ts` to propagate:
- aimock_toggle.py: prod-guard only fires when AIMOCK_URL is set;
dummy key renamed to `sk-aimock-dev-ci-only` (from previous file).
- agent_server.py: removed dead `main()`/RELOAD entrypoint (non-langgraph
Python starters).
- Dockerfile + entrypoint.sh: `NODE_ENV=production` moved from global
image ENV onto the `npx next start` invocation so the Python agent
doesn't inherit the prod-guard that would silently disable aimock.
- langgraph-fastapi/.env.example: source-package port fix already propagated
(port was already correct at :8123 for this starter; no content change
beyond reflowing the comment from the source package).
The starter regen is a single atomic commit (separate from the source
edits) so reviewers can inspect which input edit produced which starter
output without wading through unrelated edits in the same diff.
LOW CR5-2 L4: Drop the `test:python` npm script. CI runs pytest directly
(.github/workflows/showcase_validate.yml invokes `python -m pytest
tests/python/ -v` in each package dir) and the script isn't wired into
any Nx target — it's an orphan knob that would only cause drift between
"what CI runs" and "what a contributor gets from `npm test:python`".
LOW CR5-6 L1 (docs): Document the AIMOCK_URL + NODE_ENV=production
interaction in .env.example so operators know the prod-guard refuses to
apply the toggle even when AIMOCK_URL is set, and understand why.
M2: The imported `AGENT_URL_LOCALHOST_8000_RE` carries the /g flag, so
`.test()` / `.exec()` advance lastIndex — sharing the exact instance
across the describe-loop iterations coupled any two iterations that
happened to touch it. Clone the regex per-call via a `re8000()` factory
that returns `new RegExp(source, flags)` each time. Same treatment for
the companion `re8123()`.
LOW: Replace the conditional `it()` registration (only created when the
package .env.example contained `:8000`) with an UNCONDITIONAL `it()`
that internally early-returns when there's nothing to rewrite. Vitest's
reporter registers the test either way so a future regression where every
package suddenly stopped matching the pattern would surface as "test
skipped" rather than silently vanishing from CI output.
M9: `ENV NODE_ENV=production` at the Dockerfile level propagates to every
process spawned by the entrypoint — including the Python agent. The
aimock toggle (aimock_toggle.py) refuses to apply when `NODE_ENV=production`
is set, so a Docker-based smoke test with `AIMOCK_URL` exported would
silently get a no-op toggle and hit real OpenAI instead of aimock.
Move `NODE_ENV=production` out of the global Dockerfile ENV blocks and
inline it on the `npx next start` invocation in entrypoint.sh. Next.js
still treats the server as production (that's all Next looks at), but the
Python sibling process is no longer subject to a production-guard that
only ever made sense for the JS runtime.
Affected: crewai-crews package Dockerfile + entrypoint.sh, plus the
starter template (Dockerfile.python + entrypoint.template.sh) — the
starter regen in a follow-up commit propagates this to all 17 starters.
M7: Every execution path (package `pnpm dev`, starter `pnpm dev`, Docker
entrypoint, CI workflow) invokes `python -m uvicorn agent_server:app ...`
directly via CLI. The `main()` wrapper reading PORT/RELOAD from env was
dead code that:
- was never exercised by any CI path;
- defaulted PORT=8000 while starter dev scripts bound :8123 (drift);
- carried a RELOAD gate whose only purpose was to guard `reload=True`
in a function that never ran.
Remove `main()` and the `if __name__ == "__main__"` dispatch entirely, plus
the now-unused `os` import. A comment at the bottom of the module documents
why no CLI entrypoint lives here so a future contributor doesn't add one
"for consistency" with other showcase packages.
M6: The prod-guard (`NODE_ENV=production` / `ENV=production` refuses to apply
the toggle) was entering its refusal branch even when AIMOCK_URL was unset
— returning a misleading `"aimock toggle refused"` reason when in fact
nothing was being applied. Move the `aimock_url` non-empty check BEFORE the
prod-guard block; the unset-URL path now falls through to the normal "not
configured" diagnostic, reserving the refusal message for actual refusals.
Regression pinned: `test_production_env_without_aimock_url_is_silent_no_op`
now asserts (a) no WARNING is logged, (b) reason text does not contain
"refused", and (c) neither OPENAI_BASE_URL nor LITELLM_API_BASE is mutated.
LOW: Rename `_AIMOCK_DUMMY_KEY` from `"sk-aimock-DO-NOT-SET-AIMOCK-URL-IN-PROD"`
to `"sk-aimock-dev-ci-only"`. The old name embedded a directive aimed at
operators but the value is never user-provided — the toggle itself injects
it when AIMOCK_URL is set and the prod-guard refuses production anyway.
Short, honest, still satisfies SDK versions that validate `sk-` prefix.
Test `test_dummy_key_does_not_mislead_with_replace_in_prod_suffix` extended
to also reject the old "DO-NOT-SET" directive.
LOW: Pin `LITELLM_API_BASE not in env` in
`test_production_env_refuses_toggle_even_with_aimock_url` so a future
regression that mutates LITELLM_API_BASE in the refusal branch (symmetric
with the existing OPENAI_BASE_URL check) fails the test.
The package's own `pnpm dev` script binds langgraph_cli on port 8123
(via package.json "dev"), but the .env.example documented port 8000.
A contributor running `cp .env.example .env && pnpm dev` would hit
ECONNREFUSED out of the box because nothing binds :8000 in this package.
Generated starters (showcase/starters/langgraph-fastapi/.env.example)
already point at :8123; this aligns the package-side .env.example with
both its own dev script and the scaffolded starter output.
github-actions[bot]