Three interlocking fixes to generate-starters.ts that together restore
langgraph-fastapi and langgraph-typescript /api/health to "ok" after
#4099 shipped an entrypointOverride feature that accidentally reverted
earlier fixes and left gaps for the TS starter.
1. Resolution-aware python import rewrite for langgraph starters.
langgraph_cli loads agent modules standalone (not as packages), so the
generator rewrites `from .X import ...` to absolute form. The previous
flat rewrite produced `from <agentDir>.X import ...`, and the
subsequent subdir-aware variant used the file's OWN dir as the
package. Both assume sibling targets sit alongside the importing file.
langgraph-fastapi violates that: agent.py lives at
`src/agents/src/agent.py` but the `tools/` it imports lives at
`src/agents/tools/` -- one level up, not inside `src/`. Either prior
rewrite produced `from src.agents.src.tools import ...`, a path that
doesn't exist, and langgraph_cli crashed on module import with
`ModuleNotFoundError: No module named 'src.agents.src.tools'` before
ever binding 8123.
Walk UP from the file's own dir toward agentDest and rebase the
absolute import on the shallowest directory that actually contains
`<firstSeg>/` or `<firstSeg>.py`. Correct both for co-located imports
(`tools/get_weather.py` importing `.types`) and for sibling-directory
imports (`src/agent.py` importing `.tools` from `../tools`).
2. Merge langgraph-typescript agent runtime deps into root package.json.
The TS Dockerfile deliberately deletes `agent/package.json` to
collapse the ESM package boundary between the Next.js frontend and
the agent subtree. Without merging the agent's runtime deps up into
the root package.json, the langgraph-cli's runtime import of
graph.ts fails with `Cannot find module '@langchain/openai'` and
the agent never binds 8123.
Add `extraDependencies` to the langgraph-typescript framework def
with @langchain/core, @langchain/langgraph,
@langchain/langgraph-checkpoint, @langchain/langgraph-cli,
@langchain/openai, and @copilotkit/sdk-js.
3. Restore AGENT_LOG_PREFIX process-substitution helper.
#4099 inadvertently replaced `cmd &> >(awk … fflush …)` with
`cmd 2>&1 | sed …` across every getEntrypointBlock branch. After a
pipeline `$!` points at `sed`, not the agent, so `kill -0 $AGENT_PID`
and `wait -n $AGENT_PID` monitor the wrong process and mask real
crashes; `sed` also line-buffers so crash output can be lost.
Restore the helper so the committed starter entrypoint.sh files and
regenerated output stay consistent.
generate-starters.ts:
- Add entrypointOverride?: boolean to FrameworkDef so slugs whose boot sequence
diverges from the shared template (e.g. langroid's provider-aware credential
guard) can preserve their committed entrypoint.sh through regeneration.
- Snapshot the override from STARTERS_DIR/<slug>/entrypoint.sh (canonical
committed path), NOT outDir — so --check mode (which passes a temp outDir)
doesn't false-flag the override as drift.
- Force mode: 0o755 on the restored file regardless of source mode so editors
that strip the executable bit can't silently ship a broken starter.
- Throw (not warn) when an entrypointOverride slug is missing the override
file, when declared extraFiles / agent_server.py / Dockerfile are missing,
and when PIN_OVERRIDES references a stale dep. These are repo-integrity
failures — silent-warn-and-continue ships broken starters.
- Strip unused `import sys` after sys.path.insert removal (pair with the
existing osUsed detection).
Opt langroid in via entrypointOverride: true.
Also adds a regression test in __tests__/generate-starters.test.ts that
calls generateStarterToDir against a tmp dir and asserts the emitted
entrypoint.sh byte-equals the committed langroid starter entrypoint,
plus a negative test that a non-override slug does NOT preserve its file.
Port the 4084 simplification of the status generator: remove the
MOCK_VARIANTS seeding table + mockVariantTest / mockVariantQA /
mockVariantHealth helpers + Variant type. Informational demos (no
route) are skipped from health probing. The status JSON no longer
carries per-demo `variants[]` — the shell-internal dashboard renders
one row per demo without variant expansion.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Port the 4084 scripts-layer enhancements so 4085's showcase toolchain
matches the new feature shape:
- lib/manifest.ts: ManifestDemo gains optional `command` field; parser
accepts + validates it (non-empty string, frozen).
- bundle-demo-content.ts: inline `@region[name]` / `@endregion[name]`
comment-marker extraction; informational-only demos (no route, e.g.
cli-start) are skipped; markers stripped from bundled content;
regions: { file, startLine, endLine, code, language } emitted per
demo. External-highlight-file merging (4085-specific) preserved, so
backend agents under src/agents/*.py still flow into the bundle.
- validate-parity.ts: accepts demos at BOTH demos/<cell>/ (4084 layout)
and src/app/demos/<cell>/ (4085 layout); informational demos
(command field) are excluded from the parity audit.
- tests: bundle-demo-content.test.ts expectedDemos updated for the
shared-state rename; generate-registry.test.ts feature count 25→32;
validate-parity.test.ts missing-demo-dir message updated to match
the new dual-location wording.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Regenerate agno starter to pick up agno>=2.5.17 (from #4095)
- Ratchet validate-pins fail-baseline hash to match new FAIL set
(count unchanged at 110; hash rotates because agno Dojo/showcase
pair now reflects the SDK upgrade)
generate-starters.ts rewrites relative imports to absolute for langgraph
starters because langgraph_cli loads modules standalone rather than as
packages. The rewrite was flat:
from .X import ... -> from <agentDir>.X import ...
For a file at <agentDir>/tools/get_weather.py, `from .types import ...`
resolves to `tools.types` — the CURRENT package — not `<agentDir>.types`.
The flat rewrite dropped the `tools` segment and produced:
ModuleNotFoundError: No module named 'src.agents.types'
at startup. The agent crashed during module import; the entrypoint
pipe swallowed the traceback (see previous commit for the pipe bug);
the 2-3s sleep guard happened to fire while `sed` was still alive; and
Railway's /api/health probe reported `agent: "error"`.
Make the rewrite subdir-aware: compute the file's containing Python
package from its relative path under agentDest, and prepend that to the
relative import target. So `from .types import ...` inside
`src/agents/tools/get_weather.py` becomes
`from src.agents.tools.types import ...`.
langgraph-python has the same broken imports in its tools/__init__.py
but doesn't crash at runtime because main.py doesn't import from tools
(dead path). Regenerating fixes the dead code too.
The entrypoint.sh template used `cmd 2>&1 | sed 's/^/[agent] /' &`
followed by `AGENT_PID=\$!`. After a pipeline, `\$!` points to the LAST
command in the pipe (the `sed` process), not the agent. Every subsequent
`kill -0 \$AGENT_PID` and `wait -n \$AGENT_PID` was therefore monitoring
`sed`, which stays alive until its stdin closes — long after the agent has
crashed. Railway restarts the container mid-loop; the health probe sees
`{status: "degraded", agent: "down"}` for a few seconds during each cycle.
A second, compounding bug: `sed` buffers by default, and Python agents
buffer their own stdout, so a stack trace emitted during module import
could sit in userspace memory until the pipe closed — by which point the
log was discarded and the real cause of the crash was lost.
Fix both by switching to bash process substitution:
cmd &> >(awk '{print "[agent] " \$0; fflush()}') &
AGENT_PID=\$!
Process substitution does not create a pipeline, so `\$!` remains the
agent's PID. `awk` with `fflush()` flushes each prefixed line to the
container log immediately. Also export `PYTHONUNBUFFERED=1` at the
entrypoint level so Python-based agents don't buffer before awk.
Applies to all 17 starters (python, langgraph-python, langgraph-fastapi,
langgraph-typescript, mastra, typescript, java/spring-ai, csharp/
ms-agent-dotnet). Done once in generate-starters.ts + the template +
regenerated entrypoint.sh files.
## Summary
Replaces `RUN chown -R app:app /app` with `COPY --chown=app:app` across
all 17 starter Dockerfiles + 4 shared templates.
Every starter ended with a recursive chown over `/app`, which walks ~50k
files (Next.js `node_modules` dominates) to fix ownership after the
fact. Under the 23-way Depot runner fan-out used by
`.github/workflows/deploy-showcase-services.yml`, that step consistently
ran 5+ minutes under I/O contention — busting the **15-minute GH Actions
job budget** before images could finish pushing.
Failed run this fixes:
https://github.com/CopilotKit/CopilotKit/actions/runs/24621469277 (22/23
showcase services cancelled mid-push).
## What changed
- Create the `app` user right after `WORKDIR /app` in every runner stage
so `--chown=app:app` resolves by name.
- Add `--chown=app:app` to every runner-stage COPY (including
multi-stage `COPY --from=frontend` and `COPY --from=<agent-builder>`).
- Drop the trailing `RUN chown -R app:app /app` (or the tail of the
compound RUN in the TS starters).
- For langgraph starters, fold `chown app:app /app/.langgraph_api` into
the same RUN as the `mkdir`, so `langgraph_cli`'s scratch dir remains
writable by the runtime user.
- Update `showcase/scripts/generate-starters.ts` so the shared templates
(`Dockerfile.python/typescript/dotnet/java`) emit the new pattern.
Starters are regenerated from templates.
## Local timing (Docker Desktop, single starter, no contention)
| Starter | Pre-fix | Post-fix | `chown -R` step |
| --- | --- | --- | --- |
| ag2 (Python) | 3:03 | 1:39 | 50.1s → removed |
| langgraph-fastapi | n/a | 1:41 | replaced with 0.2s targeted chown on
`.langgraph_api` |
| mastra (TS) | n/a | 2:16 | removed |
The real-world win on the 23-way Depot fan-out is substantially larger
than the local 50s baseline — recursive chown degrades super-linearly
with concurrent I/O pressure, which is exactly what the 5+ min Depot
step demonstrated.
## Smoke tests
- ag2 image: container starts clean as `app` user, all files under
`/app` owned by `app:app`, `/api/health` returns 200.
- langgraph-fastapi image: `/app/.langgraph_api` exists and is owned by
`app:app`.
- mastra image: builds cleanly, ownership correct.
## Test plan
- [ ] CI green (existing showcase starter smoke suite covers startup)
- [ ] Watch the next showcase deploy workflow run — expect jobs to
finish well under 15m
## Not touched
No workflow files, no `examples/` Dockerfiles (none matched the problem
pattern), no `chmod -R` offenders (none found).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Two related defects in the showcase deploy pipeline let stale images sit
live on Railway while Slack stayed green. This PR fixes both.
### Defect 1 — Drift detector skipped all starter services
`.github/workflows/showcase_smoke-monitor.yml` listed only 19
**package** slugs in its `SERVICES=(...)` array (ag2, mastra,
llamaindex, ...). Zero **starter** slugs. As a result:
- GHCR `showcase-starter-<svc>` tags were never checked for drift.
- `gh workflow run showcase_deploy.yml -f service=starter-*` was never
auto-dispatched.
- Starter services could run with weeks-old images and no alert would
fire.
`showcase_deploy.yml` already supports `starter-*` dispatch names and
already calls `serviceInstanceRedeploy` for any service with a
`railway_id`, so no change is required there. The fix is extending
`SERVICES=(...)` to include all 17 starter slugs via a
sparse-checkout-driven filesystem enumeration (no more literal
duplication between workflow and `showcase/starters/`).
### Defect 2 — Silent deploy failures reported green
`.github/workflows/showcase_deploy.yml` emitted `::warning::` and exited
0 when a service never returned 200 on its health path within 360s. The
legacy justification (`# Don't fail — sleep-on-idle services take time
to wake`) no longer applies: Railway is on the Pro tier with no
sleep-on-idle, so a 6-minute failure to become healthy is a real
failure. Changed to `::error::` + `exit 1`.
## Round 2 fixes
Round 2 CR raised six findings against the original smoke-monitor +
validator changes. All fixed in this PR:
- **BLOCKING 1/2 — smoke-monitor guard.** Replaced the magic `-eq 19`
sentinel with `grep -c '^starter-'` so adds/removes to the literal
non-starter list can't silently disable the guard. Added `shopt -s
nullglob` around the `showcase/starters/*/` loop so an empty starters
tree no longer corrupts `SERVICES` with a `starter-*` literal.
- **BLOCKING 3 — GHCR stderr isolation.** Dropped `2>&1` on the `gh api
-i` call; captured stderr to a temp file and surfaced it only when `gh`
returns a non-zero RC with no HTTP status. Auth / rate-limit / network
noise can no longer splice into the HTTP header block and poison
`HTTP_STATUS` / `API_BODY` parsing.
- **BLOCKING 4 — validator tests.** Added
`showcase/scripts/__tests__/validate-workflow-starters.test.ts` (12
specs): happy path, missing-from-options-only, missing-from-matrix-only,
missing-from-both, empty starters dir (exit 3), template/ excluded,
substring-spoof (starter-ag2 vs starter-ag2-extended), missing workflow
file (exit 3). Also extended `VALIDATE_WORKFLOW_STARTERS_REPO_ROOT` to
re-home the starters dir for testability.
- **MEDIUM 1 — YAML parsing.** Replaced the fragile regex-over-YAML
options scanner with a real `yaml.parse()` + typed navigation down
`on.workflow_dispatch.inputs.service.options`. ALL_SERVICES stays
regex-scanned (embedded JSON in a bash heredoc, with `${{ ... }}`
interpolations that aren't valid JSON pre-execution), but the
surrounding step is now located via YAML.
- **MEDIUM 2 — Slack list truncation.** Replaced `cut -c1-200` with a
`truncate_csv` helper that drops whole comma-separated entries until
under budget and appends `…` when truncated. No more
`starter-claude-sdk-pyth` mid-slug corruption.
- **MEDIUM 3 — template/ exclusion cross-references.** Both the TS
validator's `EXCLUDED_DIRS` and `showcase_smoke-monitor.yml`'s `[
"$slug" = "template" ] && continue` now carry `# keep in sync with ...`
comments pointing at each other.
- **NIT 2 — entry-check simplification.** Dropped the
belt-and-suspenders `import.meta.url === \`file://${argv[1]}\`` branch;
kept only the canonical `fileURLToPath(import.meta.url)` form.
- **NIT 3 — jq pipeline collapse.** Single-pass `.jobs[]? | select |
"\(...)"` replaces the three-pass `map | map | .[]` chain in the notify
step.
## Files changed
- `.github/workflows/showcase_deploy.yml` — warning → error + exit 1 on
unhealthy deploy; `truncate_csv` replaces `cut -c1-200` (3 sites);
single-pass jq pipeline in notify step.
- `.github/workflows/showcase_smoke-monitor.yml` — filesystem-driven
`SERVICES=(...)`, starter-count guard, `nullglob` loop, stderr-isolated
`gh api` call, cross-reference comment.
- `.github/workflows/showcase_validate.yml` — wires
`validate-workflow-starters` into CI.
- `showcase/scripts/validate-workflow-starters.ts` — YAML-aware presence
checks; env-var override homes both starters dir and workflow path.
- `showcase/scripts/tsconfig.json` — scripts-local tsconfig for LSP type
resolution.
- `showcase/scripts/__tests__/validate-workflow-starters.test.ts` — 12
specs covering the full matrix of drift scenarios.
## Test plan
- [ ] Next scheduled `showcase_smoke-monitor` run includes starter
services in its drift scan.
- [ ] A deliberately-unhealthy deploy (simulate by pointing health_path
at a 404) fails the job and fires the Slack alert.
- [ ] `showcase_validate` CI job runs `validate-workflow-starters` and
`npx vitest run scripts/__tests__/validate-workflow-starters.test.ts`
green.
Every starter Dockerfile ended with `RUN chown -R app:app /app`, which
recursively walks the full `/app` tree (Next.js `node_modules` dominates
— ~50k files) to fix ownership after the fact. On the 23-way Depot
runner fan-out used by .github/workflows/deploy-showcase-services.yml
this step consistently took 5+ minutes under I/O contention, busting
the 15-minute GH Actions job budget before the image could even finish
pushing (run 24621469277 — 22/23 services cancelled).
The standard Docker idiom is `COPY --chown=<user>:<group>` which applies
ownership during the copy step itself — no extra layer, no full-tree
traversal, zero runtime cost.
Changes:
- Create the `app` user right after `WORKDIR /app` in every runner
stage so `--chown=app:app` resolves by name.
- Add `--chown=app:app` to every COPY instruction that lands files
under `/app` in the runner stage (including `COPY --from=frontend`
and `COPY --from=<agent-builder>` multi-stage copies).
- Drop the trailing `RUN chown -R app:app /app` (or the tail of the
compound RUN in the TS starters).
- For langgraph starters, fold `chown app:app /app/.langgraph_api`
into the mkdir RUN so langgraph_cli's scratch dir is still writable
by the runtime user.
- Update showcase/scripts/generate-starters.ts so the shared templates
(Dockerfile.python/typescript/dotnet/java) emit the new pattern and
the framework-specific COPY lines (`COPY ${dest} ./` for extra files
and `COPY agent_server.py ./` for Python starters) include `--chown`.
Local verification (Docker Desktop, single starter, no contention):
ag2 pre-fix: 3:03 total, `RUN chown -R app:app /app` = 50.1s
ag2 post-fix: 1:39 total, no chown step
langgraph-fastapi post-fix: 1:41 (langgraph_api chown is 0.2s)
mastra post-fix: 2:16
Smoke test on ag2 image: container starts clean, all files under /app
owned by app:app, /api/health returns 200.
Under the 23-way Depot fan-out the real-world win is substantially
larger than the local 50s baseline because recursive chown degrades
super-linearly with concurrent I/O pressure.
Failed run that motivated this: https://github.com/CopilotKit/CopilotKit/actions/runs/24621469277
Cold-start on showcase packages with heavy Python agents (agno in particular)
consistently lands just over the 25s budget, producing 502s on first probe
with latency_ms: 25001 and stage: "timeout". Raise the upstream
AbortSignal.timeout on /api/smoke from 25s to 45s across all 16 showcase
packages that exercise the full agent round-trip, and bump Next.js
maxDuration from 30s to 60s so the route can actually run that long.
Also bumps the create-integration template so new packages inherit the
new budget.
Tail-latency beyond 45s will still alert — which is the intent.
Evidence: agno 502 alerts tonight at 18:40 PDT and 19:42 PDT, both with
latency_ms: 25001, stage: "timeout" on /api/smoke. /api/health already
200 on both probes — pure cold-start boundary issue.
## Summary
Adds a one-command local smoke harness so the full 17-integration suite
can be exercised against Docker on the dev machine instead of Railway.
Useful when Railway is degraded (aimock OOM, rate limits, cold-start
drift) or when validating changes that haven't been deployed yet.
## Usage
```bash
# one-time
cp showcase/.env.example showcase/.env # fill in keys
pnpm --filter @showcase/e2e-smoke install
# full L1-L4 smoke
pnpm --filter @showcase/e2e-smoke smoke:local
# single level / keep containers up between runs
pnpm --filter @showcase/e2e-smoke smoke:local:L1
pnpm --filter @showcase/e2e-smoke smoke:local:keep
pnpm --filter @showcase/e2e-smoke smoke:local:nobuild
```
## What's in here
- **`docker-compose.local.yml`**: `aimock` added as 18th service →
integration containers reach `http://aimock:4010` on the compose
network, mirroring Railway's `showcase-aimock`.
- **`integration-smoke.spec.ts`**: `LOCAL_PORTS=1` env gates URL
rewriting from `https://showcase-<slug>-production.up.railway.app` →
`http://localhost:<port>` via `shared/local-ports.json`. Starters are
skipped under the flag because they're not in `local-ports.json`.
- **`scripts/smoke-local.sh`**: thin orchestrator — `build → up → wait
20s → playwright → down`. Flags: `--level=L1|L2|L3|L4`, `--keep`,
`--no-build`.
- **`tests/package.json`**: `pnpm smoke:local[:L1|:keep|:nobuild]`
wrappers.
- **`.env.example`**: documents optional
`OPENAI_BASE_URL`/`ANTHROPIC_BASE_URL` + `GitHubToken` (ms-agent-dotnet)
and `GOOGLE_API_KEY` (google-adk).
## Verification
Run locally against a fresh checkout of this branch:
- `LOCAL_PORTS=1 SMOKE_ALL=true npx playwright test integration-smoke
--grep @health` → **17/17 pass in 478ms**
- Full L1-L4 against the local stack → **42/51 pass** (9 failures in
L3/L4 for mastra, google-adk, ms-agent-dotnet, strands, langroid,
spring-ai — these are test-data / fixture gaps unrelated to this
infrastructure and will be filed separately)
- `docker compose -f showcase/docker-compose.local.yml config` validates
with 18 services
## Scope
Pure dev-ergonomics addition. No runtime behaviour changes in the
shipped containers. `LOCAL_PORTS` is opt-in; unset = existing
Railway-URL behaviour preserved.
## Test plan
- [ ] `Validate Showcase` CI still green (no package-source changes)
- [ ] No unrelated CI regressions
- [ ] Follow-up PR will investigate and fix the 9 L3/L4 failures
surfaced by local smoke
- test-cleanup.ts: `new Error(msg, { cause })` is ES2022; workspace lib is
ES2020 so the two-arg overload is missing. Replaced with an
`errorWithCause()` helper that assigns `.cause` after construction.
Runtime is identical (Node >=16.9); only the TS signature differs.
- test-cleanup.ts: retyped `SAFE_STDIO` as `StdioOptions` (still frozen at
runtime to keep `test-cleanup.test.ts` freeze assertion green) so
spreading `SAFE_EXEC_OPTS` into `execFileSync(..., opts)` no longer trips
the readonly-vs-mutable-array mismatch on `stdio` (fixes the error at
create-integration.test.ts:132).
- create-integration/index.ts: dropped unused `devCmd` local and unused
`args` parameter on `generateDemoPage` (+ call site); both were dead code
introduced during the parallel-isolation refactor.
- validate-pins.parsers.test.ts: annotated all `withTmp((tmp) => ...)`
callbacks as `(tmp: string)` for robustness under LSP module-resolution
glitches. Matches the contract in `validate-pins.shared.ts`.
Tests: 1061/1061 pass (`pnpm nx run @copilotkit/showcase-scripts:test`).
Prior reasons for `fileParallelism: false` are resolved:
- Env-var mutation races (VALIDATE_PINS_REPO_ROOT, SHOWCASE_AUDIT_ROOT,
VALIDATE_PARITY_REPO_ROOT) are moot under `pool: 'forks'` — every file
already gets its own node process with its own `process.env`.
- `.git/index.lock` races between suites that call `restoreFromGitHead`
are fixed by the cross-process lock in test-cleanup.ts.
- The create-integration vs generate-registry collision on
`showcase/packages/` is fixed by the create-integration tmpdir
isolation.
Under fork-per-file + parallel, each test file also gets a fresh 60s
birpc `onTaskUpdate` budget (vitest #6129), eliminating the cumulative
RPC back-pressure that tripped unit(20.x/22.x/24.x) on #4018/#4068/#4079.
Empirical: local full suite 158s → 12s, 1061/1061 passing across three
consecutive `--skip-nx-cache` runs with zero timeouts, zero ENOENT, zero
index.lock contention.
audit.test.ts is 3034 lines / 119 tests and on Node 22 CI its single-file
runtime grew from 36.7s (PR #4071) to 71.4s (PR #4081) — over the hardcoded
60s birpc onTaskUpdate RPC window (vitest #6129). Same cliff that motivated
the validate-pins split earlier in this PR.
Extract makeTmpTree, makeConfig, writePackage, makeExampleDir, anomalyStrings,
and the AUDIT_SCRIPT path constant into audit.shared.ts so the forthcoming
split files can share them without duplication. No behavior change — the
original audit.test.ts still re-declares its own local copies until the
split commit removes them.
Break the 3567-line validate-pins.test.ts into five smaller test files so
each one fits comfortably under vitest's hardcoded 60s birpc onTaskUpdate
RPC window (upstream vitest #6129: DEFAULT_TIMEOUT = 6e4 in the bundled
birpc). pool: 'forks' + fileParallelism: false already gives each file a
fresh 60s RPC budget; splitting ensures no single file is anywhere near
that cliff even when CI is slow.
Split buckets, chosen for logical cohesion and balanced subprocess load:
- validate-pins.parsers.test.ts - pure parsers (no validateAll, no subprocess)
- validate-pins.validate-all.test.ts - in-process validateAll + drift detection
- validate-pins.cli.test.ts - CLI subprocess exit codes
- validate-pins.eacces.test.ts - chmod/EACCES-routed subprocess tests
- validate-pins.r-scenarios.test.ts - R29/R33 regression scenarios
Test count is identical pre/post-split (134 tests). Behaviour-neutral
refactor: no test logic changes, only file boundaries + shared helper
imports.
Hoist tmpdir/write/withTmp helpers and FIXTURES_DIR/VALIDATE_PINS_SCRIPT
path constants out of validate-pins.test.ts so the forthcoming split files
can share them without duplication. Behaviour-neutral.
create-integration.test.ts invoked the real generator against
`showcase/packages/` and `.github/workflows/`, then healed the mutations
in afterEach. Under `fileParallelism: true` that collided with
generate-registry.test.ts (concurrent readdirSync of `showcase/packages/`
observed partial state → ENOENT) and with every suite that restored
workflow YAMLs from git (`.git/index.lock` contention).
Teach `create-integration/index.ts` to honor two env overrides —
`CREATE_INTEGRATION_PACKAGES_DIR` and `CREATE_INTEGRATION_WORKFLOWS_DIR`
— that redirect its writes to any directory. Production behavior
unchanged (defaults resolve to the real paths as before).
Rewrite the test to create a per-suite `os.tmpdir()`-backed root, seed
it with copies of the three real workflow YAMLs so the generator's
regex-based edits still match, and point both env vars there. The test
now never mutates a tracked file — no restorer, no git invocation, no
cross-suite shared state. generate-registry can scan real
`showcase/packages/` concurrently without collisions.
The regression-guard test still exercises the same cleanup semantics,
just against the tmpdir-backed baseline map instead of
`workflowRestorer.snapshotMap`.
`restoreFromGitHead` runs `git checkout HEAD -- <paths>` inside three
sibling suites (bundle-demo-content, generate-registry, create-integration)
plus concurrent `git` from the pre-commit hook. Every one of those grabs
`.git/index.lock` — parallel callers race for it and flake with
"fatal: Unable to create .git/index.lock: File exists".
Acquire a cross-process advisory lock (atomic `fs.mkdirSync` of
`/tmp/copilotkit-showcase-git-restore.lock`) around every git invocation
in this module: partition, pre-heal diff, checkout, post-heal diff. Held
for the entire sequence so intermediate state is consistent from the
caller's perspective. Stale locks (> 60s) are reaped before the wait loop
so a hard-killed previous run can't wedge subsequent runs.
Unblocks enabling vitest `fileParallelism: true` — the three consumer
suites can now run in parallel forks without stepping on each other's
git operations or on the pre-commit hook's.
Fixes upstream birpc onTaskUpdate timeout (vitest-dev/vitest#8164, fixed
by #8297, v4-only). Root package.json already on ^4.1.3; this aligns
showcase/scripts, which held the only remaining ^3.0.0 pin and was
therefore the only package affected by the bug.
Local verification: 3 consecutive `pnpm nx run
@copilotkit/showcase-scripts:test --skip-nx-cache` runs on Node 20.20.2,
all green, 1061/1061 passing, zero onTaskUpdate errors, zero unhandled
exceptions (wall times: 34s / 30s / 29s).
- Shell /code viewer now builds a recursive file tree with core-only
(★ highlighted) and show-all-files toggle via ?view=all; collapses the
legacy flat files + backend_files arrays into one tree
- bundle-demo-content: strict mode — errors on missing highlight paths;
drop backend_files field; pull in external backend files referenced
by highlight: (column-relative paths) alongside demo-folder contents;
stable page-first ordering
- Update tests to reflect new column-relative filename shape
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Split generative-ui category into 4 (controlled/declarative/open/
operational), add chat-customization-css + tool-rendering-frontend-tools,
replace old tool-rendering-status/-result IDs with default-catchall /
custom-catchall. Port langgraph-python manifest features + highlight:
paths (column-relative to preserve pre-existing Docker structure).
Update tests for new category/feature counts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the regex-over-YAML options-block scanner with a real YAML
parse + typed navigation down on.workflow_dispatch.inputs.service.
options. The previous regex depended on a brittle terminator
(`^\S | \n\s*\w+:\s*\n\s{6,}\w+:|\nconcurrency:`) that
would silently break under trivial reformats — adding a top-level key
after `on:` or reordering `concurrency:` would have produced a
false-positive pass.
ALL_SERVICES stays regex-scanned because its matrix is embedded as a
bash heredoc inside a run: step, not as a YAML sub-structure. We do
locate the step via YAML (jobs.detect-changes.steps[*] with run body
referencing ALL_SERVICES) and only then scan `"dispatch_name":"X"`
occurrences inside that step's run body. Cannot JSON.parse the matrix
directly — it interpolates ${{ github.sha }} expressions that aren't
valid JSON pre-execution.
NIT 2: drop the dual `import.meta.url === \`file://${argv[1]}\``
branch and keep only the canonical `fileURLToPath(import.meta.url)`
comparison. The file:// form was belt-and-suspenders for a
pre-fileURLToPath era of Node; Node 20+ handles the modern form
uniformly.
Verified:
- All 12 validator tests pass (substring-spoof, missing-from-both,
template-excluded, empty-starters, missing-workflow).
- Real showcase_deploy.yml returns OK: all 17 starter(s) registered.
Every peer validator under showcase/scripts/ already has a matching
__tests__ suite — this one shipped without. Model the new suite on
validate-parity.test.ts: per-test tmpdir fixtures + CLI subprocess
runs gated by VALIDATE_WORKFLOW_STARTERS_REPO_ROOT.
Extend the existing env-var override to re-home STARTERS_DIR as well
(previously it only redirected .github/). One env var, one root,
matches the pattern used by the other validators.
Coverage:
- happy path (all slugs registered, exit 0)
- slug missing from workflow_dispatch options only (exit 1)
- slug missing from ALL_SERVICES matrix only (exit 1)
- slug missing from both (exit 1, both sources named)
- empty showcase/starters/ (exit 3, refuses trivial pass)
- template/ excluded (not flagged as missing)
- substring-spoof: starter-ag2 missing vs starter-ag2-extended present
— regression guard for the word-boundary anchor
- showcase_deploy.yml absent (exit 3)
All 12 specs pass against the existing regex-based checks. The
substring-spoof case in particular pins the word-boundary contract so
a later refactor can't silently lose it.
The SERVICES=(...) array in showcase_smoke-monitor.yml's 'Check image
drift' step was a hardcoded copy of the 17 starter slugs already
declared by showcase/starters/*/ directory names. Every new starter
required a manual edit in three places; the parity validator caught
drift after the fact but couldn't prevent it.
This commit:
- Adds a sparse actions/checkout step for showcase/starters/ only.
- Replaces the literal starter-* entries with a filesystem enumeration
(for dir in showcase/starters/*/; do ... done), skipping template/.
- Fails loudly if the enumeration produces zero starters, so a broken
checkout can't silently under-check drift.
- Updates validate-workflow-starters.ts to drop the smoke-monitor check
(drift is now structurally impossible) while keeping the two remaining
literal-list checks against showcase_deploy.yml (workflow_dispatch
options must be literal pre-checkout; ALL_SERVICES matrix carries
per-starter deploy metadata like railway_id).
Non-starter services stay literal — they don't live under
showcase/starters/ and are provisioned differently.
The scripts under showcase/scripts/ run via tsx at runtime and had no
tsconfig.json, which left LSP falling back to no-config defaults and
flagging bogus errors ("Cannot find module 'fs' / 'path' / 'url'",
"Cannot find name 'process'"). Adds a minimal tsconfig that matches
tsx's runtime semantics: bundler resolution (so existing extensionless
relative imports keep working), ES2022 + DOM libs, Node types, strict
discriminated-union narrowing. Excludes __tests__/ to avoid surfacing
pre-existing type issues outside this PR's scope.
The starter slug list is duplicated across at least three places:
- .github/workflows/showcase_deploy.yml workflow_dispatch options
- .github/workflows/showcase_deploy.yml ALL_SERVICES matrix entries
- .github/workflows/showcase_smoke-monitor.yml SERVICES bash array
Adding a new starter under `showcase/starters/` but forgetting any of
these leaves the service deployable in theory but invisible to the
dispatch UI and/or drift detection — exactly the failure mode this PR
is trying to close.
Add `showcase/scripts/validate-workflow-starters.ts`. It enumerates
every directory under `showcase/starters/` (excluding `template/`)
and confirms `starter-<slug>` is present in each of the three
workflow locations, emitting a precise "missing from: <source>"
diagnostic per gap.
Wire it into showcase_validate.yml right after `validate-parity` so
it gates every PR + main push touching `showcase/**` or the relevant
workflow files. Also extend that workflow's `on.paths` filter to
include `showcase_deploy.yml` and `showcase_smoke-monitor.yml` so
edits to those files trigger the parity check.
Wires up a single-command path to run the full integration smoke suite
against a local Docker stack instead of Railway. Useful when Railway is
degraded (OOM, rate limits) or when testing changes that haven't been
deployed yet.
Additions:
- showcase/docker-compose.local.yml: add `aimock` as 18th service so
integration containers can reach http://aimock:4010 on the compose
network, mirroring the Railway setup where they call showcase-aimock.
- showcase/tests/e2e/integration-smoke.spec.ts: `LOCAL_PORTS=1` env
rewrites each integration's Railway URL to http://localhost:<port>
via showcase/shared/local-ports.json. Starters are skipped under this
flag because they aren't in local-ports.json.
- showcase/scripts/smoke-local.sh: orchestrates build → up → wait → run
Playwright → tear down. Supports --level=L1/L2/L3/L4, --keep, --no-build.
- showcase/tests/package.json: `pnpm smoke:local[:L1|:keep|:nobuild]`
scripts delegate to the helper.
- showcase/.env.example: document optional OPENAI_BASE_URL +
ANTHROPIC_BASE_URL (route through local aimock) and package-specific
GitHubToken + GOOGLE_API_KEY (ms-agent-dotnet, google-adk).
Verified locally: `pnpm smoke:local:L1` → 17/17 L1 green against the
local stack.
Real pin-fix PRs have landed on `main` since the baseline was last
refreshed, reducing the FAIL set from 111 to 109 and invalidating the
stored hash. The ratchet gate now rejects every PR (including ones
that don't touch pins) with a "ratchet down" instruction.
Refresh the baseline to reflect the actual state of `origin/main`:
validatePinsFailCount: 111 -> 109
validatePinsFailHash: 77b586b7 -> d03716b5
Values computed by running the canonical pipeline from
`.github/workflows/showcase_validate.yml` ("Run validate-pins (ratchet)"
step) against a clean `origin/main` worktree:
pnpm exec tsx showcase/scripts/validate-pins.ts 2> stderr
grep ^Summary stdout -> FAIL=109
grep '^\[FAIL\]' stderr | LC_ALL=C sort -u | shasum -a 256
-> d03716b5...f597e81d
This is a pure ratchet-down to match reality, not a policy change.
No validator behavior, workflow, or pin change is included. Actual
pin drift cleanup (109 -> 0) continues as a separate effort.
Unblocks #4068 and any other PR stalled on the same ratchet.
To support exhaustive E2E testing via multiple variants per feature ×
framework, extend the status model with an optional `variants[]` array
per demo (each variant has the same demo/code/E2E/Smoke/QA/health
breakdown) and mount five different visual treatments so we can compare
side-by-side before committing to one:
- `/variants-stack` — each variant rendered as its own mini-row
in the cell; tall cells, all info visible.
- `/variants-tabs` — tabs at the cell top, click to switch
variant; cell stays compact.
- `/variants-aggregate` — pass/total rollups per signal +
"N variants ▾" expand button to drill down.
- `/variants-grid` — mini-matrix: rows = variants, cols =
demo/code/E2E/Smoke/QA/health.
- `/variants-strip` — one colored chip per variant per signal;
hover chip for variant name, click for URL.
Refactor: the grid chrome moves to `components/feature-grid.tsx`
(accepts a `renderCell` callback). Main `/` keeps the existing
single-variant layout via `components/cell-single.tsx`. Shared badge /
links helpers live in `components/badges.tsx` and
`components/variant-pieces.tsx`.
Mock variant data is seeded on 4 demos (langgraph-python's
agentic-chat, gen-ui-tool-based, hitl-in-chat; langgraph-typescript's
agentic-chat) so each option shows variants in context alongside
ordinary cells.
Variant-specific deep links append `?variant=<name>` to the shell
preview / code / hosted URLs — the shell routes can pick that up
later to highlight variant-specific files or payloads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New `scripts/generate-status.ts` — probes every
`integration.backend_url + demo.route` in parallel (~166 URLs) and
writes health status per demo to `shell/src/data/status.json`. E2E,
Smoke, and QA stay mock with explicit `TODO(wire-*)` comments; real
readers for those ingest from `showcase_aimock-e2e.yml`,
`showcase_smoke-monitor.yml`, and `showcase_qa-sync.yml` later.
`GENERATE_STATUS_MOCK_HEALTH=1` offline override for dev.
- Health badge in the feature-matrix cell is now clickable — opens the
hosted URL (`integration.backend_url + demo.route`) in a new tab,
with a tooltip noting the last probe time + status.
- Ran the probe once against Railway: 10/22 langgraph-python demos up
(pre-merge features), 12/22 down (new demos on this branch, not yet
deployed). Other 16 integrations fully live.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
Three showcase test suites leak working-tree drift (workflow YAMLs +
shell data JSONs) on every run. This fixes the leaks and adds a shared
`FileSnapshotRestorer` + `restoreFromGitHead` harness so the suites are
idempotent.
Also moves `showcase/scripts/vitest.config.ts` from the thread pool to
the fork pool, which is required under Node 20 for the test subprocess
churn in `validate-pins` + the three generator-invoking suites.
## What this PR does NOT fix — vitest 3.2.4 RPC timeout on Node 20
(upstream)
`unit (20.x)` still reports `Timeout calling "onTaskUpdate"` ->
ELIFECYCLE **after** all 14/14 test files and 1011/1011 tests pass.
Known upstream bug: https://github.com/vitest-dev/vitest/issues/6129.
The birpc timeout is hardcoded at 60 s (`DEFAULT_TIMEOUT = 6e4` in
vitest's bundled `index.B521nVV-.js`) and is NOT exposed to
`vitest.config.ts`. No `poolOptions.forks.*` / `teardownTimeout` /
`hookTimeout` knob influences it:
- `singleFork: true` made the run STRICTLY WORSE — only 1/14 files
completed (validate-pins consumes the whole 60 s budget on its own, run
24602985507).
- `fork-per-file` (default) gives each file a fresh RPC channel, and
every file passes — but the final pool-teardown RPC still races on Node
20 and surfaces as a process-level exit 1.
Under vitest 3.2.4 the fix requires either (a) upgrading to vitest 4.x
(out of scope — monorepo-wide upgrade), or (b) a pnpm patch against the
bundled `DEFAULT_TIMEOUT` constant (cross-cutting change; declined
here). The observable reality: this PR makes the showcase-scripts suites
idempotent and green; the remaining `unit (20.x)` redness is a known
vitest flake orthogonal to what this PR is trying to fix.
## CI-killer error (verbatim)
```
Error: Package directory already exists: /home/runner/work/CopilotKit/CopilotKit/showcase/packages/test-integration-tmp
⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯
Error: [vitest-worker]: Timeout calling "onTaskUpdate"
ELIFECYCLE Test failed.
Failed tasks:
- @copilotkit/showcase-scripts:test
```
## Root cause (fixed here)
Three test suites invoke real generator scripts that write to tracked
files OUTSIDE any tmp dir, leaking drift on every `nx run-many -t test`:
- `create-integration.test.ts` scaffolds
`showcase/packages/test-integration-tmp/` AND mutates three CI workflow
YAMLs (`showcase_deploy.yml`, `showcase_drift-detection.yml`,
`starter-smoke.yml`).
- `generate-registry.test.ts` rewrites
`showcase/shell/src/data/registry.json` + `constraints.json`.
- `bundle-demo-content.test.ts` rewrites
`showcase/shell/src/data/demo-content.json`.
## Fix (9 commits, by area of concern)
1. **`test(showcase/scripts): add shared test-cleanup snapshot/restore
helper`** — new `__tests__/test-cleanup.ts` + `__tests__/paths.ts` +
direct unit coverage in `__tests__/test-cleanup.test.ts`:
- `FileSnapshotRestorer` — snapshots file content as `Buffer`
(byte-exact, preserves non-utf8), restores only files that drifted,
writes atomically via temp-file + rename, recreates parent dirs on write
ENOENT. Temp filenames use `crypto.randomBytes(8).toString("hex")` so
concurrent writes can't collide and the `snapshot()` sweep can
unambiguously identify stragglers. On `snapshot()`, sweeps
`.<basename>.<16-hex>.tmp` stragglers **scoped to the snapshotted
basenames only**.
- `restoreFromGitHead(repoRoot, paths)` — partitions the input via `git
ls-files --error-unmatch` BEFORE any destructive op. **Narrow catch**:
only genuine exit-1 pathspec errors are treated as untracked; ENOENT /
EACCES / non-exit-1 failures re-raise with captured stderr. Uses
`execFileSync` (no shell), forces `LC_ALL=C` / `LANG=C`, scrubs all
`GIT_*` environment overrides, frozen exec options.
- `test-cleanup.test.ts` itself strips `GIT_*` from child env when it
creates tmp repos — pre-commit hooks (lefthook) run with `GIT_DIR` /
`GIT_INDEX_FILE` set, which would otherwise cause tmp-repo `git commit`
calls to write to the HOST working-tree HEAD.
2. **`fix(showcase/test-integration): clean up test-integration-tmp
between runs`** — `create-integration.test.ts` wires
`FileSnapshotRestorer` + `restoreFromGitHead` into the suite, wraps
`rmSync` in `try/finally` so workflow restoration always runs, and
migrates `execSync(string)` -> `execFileSync("npx", [...args])` via a
shared `runGenerator()` helper.
3. **`fix(showcase/test-integration): stop generate-registry +
bundle-content leaks`** — same pattern applied to
`generate-registry.test.ts` and `bundle-demo-content.test.ts`; drops a
redundant bundler pre-run in the latter.
4. **`fix(showcase/scripts): switch vitest to forks pool for Node 20
stability`** — `vitest.config.ts`: thread -> fork pool.
5. **`docs(showcase/scripts): tidy test-cleanup comments and JSDoc`** —
documentation cleanup.
6. **`fix(showcase/scripts): pin vitest to a single fork + bump teardown
timeouts`** — SUPERSEDED by commit 9 below (left in history for
auditability).
7. **`fix(showcase/scripts): add post-heal drifted-baseline guard`** —
the PR had claimed a drifted-baseline guard on CI for
`restoreFromGitHead`, but no post-heal `git diff --quiet` was actually
running. Adds the missing check: on CI, any tracked path still drifted
post-heal throws `drifted-baseline guard: post-heal diff failed`; off-CI
warns. Red-green unit coverage via a counter-based git shim that
selectively fails the N-th `diff --quiet` (so the post-heal diff is
targeted independently of the off-CI pre-checkout diff).
8. **`fix(showcase/scripts): decouple generate-registry test 2 from test
1 output`** — `sorts integrations by sort_order` was reading
`registry.json` without invoking the generator, so `afterEach(restore)`
between tests meant it was exercising the committed baseline rather than
live output. Adds a `runGenerator()` call at the top.
9. **`fix(showcase/scripts): revert singleFork — fork-per-file is
strictly better`** — empirical data from run 24602985507 proved
`singleFork: true` was worse than fork-per-file (1/14 vs 14/14 files
completing before RPC timeout). Reverts the
`poolOptions.forks.singleFork` change; keeps the 30 s `teardownTimeout`
/ `hookTimeout` bumps. Comments now accurately reflect that the RPC
timeout is upstream-hardcoded in birpc and NOT tunable via vitest
config.
## Proof of idempotence
```
pnpm nx run @copilotkit/showcase-scripts:test --skip-nx-cache
Run 1: Test Files 14 passed (14), Tests 1011 passed (1011)
Run 2: Test Files 14 passed (14), Tests 1011 passed (1011)
git status after each: only the intentional test file edits.
```
Red-green verified for the HIGH CR-findings:
- narrow `partitionTrackedPaths` catch: unit test with empty PATH
reproduces ENOENT; pre-fix hid it as "untracked", post-fix throws.
- basename-scoped tmp sweep: unit test places both target-basename and
unrelated `.something-else.<hex>.tmp`; pre-fix swept both, post-fix
sweeps only target.
- post-heal drifted-baseline guard: counter-based git shim fails the 2nd
`diff --quiet`; pre-fix tests pass (guard absent), post-fix tests throw
on CI / warn off-CI with the advertised message.
## Test plan
- [x] `pnpm nx run @copilotkit/showcase-scripts:test` passes 1011/1011
twice in a row (locally, Node 25)
- [x] Red-green: disabling `restore()` fails the regression + safety-net
tests
- [x] Red-green: disabling the post-heal drift guard fails the new guard
tests
- [x] Working tree clean after full run
- [x] `prettier --check` + `oxlint` clean on touched files
- [x] `GIT_*` scrub in test harness prevents pre-commit-hook-induced
pollution of real HEAD
## CI status
- **`unit (22.x)`**: pass
- **`unit (24.x)`**: pass
- **`unit (20.x)`**: all 14/14 files + 1011/1011 tests pass; post-suite
`onTaskUpdate` RPC timeout emits exit 1. Upstream bug
https://github.com/vitest-dev/vitest/issues/6129; not fixable at
`vitest.config.ts` level on vitest 3.2.4.
## Caveats
- Local verification ran on Node v25.8.0. No Node 20 binary on this dev
host; Node 20 CI was the final gate.
- The residual `unit (20.x)` failure is orthogonal to this PR. To
resolve it we would need to upgrade vitest to 4.x (monorepo-wide change)
or apply a pnpm patch to bump `DEFAULT_TIMEOUT` in vitest's bundled
`birpc`. Both are tracked separately.
Problem: shell's /code viewer was showing every `.py` file under
`src/agents/` for every demo of a package. Visually contaminating:
opening gen-ui-tool-based (Controlled Gen-UI Display) showed
a2ui_dynamic, a2ui_fixed, mcp_apps_agent, open_gen_ui_agent,
reasoning_agent, interrupt_agent, and tool_rendering_agent in the
file picker even though none of them are relevant to that demo.
Root cause: `bundle-demo-content.ts` ran `discoverBackendFiles()` once
per package and attached the same union of all agent files to every
demo. This was fine when all demos shared one graph, but since we
split demos into dedicated graphs the bundle stopped matching reality.
Fix:
- `manifest.schema.json`: add optional `backend_files` field per demo
(string array, paths relative to the package root).
- `bundle-demo-content.ts`: when `demo.backend_files` is present, bundle
exactly those. Otherwise fall back to the legacy full-package scan
so packages that haven't adopted the field still work as before.
- `langgraph-python/manifest.yaml`: populate `backend_files` for every
demo. Each demo bundles `src/agent_server.py` plus only the agent
file its graph routes to (main.py for shared-graph demos;
reasoning_agent.py / interrupt_agent.py / a2ui_dynamic.py /
a2ui_fixed.py / mcp_apps_agent.py / open_gen_ui_agent.py /
tool_rendering_agent.py for demos with dedicated graphs).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
parseManifest (lib/manifest.ts):
- Shape errors for demos[i].route: number / null / object / empty string
- Shape error for route not starting with /demos/
- Happy path persists frozen demo.route on the parsed entry
- Backward-compat: demos[i] without route is accepted with undefined route
validate-parity:
- Negative-case regression: missing-demo-dir's expectedDir is derived from
demo.route (not demo.id) when route is present; mismatched id + route
was silently hiding drift.
- Fallback-case: demo with no route resolves expectedDir from demo.id
- routeToDirName unit tests: undefined / bare /demos/ / normal tail segment
TDD verified: mutating the /demos/ prefix guard failed the relevant test
(RED), restoring the guard passed it (GREEN). Mutating expectedDir to
demo.id-only failed the negative-case test (RED), restoring passed
(GREEN).
demo.id is the CATALOG identifier (matched to qa/spec filenames and
shell registry entries). demo.route is the URL + filesystem path
(/demos/<dir> → src/app/demos/<dir>/). They are deliberately separate
— a manifest with id: hitl-in-chat and route: /demos/hitl lives at
src/app/demos/hitl/.
validate-parity.ts previously resolved the demo directory from
demo.id, producing a spurious missing-demo-dir MUST for every such
split. Fix:
- lib/manifest.ts: add optional route field to ManifestDemo; if present,
parser requires it to be a non-empty string beginning with "/demos/".
- validate-parity.ts: introduce routeToDirName helper (matches
bundle-demo-content.ts idiom); loop over demos resolving expected
dir from route and falling back to id. missing-demo-dir PackageIssue
now carries both demoId and expectedDir so deriveMessage can flag
route-resolved paths distinctly.
- __tests__/validate-parity.test.ts: red-green regression test — a
package with id: hitl-in-chat, route: /demos/hitl, and dir
src/app/demos/hitl/ must PASS (no missing-demo-dir error).
Parallel review subagents each need a fresh
showcase-langgraph-python build to screenshot. Running
`dev-local.sh up langgraph-python` N times in parallel serializes on
Docker and wastes time rebuilding identical source.
`scripts/agent-notes/rebuild-coord.sh` aligns rebuild attempts to a
30-second clock tick with 0-5s jitter. First agent on a tick acquires
a `mkdir`-based lock, rebuilds, and writes a per-tick done marker at
`/tmp/showcase-rebuild.done-<tick>`. Peers on the same tick skip their
own build and wait up to 30s for that marker. If no marker appears,
they retry the next tick (up to 3 ticks).
Agents call `bash scripts/agent-notes/rebuild-coord.sh` instead of
`./scripts/dev-local.sh up langgraph-python` when they need the live
container refreshed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds rows for the demos being built in this branch:
- Split `chat-prebuilt` into `prebuilt-chat`, `prebuilt-sidebar`,
`prebuilt-popup` (CopilotChat / CopilotSidebar / CopilotPopup).
- Reorder `generative-ui` rows and add: `hitl-in-chat` (In-Chat HITL),
`gen-ui-interrupt`, `declarative-gen-ui` (Dynamic Schema),
`a2ui-fixed-schema` (new), `mcp-apps`, `open-gen-ui`, `gen-ui-agent`,
`tool-rendering`.
- `constraints.yaml`: expand `constrained-explicit` allowlist so the
langgraph-python manifest validates against the new feature ids.
- Bump expected langgraph-python feature/demo count in
`generate-registry.test.ts` from 10 -> 22 to match the manifest.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Run 24602985507 (singleFork: true) was strictly worse than the prior
fork-per-file run (24602657301):
singleFork (24602985507): 1/14 files completed, 64.64s, then timeout
fork-per-file (24602657301): 14/14 files passed, 169.56s, then teardown timeout
The RPC timeout under Node 20 is a hardcoded 60 s DEFAULT_TIMEOUT in
birpc (DEFAULT_TIMEOUT = 6e4 in vitest's index.B521nVV-.js, not tunable
via config). validate-pins.test.ts alone takes ~60 s because of its
134 `npx tsx` subprocess invocations, so singleFork blows its RPC
budget before a single file finishes. Fork-per-file gives each file
its own fresh 60 s RPC channel, which is why the prior run got all
14/14 files to complete before the final teardown hiccup.
Revert `poolOptions.forks.singleFork: true`. Keep the teardownTimeout
/ hookTimeout bumps to 30 s — they don't affect the RPC timeout but
do protect against slow per-hook teardowns under the same load. Note
in comments that the RPC timeout is upstream-hardcoded and tracked at
https://github.com/vitest-dev/vitest/issues/6129.
`sorts integrations by sort_order` read registry.json without invoking
the generator, making it dependent on whichever run last left disk
state in place. afterEach(dataRestorer.restore()) restores the data
files to HEAD between tests, so test 2 was reading the committed
baseline registry — not the live generator output — and would have
silently agreed with whatever state happened to be on main.
Add a `runGenerator()` call at the top of test 2 mirroring the
`runBundlerAndRead()` pattern that tests 2-5 of bundle-demo-content
already use. Test now exercises the actual generator's sort
behavior under live conditions.
The PR description advertised a drifted-baseline guard on CI for
restoreFromGitHead, but the implementation never actually ran a
post-heal `git diff --quiet HEAD -- <tracked>` check — the only
diff was the off-CI pre-checkout guard against clobbering developer
edits. A racing external mutator (or a parallel test suite we haven't
accounted for) could rewrite a tracked file between our `git checkout
HEAD --` and our subsequent snapshot, and we'd silently bake the drift
into the baseline and the afterEach restore loop would maintain it
forever.
This commit adds the missing post-heal check immediately after the
`git checkout HEAD --`. On CI it throws with `drifted-baseline guard:
post-heal diff failed` citing every offending path; off-CI it warns so
developers iterating on a dirty tree aren't blocked.
Also:
- Rephrases the `isBenignPathspec` catch comment to describe the
realistic case (belt-and-braces against a rm race, not the normal
flow) now that partitionTrackedPaths pre-filters untracked paths.
The branch is intentionally kept — cheap to tolerate, and it
guards against a race that's hard to rule out in a shared worktree.
- Red-green unit coverage: new tests use a counter-based git shim
that fails the N-th `diff --quiet` invocation, so we can target
the post-heal diff independently of the off-CI pre-checkout diff.
- CI throws on drift with the advertised message
- off-CI warns (does not throw) with the same message
- no false-positive on a clean tracked path
Verified red without the guard, green with it.
Run 24602657301 showed unit (20.x) still emitting
`Timeout calling "onTaskUpdate"` during @copilotkit/showcase-scripts:test
teardown — 1008/1008 tests passing, then ELIFECYCLE. The previous fix
switched the pool from threads to forks but left the default fork-per-file
behavior, so every file teardown tore down and reestablished the
parent↔child RPC channel. Under Node 20 the channel occasionally fails
to reestablish within the 10s default teardownTimeout — a known vitest
issue (https://github.com/vitest-dev/vitest/issues/6129).
Two defenses:
- poolOptions.forks.singleFork: true — one long-lived fork across all
test files (combined with the existing fileParallelism: false, files
still run sequentially). The RPC channel stays warm for the whole
run instead of being torn down and reestablished between every file,
which eliminates the per-file teardown as a surface for the race.
- teardownTimeout / hookTimeout bumped from the 10s default to 30s,
matching testTimeout. A slow teardown can't be the bottleneck that
kills the run.
This is the most conservative setting short of dropping to a single-
thread pool entirely. Node 22 / 24 unaffected.
- Rewrite restoreFromGitHead JSDoc to match actual behavior for tracked
vs untracked paths (including the on-CI throw / off-CI warn for an
entirely-untracked input list).
- Rephrase internal review-round markers ("CR4/CR5 HIGH/MEDIUM") in
test-cleanup.ts and test-cleanup.test.ts to describe the behavior or
invariant being guarded instead of the review history.
- Point the WINDOWS comment at the sibling test files that actually
invoke npx (this module itself doesn't).
- Drop the "belt-and-braces" afterAll disclaimer from
bundle-demo-content.test.ts and generate-registry.test.ts — the
afterAll(restore) pattern is self-explanatory.
unit (20.x) CI was failing with 'Timeout calling onTaskUpdate' during
the showcase-scripts test run — an unhandled vitest error that causes
ELIFECYCLE after otherwise-green tests. Reproducible on Node 20 only;
22.x and 24.x are green on the same code.
Root cause: vitest's default thread-based worker pool times out on the
parent-worker RPC channel when a test file spawns many subprocesses
(validate-pins.test.ts runs 134 tests each invoking a subprocess;
create-integration / generate-registry / bundle-demo-content each
spawn npx tsx via execFileSync). Under Node 20 the stdio / signal
traffic from these children contends with the worker-thread RPC
channel and surfaces as an unhandled timeout mid-suite.
Switch to pool: 'forks' — the fork pool uses node IPC for the RPC
rather than worker-thread messageports and is robust under the same
load. fileParallelism: false keeps files sequential so shared-env /
tmp-dir mutations don't race, but each file now gets its own fresh
fork so one file's subprocess churn can't stall the RPC for
subsequent files. Node 22/24 unaffected either way.
Also pipes stdio explicitly on every git subprocess in
test-cleanup.test.ts — inherited stdio on a fork vitest worker
interleaves with the worker's own stdout/stderr and is another input
to the RPC contention under Node 20.
Jordan Ritter