dispatch-gate hardcoded --scope affected, so a manual dispatch always
ran the affected set. On a large PR that touches shared files, affected
resolves to every integration, and the eval's fleet bring-up then tries
to build 20+ images on one runner and dies (docker compose up -d exit
255). The comment path already supports targeting one slug
(/eval d5 mastra -> --slug mastra); this gives workflow_dispatch the same
lever via an optional `slug` input, validated with the same ^[a-z0-9-]+$
rule and read as untrusted env. Empty slug keeps the affected default.
This also makes the /eval fix validatable pre-merge: dispatch with
`-f slug=<one>` reproduces the real targeted comment path with a bounded
bring-up instead of the pathological affected=all case.
Dropping --ci moved the /eval job onto the CLI's Docker lifecycle, but
that immediately fails on a bare runner: docker-compose.local.yml declares
`env_file: .env` on every service and showcase/.env is gitignored, so
`docker compose` errors with "env file .../showcase/.env not found" before
any container starts. Validated: dispatch runs 29846236558 and 29845135213
both died ~6s into the eval step on exactly this.
Add a step that writes showcase/.env with dummy values before the eval.
aimock serves the recorded fixtures and never validates tokens, and the
aimock base URLs are already hardcoded in the compose environment block,
so the keys/URLs here just satisfy env_file and document intent.
The /eval job runs on a bare depot runner with no step that starts the
showcase fleet, but passed --ci to the eval CLI. --ci makes the CLI skip
the Docker lifecycle and assume services are already running, so it found
no healthy container and failed in ~1s with zero test results.
Drop --ci so the CLI's non-ci path builds + starts the in-scope slug(s) +
aimock and health-checks them before running. Safe with --json retained:
compose() uses piped (captured) stdio and all progress logs are
!opts.json-guarded, so stdout stays clean JSON for the post-result job.
The notify-all-builds-failed and notify jobs keyed off a 'failure' rollup /
bare failure(), so a build where every real service failed but one leg was
cancelled (contention) rolled up to 'cancelled' and sent no alert — the same
blind spot as the redeploy guard. Fire on any_success == 'false' (guarded by a
status function so a user-cancelled run stays silent). Extends the guard test.
## Showcase deploy-mechanism consolidation
Consolidates the showcase Railway deploy path onto a single
**CI-explicit** mechanism, so we can safely retire Railway's registry
auto-watch (the source of the surprise "Service aimock upgraded to
latest" emails). Design proposal: [Notion — Showcase Deploy-Mechanism
Consolidation](https://app.notion.com/p/3a33aa38185281e4b64cc5bebde92d91).
### What & why
The "aimock upgraded to latest" email was never a per-service config
choice — it was a **CI bug** letting Railway's watcher win a race: a
Renovate PR that only touches `showcase_build.yml` forces a full-fleet
rebuild; the LFS `shell` leg gets cancelled under runner contention; and
the `redeploy-staging` guard (`needs.build.result != 'cancelled'`) then
skipped the CI redeploy for the **whole fleet**, orphaning aimock's
fresh digest for Railway's watcher to pick up. The `autoUpdates` setting
itself had also silently drifted (24 services `minor` / 17 none) —
tracked in no SSOT, gated by nothing.
### The four changes (one commit each)
1. **`fix(showcase)` — the P0 guard bug.** Relax the `redeploy-staging`
**and** `redeploy-staging-starters` guards so a cancelled sibling leg no
longer skips the fleet's staging redeploy; they now redeploy the
already-computed successful-service list. A guard-evaluation test reads
the live workflow `if:` strings and models GitHub's matrix rollup.
2. **`feat(showcase)` — autoUpdates SSOT (per-env, staging-first).** Add
a **per-env** `autoUpdates` policy to every service in `railway-envs.ts`
— **staging: `disabled`** (enforced), **prod: `unmanaged`** (left
exactly as-is until a later migration). Regenerate
`railway-envs.generated.json`. CI-explicit redeploy becomes the single
deploy path on staging.
3. **`feat(showcase)` — drift gate.** New CI gate fails when a live
Railway service's `autoUpdates` diverges from the SSOT. Reads
`Environment.config` (autoUpdates isn't on the typed `ServiceSource`
output), **enforces managed (`disabled`) envs and skips `unmanaged`
ones** (so prod is untouched), **fails closed per-env** on zero-checked,
and skips cleanly on fork PRs with no Railway token.
4. **`feat(showcase)` — scheduled reconcile.** CI-owned self-heal (every
15m) comparing each staging service's deployed digest against GHCR
`:latest`, re-running the staging redeploy for lagging services and
alerting Slack. Invariant: **green ⟺ every in-scope service confirmed
current**; any unconfirmed service (lag, digest error, dropped redeploy,
empty scope, thrown redeploy) alerts and exits non-zero.
### Verification
- Every behavior change carries red-green tests; **230 tests pass**,
`tsc` clean, `oxfmt`/`oxlint` clean, generated JSON in sync, workflows
parse.
- Reviewed via a full CR loop (Tier 3, 5 rounds to convergence); the
reconcile's fail-loud invariant was hardened across rounds (silent-green
holes, stale-digest ordering, expansion false-positives, test hygiene).
### Rollout (staging-first)
- **Staging is flipped live as part of this change** — `autoUpdates`
disabled on all staging services (snapshot-first, verified only
`autoUpdates` changed). The drift gate now enforces staging.
- **Prod is untouched** — its `autoUpdates` stay exactly as-is and the
gate marks prod `unmanaged` (skipped). Migrating prod is a deliberate
follow-up (flip prod live + change prod SSOT `unmanaged`→`disabled`
together) once we're comfortable with staging on the new mechanism. No
transition window where anything is unguarded.
### Follow-ups (from CR, non-blocking)
- Dedup the reconcile alert's `unconfirmed` list by service key
(cosmetic double-listing; exit code already correct).
- Harden the sibling `notify-all-builds-failed`/`notify` jobs against
the same all-legs-cancelled rollup (pre-existing, in a job this PR
doesn't touch).
- Minor: `postSlackAlert` try/catch belt; a few added test assertions;
comment/doc accuracy.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Adds a CI-owned reconcile (every 15m) that compares each staging service's
deployed digest against GHCR :latest and re-runs the staging redeploy for
lagging services, alerting Slack. The run is green only when every in-scope
service is confirmed current; any unconfirmed service (lag, digest error,
dropped redeploy, empty scope, or a thrown redeploy) alerts and exits non-zero.
Exposes per-service redeploy records from redeploy-env for accurate per-service
remediation confirmation.
New CI gate fails when a live Railway service's autoUpdates diverges from the
SSOT (every service must be disabled). Reads Environment.config (autoUpdates is
not on the typed ServiceSource output), fails closed per-env when it verifies
zero services, and skips cleanly on fork PRs that lack a Railway token.
The redeploy-staging and redeploy-staging-starters jobs guarded on
needs.build.result != 'cancelled', so a single cancelled matrix leg (e.g. the
Git-LFS shell build under runner contention) skipped the whole fleet's staging
redeploy even when the other 27 services built fine. Relax both guards to
redeploy the already-computed successful-service list. Adds a guard-evaluation
test that reads the live workflow if: strings and models GitHub's matrix rollup.
Add an emoji reaction to the original promote-notify init message
reflecting the net run outcome, so operators can see success/failure at
a glance without opening the thread reply:
success -> white_check_mark (checkmark)
partial -> warning
total -> x
The live workflow calls reactions.add on the init post (guarded on a
successful init post, warn-only on failure to mirror the thread reply).
The dry-run harness emits the reaction it would add, using a
byte-identical case mapping enforced by a new anti-drift bats guard.
Adds bats coverage asserting the emitted reaction name per fixture.
## The incident
PR #6061 (`fix(starters): add python-multipart to agno starter`) merged,
its
`starter-agno` image built to GHCR via `showcase_build.yml`'s
`build-starters`
job — and then was **never deployed to Railway**. `starter-agno` stayed
crashed
until someone manually redeployed. That is not a one-off: it is
structural. Any
change touching **only** starter files
(`examples/integrations/<slug>/**`) hits
the same hole.
## Root cause (job-graph)
`showcase_build.yml` has two independent lanes:
| Lane | Detect | Build | Aggregate | Redeploy |
| --- | --- | --- | --- | --- |
| **Showcase fleet** | `detect-changes` | `build` (matrix) |
`aggregate-build-results` | `redeploy-staging` |
| **Starters** | `detect-starter-changes` | `build-starters` (matrix) |
— | **(none)** |
`redeploy-staging` is scoped **entirely to the showcase fleet**:
```
redeploy-staging:
needs: [detect-changes, build, aggregate-build-results]
if: >-
!cancelled()
&& needs.detect-changes.outputs.has_changes == 'true'
&& needs.build.result != 'skipped'
&& needs.build.result != 'cancelled'
&& needs.aggregate-build-results.outputs.any_success == 'true'
```
(`.github/workflows/showcase_build.yml:780-799`)
On a **starter-only** push:
- `detect-changes` filters are all `showcase/**` paths → none match
`examples/integrations/**` → `has_changes=false` → `build` skips →
`aggregate-build-results` skips → `redeploy-staging` skips (fails its
`has_changes=='true'` and `build.result != 'skipped'` clauses).
- `detect-starter-changes` matches → `build-starters` builds
`starter-<slug>:latest` to GHCR — **and stops.** There is no aggregate
and
**no redeploy job for the starter lane at all.**
Net: the starter image is built and pushed, but nothing ever calls
Railway's
`serviceInstanceRedeploy`, so the running container keeps the stale
image.
The `starter-build-result-*` per-slot artifacts already emitted by
`build-starters` were **write-only** — nothing consumed them.
## The fix
A new `redeploy-staging-starters` job that mirrors `redeploy-staging`
for the
starter lane, reusing existing mechanisms (no new script, no new
aggregator):
```
redeploy-staging-starters:
needs: [detect-starter-changes, build-starters]
if: >-
!cancelled()
&& needs.detect-starter-changes.outputs.has_changes == 'true'
&& needs.build-starters.result != 'skipped'
&& needs.build-starters.result != 'cancelled'
```
Steps:
1. Download the already-emitted `starter-build-result-*` artifacts
(`pattern` download → succeeds with zero matches if the build crashed
before writing any).
2. Compute `matrix ∩ build-success`: read each per-slot
`{"service":"<raw slug>","status":...}`, keep `status:success` slugs,
map
each **raw slug → `starter-<slug>` SSOT key** via the starter matrix
`.image` field. (The raw slug must NOT be passed to `redeploy-env.ts` —
e.g. `"agno"` collides with the **showcase** `agno` dispatch_name.)
3. If the CSV is non-empty, `npx tsx showcase/scripts/redeploy-env.ts
staging
--services <csv>` — the exact same invocation `redeploy-staging` uses.
`redeploy-env.ts` already resolves each `starter-<slug>` as an SSOT key
(verified: all 12 starter `.image` values exist as keys in
`railway-envs.ts`).
**Deploy-on-failure is impossible:** the "don't deploy on build failure"
guard
is the per-slot success intersection, not the job `if:`.
`build-starters`
`result == 'failure'` still enters the job (fail-fast is false, so some
slots
may have succeeded), but only `status:success` slots are redeployed; an
all-failed or crashed build yields an empty CSV → the redeploy step is
skipped.
This is the same net guarantee `redeploy-staging` gets from its
`any_success`
guard, computed inline to avoid standing up a second aggregator job.
**No `redeploy-summary` artifact is written** by this job on purpose:
that
name is owned by `redeploy-staging` and downloaded by
`showcase_deploy.yml` by
exact name — a second same-named upload would collide on a combined
push.
Starter *staging verification* is intentionally out of scope for this
deploy-gap fix (starters are already smoke-covered by
`test_smoke-starter.yml`
and the harness `starter_smoke` axis). `redeploy-staging-starters` was
also
added to the `notify` job's `needs` so a starter redeploy failure
alerts.
## Before / after truth table
| Scenario | `build` | `redeploy-staging` | `build-starters` |
`redeploy-staging-starters` |
| --- | --- | --- | --- | --- |
| **(a) main-fleet-only change** | runs | **redeploys fleet** | skipped
| skipped |
| **(b) starter-only change** | skipped | skipped | runs | **redeploys
starter (NEW)** |
| **(c) both changed** | runs | **redeploys fleet** | runs | **redeploys
starter (NEW)** |
| **(d) starter build failure** | (n/a) | (n/a) | failure | runs, but
CSV empty → **no redeploy** |
- **(a)** unchanged — the showcase lane is untouched.
- **(b)** is the fix: the starter now auto-deploys instead of sitting on
GHCR.
- **(c)** unchanged for the fleet; the starter additionally deploys. No
artifact
collision because `redeploy-staging-starters` uploads no
`redeploy-summary`.
- **(d)** partial failure redeploys only the slots that succeeded; a
wholesale
failure redeploys nothing.
## Fail-loud hardening (CR follow-up)
A CR flagged that the new `redeploy-staging-starters` job could itself
**silently under-deploy** — re-opening the very hole it exists to close.
Two
guards added to the `Compute successfully-built starter services` step,
mirroring
the sibling `redeploy-staging` job's empty-intersection guard:
1. **Empty deploy-set → fail loud.** When `build-starters.result ==
'success'`
(all slots built) but the `matrix ∩ success` CSV is **empty**, the job
now
`exit 1`s with an actionable `::error::` instead of silently skipping
the
redeploy at green CI (a slug↔`.image` contract skew, or a success set
that
maps to no matrix entry). Gated on `'success'` so a partial/total build
**failure** keeps the legitimate no-deploy path and is not
double-reported —
that failure is already surfaced by `build-starters` itself.
2. **Missing/unreadable result artifact → fail loud.** Dropped
`2>/dev/null` on
the `result.json` read so a read error surfaces and trips `pipefail`,
and
added a parsed-record-count vs matrix-slot-count assertion (every slot
writes
a `result.json` via `if: always()`, so on a full-success build the
counts must
match). A missing/expired `starter-build-result-*` artifact — which
would
otherwise silently drop a built starter from the redeploy set — now
fails the
job. Also gated on `'success'` so a crashed slot's legitimately-absent
artifact isn't double-reported on a build failure.
Updated truth table with the new fail-loud row:
| Scenario | `build-starters.result` | deploy CSV |
`redeploy-staging-starters` |
| --- | --- | --- | --- |
| success + non-empty CSV | success | non-empty | **redeploys** |
| success + **empty** CSV | success | empty | **exit 1 (NEW fail-loud)**
|
| build failure (partial) | failure | subset | redeploys successful
subset, no spurious exit |
| build failure (all/crash) | failure | empty | no deploy, failure
surfaced, no spurious exit |
| skipped (no starter changes) | skipped | (n/a) | job `if:` excludes it
— never runs |
Both guards were locally red→green exercised: pre-fix the empty-CSV and
missing-artifact cases went **green with nothing/partial deployed**;
post-fix
they `exit 1`. actionlint still 9/9 (no new findings).
## actionlint
Clean. Baseline (`origin/main`) = 9 findings; this branch = 9 findings,
all at
pre-existing lines (custom `depot-*` runner label + pre-existing SC2086
infos).
**Zero new findings** from the added job. The `matrix ∩ success` jq
mapping was
locally exercised across the four scenarios above
(success/partial/all-fail/
crash-before-artifacts) and produced the expected CSVs.
## Prod-promote path
**Does NOT share the gap.** `showcase_promote.yml` is
`workflow_dispatch`-only
("Humans trigger. No automatic prod promotes.") and already lists all 12
`starter-*` services in its choices with `resolve-targets` handling
them. There
is no push-driven prod path to fix.
## Residual verification (honest note)
Workflows can't be safely dry-run end-to-end (the redeploy path hits
live
Railway). Static validation is complete (actionlint clean, jq logic
exercised,
all 12 starter SSOT keys confirmed, `redeploy-env.ts` reused unchanged),
but
the true end-to-end confirmation is the **next starter-only change
auto-deploying to staging**. That first real starter-only merge after
this
lands should be watched to confirm `redeploy-staging-starters` fires and
the
Railway service picks up the new image.
---
Draft — do not merge until reviewed.
The new redeploy-staging-starters job pinned actions/checkout to
9c091bb (tag v7.0.0) but commented it # v7. zizmor's ref-version-mismatch
flagged the discrepancy: the v7 moving tag points to 3d3c42e, not 9c091bb.
Repin to 3d3c42e # v7 — the canonical checkout pin already used across
every other job on main — so the comment matches the SHA's tag.
The redeploy-staging-starters job could go green while deploying
nothing (all starters built, empty deploy CSV) or silently drop a
built-but-missing starter from the redeploy set. Both re-open the exact
gap this job exists to close.
- Finding 1: when build-starters.result == 'success' but the matrix ∩
success CSV is empty, fail the job (exit 1) instead of silently
skipping the redeploy. Mirrors redeploy-staging's empty-intersection
guard. Gated on 'success' so a partial/total build FAILURE keeps the
legitimate no-deploy path and isn't double-reported.
- Finding 2: drop 2>/dev/null on the result.json read (surface read
errors via pipefail) and assert parsed-record count == matrix slot
count when all slots built, so a missing/expired starter-build-result-*
artifact fails loud instead of silently under-deploying.
A push touching only starter files (examples/integrations/<slug>/**) built a
fresh starter-<slug>:latest image to GHCR via build-starters but never
redeployed it to Railway: the starter lane (detect-starter-changes ->
build-starters) ended at the GHCR push, and redeploy-staging only covers the
showcase build lane. Starter fixes sat undeployed until a manual redeploy
(the #6061 agno incident).
Add a redeploy-staging-starters job that mirrors redeploy-staging for the
starter lane: it reads the already-emitted per-slot starter-build-result-*
artifacts, intersects the build matrix with the build-success set (mapping raw
slug -> starter-<slug> SSOT key), and redeploys only the successfully-built
starters to staging. No deploy on build failure (empty success set -> redeploy
step skipped). Reuses redeploy-env.ts unchanged.
Add a `react-version` axis (18, 19) to the unit test workflow, spanning
the supported peer range (^18 || ^19) declared by react-core, react-ui,
and a2ui-renderer. React 19 installs against the committed lockfile;
React 18 overrides the root pnpm React version and installs unfrozen. A
guard step verifies the installed React matches the matrix leg.
Fixes a latent React 18 incompatibility the new matrix surfaces: the
`window = {}` test pattern crashes React 18's concurrent renderer with
"Should not already be working." mid-commit (22 failures across 5 files
on current main). Replace it with a `stubWindowLocation` helper that
clears `window.location` while keeping the real jsdom window intact. Add
React-version-gated assertions where R18 effect batching legitimately
differs from R19.
Reconstructs the durable parts of #4221 (Tyler Slaton) onto current
main, scoped to the supported React range — React 17 is dropped, as it
is no longer a supported peer version and carried the bulk of that PR's
complexity.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## What
Adds `wolfmanfx` (Murat Sari) and `rainerhahnekamp` (Rainer Hahnekamp)
as code owners of the Angular package paths in `.github/CODEOWNERS`:
- `packages/angular/`
- `packages/v2/angular/`
Existing core-dev owners are retained on both lines, so this is purely
additive.
## Notes
- **Access is granted separately.** This CODEOWNERS entry only assigns
review ownership; it does not grant repo access. The two contributors
still need to be added as repo collaborators via GitHub settings for the
entry to take effect (GitHub permissions are repo-level, not
folder-scoped).
- **Branch protection is currently disabled on `main`**, so this assigns
reviewers but does not gate merges until protection requiring code-owner
review is enabled.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The repo uses a flat package layout; there is no packages/v2/ directory,
so that CODEOWNERS entry never matches and fails path validation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The notify job's intent gate treated ANY workflow_dispatch as stable-release
intent (npm_intended=true), including prerelease/canary dispatches. canary.yml
orchestrates publish-release.yml with mode=prerelease on a short-lived canary/*
ref, so a canary whose build fails (e.g. a stale pnpm-lock.yaml on the source
branch tripping pnpm install --frozen-lockfile) flipped the notify job to
failure() and fired the self-watchdog's best-effort Slack post — a red
'a release alert may have been swallowed' page for a release that was never
going to publish.
Narrow the npm_intended dispatch arm with 'inputs.mode != prerelease' so a
canary yields npm_intended=false and never self-pages, while a stable dispatch
(default mode, and the main-only retry escape hatch) and a merged
release/publish/* PR still page correctly on genuine failure. This mirrors the
existing 'mode != prerelease' guards on the publish/tag steps and the builder's
should_post=false suppression for canaries.
Restore the 12 Python integration tools/ dirs to symlinks into
shared/python/tools. They had eroded to real, drifting copies via an
accidental stage_shared() leak (commit 534cd1efa7) — the structural root
cause of showcase divergence bugs. Symlinking re-establishes the single
source of truth; content is identical to shared (only render_a2ui naming
and the shared roll_dice/sanitize additions are adopted).
Add showcase/AGENTS.md documenting the 4 iron rules and the single-source
symlink mechanism, plus a validate-shared-symlinks CI guard (shrink-only
baseline) that fails on any NEW erosion.
## Two silent-failure gaps in the showcase build/deploy/notify pipeline
These are **pre-existing** silent-failure holes surfaced in code review
(not
caused by any recent PR). This PR fixes the two load-bearing ones.
### 1. Green-but-zero-redeploy (silent "we thought we shipped but
didn't")
The `redeploy-staging` job computes the redeploy set as the intersection
of the
build matrix and the build-success set. This job **only runs when
`aggregate-build-results.outputs.any_success == 'true'`** (job-level
`if:`
guard). So if that intersection comes back **EMPTY**, it does NOT mean
"nothing
to deploy" — it means at least one slot built successfully yet none of
those
successes maps back to a matrix `dispatch_name`. That's a
`dispatch_name`↔
`service` contract skew (the aggregator's `service` values and the
matrix's
`dispatch_name` values drifted apart).
The old code emitted `services=` (empty) and exited 0 → the build went
**GREEN
while redeploying NOTHING**, silently.
**Fix:** on an empty intersection in this any_success-guaranteed step,
fail loud
(`::error::` + `exit 1`) with a diagnostic naming both sides of the
skew.
The legitimate "nothing changed / nothing succeeded" no-op paths are
guarded at
the **job level** (`has_changes=='true' && any_success=='true'`), so the
fixed
step never runs there — no false-red.
### 2. Starter build failures had no alert surface (invisible failures)
The `notify` job's `needs` (and its `if: failure()`) omitted
`detect-starter-changes` and `build-starters`, and `build-starters`
wrote no
per-slot build-result artifact. So a **failed starter image build
produced NO
Slack alert and NO PR comment** — it shipped silently.
**Fix:**
- Added `detect-starter-changes` + `build-starters` to `notify.needs` so
`if: failure()` sees a starter build failure → Slack alert + PR comment.
- Gave `build-starters` a per-slot build-result artifact **mirroring the
main
`build` matrix** (same `{service,status}` shape, `cancelled→skipped`
normalization, `if: always()`, `if-no-files-found: error`), using a
**distinct `starter-build-result-*` prefix** so it never matches the
aggregator's `build-result-*` download pattern (starters must not
pollute the
showcase redeploy set keyed by `dispatch_name`).
### Red / Green
**Finding #1** — extracted the step's shell/jq logic and drove it with
synthetic
inputs:
RED (pre-fix), any_success=true + empty intersection:
```
No services in matrix ∩ success-set — skipping redeploy.
Computed services CSV (matrix ∩ build-success):
EXIT=0 # $GITHUB_OUTPUT: services= -> silent pass, redeploys NOTHING
```
GREEN (post-fix), same inputs:
```
::error::Build succeeded (any_success=true) but matrix ∩ success-set is EMPTY — dispatch_name/service contract skew; nothing would be redeployed.
Successful build service values: ["shell-RENAMED","mastra-RENAMED"]
Scheduled matrix dispatch_name values: ["shell","mastra"]
EXIT=1 # fails loud
```
No-regression: non-empty intersection → `EXIT=0 ; services=shell`. The
nothing-changed/nothing-succeeded paths are skipped at the job level
(never
reach the step) → no false-red.
**Finding #2** — modeled `if: failure()` (fires iff any `needs` job
result is
`failure`):
```
BEFORE (starters NOT in needs), starter=failure -> notify fires = False (INVISIBLE, the bug)
AFTER (starters IN needs), starter=failure -> notify fires = True (FIXED)
AFTER no-regression, starters=skipped, all green -> notify fires = False (quiet)
```
### Validation
- `python3 yaml.safe_load` parses OK.
- `actionlint`: only pre-existing findings remain (matrix jq SC2086 +
the known
`depot-ubuntu-24.04-4` runner-label warning); no new errors in edited
regions.
- `yamllint`: only pre-existing line-length/document-start/truthy
warnings.
### Scope
Touches **only** `.github/workflows/showcase_build.yml`, and only these
two
concerns. Does NOT touch the `shell_dashboard` paths-filter region (PR
#5955's
domain), nor the other backlog debt (false-root-cause comment,
double-alert,
check-lockfile guard). Self-contained; not stacked on #5955.
Two pre-existing silent-failure gaps in the showcase build/deploy/notify
pipeline (surfaced in code review):
1. Green-but-zero-redeploy: the redeploy-staging job computes the redeploy
set as (build matrix ∩ build-success). This job only runs when
any_success=='true', so an EMPTY intersection means builds succeeded but
none maps to a matrix dispatch_name — a dispatch_name/service contract
skew. The old code emitted an empty services= and exited 0, going GREEN
while redeploying nothing. Now it fails loud with a diagnostic naming both
sides of the skew. The legitimate nothing-changed/nothing-succeeded no-ops
stay guarded at the job level, so they are unaffected.
2. Starter-failure-invisible: the notify job's needs omitted build-starters,
so a failed starter image build produced no Slack alert and no PR comment.
Added detect-starter-changes + build-starters to notify.needs, and gave
build-starters a per-slot build-result artifact mirroring the main build
matrix (distinct starter-build-result-* prefix so it never pollutes the
showcase aggregator's build-result-* set).
PR #5952 (9a8cf615) added explicit `.js` extensions to the relative imports
inside the harness's shared cell-model fold
(showcase/harness/src/shared/cell-model/{cell-model,live-status,staleness}.ts)
— REQUIRED for the harness's pure-Node-ESM runtime and correct as-is.
But the dashboard re-exports that fold via shims
(showcase/shell-dashboard/src/lib/{cell-model,live-status,staleness,format-ts}.ts
`export * from "../../../harness/src/shared/cell-model/*"`), pulling the fold
into the dashboard's `next build`. `export *` does not rewrite the fold's
INTERNAL `.js` edges, and the dashboard's empty next.config.ts had no
extensionAlias, so webpack resolved `./live-status.js` literally, found only
the `.ts` source, and failed:
Module not found: Can't resolve './live-status.js'
Module not found: Can't resolve './staleness.js'
Module not found: Can't resolve './format-ts.js'
> Build failed because of webpack errors
Two-part fix (one coherent subject):
1. Resolution: add `webpack.resolve.extensionAlias` to
showcase/shell-dashboard/next.config.ts so `.js`/`.mjs` specifiers resolve
to `.ts`/`.tsx`/`.mts` sources — the bundler complement to TS NodeNext's
`.js`-import convention. Covers the `next build` (webpack) path CI uses.
The harness fold's `.js` imports are left untouched (they are correct).
2. CI gap: the dashboard build did not run on #5952 because the build matrix
is path-filtered and #5952 only touched `showcase/harness/**`, which
selects `showcase_harness` but not `shell_dashboard`. Add
`showcase/harness/src/shared/**` to the `shell_dashboard` paths-filter so
any change to the shared fold the dashboard compiles in also selects the
dashboard build — a fold change can never again ship an unbuilt dashboard.
Local red-green proof:
- RED (main, before fix): `next build` in showcase/shell-dashboard emitted the
4 fold-resolve errors above.
- GREEN (after extensionAlias): same build → 0 fold-resolve errors; the fold
resolves. Remaining `@/data/*.json` errors are the prebuild-generated files
(generate-registry/probe-docs) skipped in the local repro, produced in CI's
Docker build — unrelated to this fix.
The boot-smoke previously failed only when the thrown error carried a
module-RESOLUTION code (ERR_MODULE_NOT_FOUND + siblings, walked through
the cause chain / AggregateError members) and passed everything else.
That defaults-to-pass on module-EVALUATION crashes — a top-level throw,
an await-rejection, a bad named binding, or a SyntaxError — none of which
carry a resolution code, so a real boot-crashing regression of that class
would ship green.
The smoke runs `node -e "import('./dist/orchestrator.js')"` with
process.argv[1] UNSET, so bootFleet() (the env/PocketBase validation that
legitimately throws) never runs — only the module graph is linked and
evaluated. A clean build therefore loads with no thrown error, so ANY
error thrown by import() here is a boot regression and must fail the gate.
Now: any rejection -> BOOT_FAIL / exit 1 (resolution AND
evaluation/link/binding/syntax/top-level-throw). Successful load ->
BOOT_OK / exit 0. The collectErrorCodes cause/AggregateError walk is
retained ONLY to label the failure ("module-resolution failure" vs
"boot/evaluation failure") — both exit 1, richer diagnostics preserved.
Kept process.exit(0) on success, the timeout 120s wrapper, and
timeout-minutes: 5 (a hang still fails).
No-false-red proof: built the real harness dist and ran the strict guard
against the real dist/orchestrator.js under node -e (argv[1] unset) —
BOOT_OK, exit 0, ~0.28s, no hang, confirming a clean graph loads without
throwing and the strict guard does not false-red real CI.
The boot-smoke gate classified pass/fail using only the top-level `e.code`.
A module-resolution error that arrives WRAPPED — nested in `e.cause`
(possibly a chain), bundled inside an `AggregateError` (`e.errors[]`), or
rethrown without preserving `.code` at the top — showed no code to the
`MODULE_RESOLUTION_CODES.has(e.code)` check and was misclassified as
BOOT_OK, defeating the gate.
Add a `collectErrorCodes` helper that gathers every code reachable from
the thrown error: the error itself, its cause chain (recursively), and any
AggregateError members (recursively), with a depth cap to bound cause
cycles. If ANY collected code is a module-resolution code -> BOOT_FAIL /
exit 1. Purely additive to the FAIL set: direct top-level codes still
redden, and a benign non-resolution runtime error (e.g. the
`HARNESS_ROLE must be set` guard, which carries no such code anywhere)
still passes as BOOT_OK / exit 0. The `timeout 120s` wrapper,
`timeout-minutes: 5`, and success/expected-error `process.exit(0)` are
unchanged.
Local red-green (classifier extracted to a temp file, driven against
synthetic errors):
- RED (top-level-only): wrapped cause -> BOOT_OK exit 0 (swallowed);
AggregateError member -> BOOT_OK exit 0 (swallowed).
- GREEN (hardened): wrapped -> exit 1; aggregate -> exit 1; direct
ERR_MODULE_NOT_FOUND -> still exit 1; benign ERR_INVALID_ARG_TYPE and
HARNESS_ROLE error -> BOOT_OK exit 0; real built dist/orchestrator.js ->
BOOT_OK exit 0 in <200ms (prompt exit, no hang).
The boot-smoke step only treated ERR_MODULE_NOT_FOUND as failure, so other
module-resolution regressions (ERR_UNSUPPORTED_DIR_IMPORT,
ERR_PACKAGE_PATH_NOT_EXPORTED, ERR_UNKNOWN_FILE_EXTENSION,
ERR_INVALID_MODULE_SPECIFIER) were swallowed as BOOT_OK/exit 0 — the very
class of bug this gate exists to catch could slip through. It also had no
process.exit(0) on the success/expected-error paths and no bounded timeout,
so a future open handle at import time could hang node -e to the job's
25-minute ceiling.
- Broaden the failure condition to a MODULE_RESOLUTION_CODES set (any of the
five codes => BOOT_FAIL, exit 1). Non-module-resolution runtime errors
(e.g. the HARNESS_ROLE env guard, no such code) stay BOOT_OK/exit 0.
- Add explicit process.exit(0) on both the success and expected-error paths.
- Wrap the node invocation in `timeout 120s` (non-zero on timeout => step
fails) and add step-level timeout-minutes: 5.
Red-green proof (extracted guard logic vs synthetic modules): current logic
passes ERR_UNSUPPORTED_DIR_IMPORT / ERR_PACKAGE_PATH_NOT_EXPORTED at exit 0
(RED gap); hardened logic fails all five codes at exit 1, keeps benign
runtime error at exit 0, and against the real built dist/orchestrator.js
reports BOOT_OK and exits promptly (779ms, no hang).
CI missed the extensionless-import regression because tsc (bundler
resolution), vitest, and tsx all resolve extensionless relative
specifiers fine — no existing step ever ran the real node dist module
graph, which is what the container actually does at boot.
Add a boot-smoke to the Validate Showcase job (already gated on
showcase/harness/**): after building the harness dist, load
dist/orchestrator.js via a node import() and fail hard on
ERR_MODULE_NOT_FOUND. A later runtime error from missing env/PocketBase
is expected and passes — only a module-resolution failure reddens the
build. Verified red-green: the guard exits 1 on the pre-fix
extensionless imports and 0 once the .js extensions are added.
Round-3 CR fixes for the LGT persistence-disable preload.
HIGH-1: after installing the fs-write patches, import the node:fs/promises
namespace and assert each patched member is identity-equal to the installed
function; throw (fail boot) naming any mismatched member. Catches the
load-order case where fs/promises was linked before the reassignment and the
namespace snapshotted the original fn (silent bypass -> disk-growth recurrence).
HIGH-2: make the real-package behavioral test non-skippable under
LGT_REQUIRE_BEHAVIORAL=1 (missing runtime fails, not skips), and wire the
python-unit-tests job to set up Node, npm install the agent deps, and run the
langgraph-typescript pytest with that flag so a green check proves interception.
LOW: tolerant writer-shape guard regex (quote/whitespace/alias agnostic; still
trips on a named-import switch); read-only open/openSync reject ENOENT for
suppressed paths (write-intent still no-ops); mkdir recursive returns the
topmost-created dir per the real fs contract.
Adds a HIGH-1 guard-fires regression test.
Four workflow steps installed CLI tools ad-hoc via `npm install -g`, which
zizmor's `adhoc-packages` audit flags (install outside a lockfile). Replace
each with a lockfile-managed or pinned-action install, preserving behavior:
- aimock (test_integration-docs, test_e2e-showcase-on-demand): invoke the
workspace-pinned @copilotkit/aimock `llmock` bin from the frozen lockfile
(already a dep of @copilotkit/showcase-scripts) instead of `npm install -g`.
Kept lockfile-devDep rather than the CopilotKit/aimock composite action:
the action wraps the newer config-only `aimock` CLI and can't do the
multi-`--fixtures` / `--validate-on-load` / `/__aimock/health` invocation
these jobs need.
- claude-code (social_copy-generator): pin @anthropic-ai/claude-code as a root
devDependency, install from the frozen lockfile, invoke via its documented
cli-wrapper.cjs entrypoint. Kept lockfile-devDep rather than
anthropics/claude-code-action: the job uses claude as a scripted `-p` CLI,
not PR/issue automation.
- oxfmt (static_quality): already a root devDependency; install from the frozen
lockfile and put node_modules/.bin on PATH instead of `npm install -g`.
- ruff (static_quality): switch `pipx install` to the pinned official
astral-sh/ruff-action@278981a (v4.1.0) with the same 0.15.13 version.
zizmor --min-severity low --config .github/zizmor.yml .github/workflows:
before: exit 12, 4 adhoc-packages findings
after: exit 0, 0 adhoc-packages findings, 0 unpinned-uses (no findings)
Mike Ryan