The verify-image-refs gate (verify-railway-image-refs.ts) failed because the
live Railway service `showcase-crewai-conversational-flows` had no entry in the
SERVICES map, tripping the Railway->SSOT drift check (1 untracked service).
Add the service as a STAGING-ONLY entry: the live Railway service currently has
a serviceInstance in staging only (no prod instance is provisioned), so the
env-map schema declares only the env that exists. All values (serviceId,
staging instanceId, domain, healthcheckPath) are read verbatim from the live
Railway API, not guessed. ciBuilt:false because the integration is wired only
into the PR-check build (showcase_build_check.yml), not showcase_build.yml's
ALL_SERVICES matrix, so it stays out of CI_BUILT_SERVICES.
Regenerate railway-envs.generated.json and update the count/coverage
assertions in the affected tests (SSOT now has 42 services; findMissingServices
is intentionally asymmetric for the staging-only entry: 41 prod, 42 staging).
The auto-format bot reflowed both captured registry payloads. Harmless this
time — oxfmt only rewraps arrays, it does not reorder keys — but the whole
value of these two files is that they are verbatim `imagetools inspect`
output. A formatter in the loop means the next re-capture churns, and the
"this is exactly what the registry emits" claim stops being checkable.
Exempt the fixture directory in .oxfmtrc.json, restore the captured bytes,
and say so in the provenance comment so nobody tidies them by hand.
The previous suite was 37/37 green against a workflow that could not have
run at all: the guard step had no `packages: write` and no GHCR login, so
every `imagetools create` would have 401'd. Asserting that a step EXISTS
proves nothing about whether it can succeed. Each change below was checked
by reintroducing the defect and confirming the test goes red.
Registry-auth precondition (the miss that let the above ship)
- Both redeploy jobs must declare `packages: write` AND a `docker/login-action`
step for ghcr.io, ordered BEFORE the guard. Dropping either reds the suite.
The intersection, executed rather than restated
- The old test only checked that the string `images=` appears in the compute
step; swapping its jq for the full matrix — so a FAILED build moves
`:latest` — kept it green. The real `changed` shell now RUNS, against the
real ALL_SERVICES matrix read out of the workflow, and the emitted
$GITHUB_OUTPUT is asserted: a failed build is in neither set, and the
`skip_build` slot is in `services` but NOT in `images` (handing it to the
guard fails "manifest unknown" and blocks the redeploy for the whole fleet).
Same treatment for the starter lane.
Fixtures joined to real registry output
- `extractRevisionLabel` was pinned to hand-written payloads that were never
compared with reality. Since `readLatestRevision` maps every failure to
null, and null ADVANCES, a parser that silently never matches yields a
permanently-blind guard with a fully green suite. Both fixtures are now
verbatim `docker buildx imagetools inspect --format '{{json .Image}}'`
output (buildx v0.35.0), unformatted so key order survives: a platform-keyed
multi-arch image carrying the label, and our own `:latest`, which turns out
to be a bare config object with NO labels at all. The label key used in the
injection test is read from the workflow's own `labels:` input, so parser
and producer cannot drift apart silently.
Closed vacuities
- The `:latest` ban read one step's `tags` and was vacuously green on an empty
list. It now covers every tagged step plus hand-rolled `docker push`/`docker
tag`/`imagetools create`, and fails on an empty `tags`.
- The `cancel-in-progress` ban read only top-level config; a job-level
`concurrency` on `build` reproduced the exact harm and stayed green. All
jobs are checked now.
- `already-current` is keyed on the digest, matching the script.
- The unreachable `(null, "ahead")` row is labelled as the defensive
input-space case it is, and a new test pins that the real flow never calls
compare() with an unknown revision.
New coverage: classifyProbeFailure (incl. a REAL execFileSync timeout, and
the two precedence traps — gh's rate-limited 403 is throttling, not auth; a
registry's 404-with-denied is auth, not absent), readFlag, escapeAnnotationData
(a forged `\n::error::` stays inert), isDirectInvocation through a symlink,
the `::error` annotation on fleet failure, and the digest-mismatch advance.
Also: GITHUB_SHA / GITHUB_REPOSITORY are no longer shadowed in the workflow.
The runner exports both and the script reads process.env, so the old
assertions pinned a redundancy rather than a capability — removed together,
as the comment there required. Workflow-reading scaffolding duplicated with
redeploy-guard.test.ts is extracted and the YAML parse memoized.
Restore the 12 Python integration tools/ dirs to symlinks into
shared/python/tools. They had eroded to real, drifting copies via an
accidental stage_shared() leak (commit 534cd1efa7) — the structural root
cause of showcase divergence bugs. Symlinking re-establishes the single
source of truth; content is identical to shared (only render_a2ui naming
and the shared roll_dice/sanitize additions are adopted).
Add showcase/AGENTS.md documenting the 4 iron rules and the single-source
symlink mechanism, plus a validate-shared-symlinks CI guard (shrink-only
baseline) that fails on any NEW erosion.
OSS-451 shipped because nothing linked a demo page's CopilotKit runtimeUrl
to the existence of the /api route it names. The only automatic pre-merge
gate for showcase/** is a Docker build, which compiles a page that
references a non-existent route just fine (runtimeUrl is an unchecked
string) — so the page-404-on-load class was invisible.
Add a static validator (validate-runtime-routes.ts) that, for every SHIPPED
demo (a demo listed in its integration's manifest `features`), asserts its
runtimeUrl resolves to a real route dir under src/app/api. Unshipped /
experimental demos (not in `features`) and not_supported_features are
skipped, so incomplete placeholders don't fail the gate — but promoting one
into `features` immediately starts enforcing it. A baseline file can
grandfather pre-existing violations; the fleet is currently clean (0).
Wire it into a new pre-merge workflow (showcase_validate-wiring.yml) that
runs on every showcase/integrations PR alongside the build check. Add it to
branch-protection required checks to make it blocking.
Regression test proves it flags the exact OSS-451 shape (shipped demo,
missing route) while passing existing/base routes and skipping unshipped.
Verified: npm run validate-routes -> clean fleet-wide; removing the 3
OSS-451 routes -> flags exactly those 3; full showcase/scripts vitest suite
(2151 tests) green.
Refs OSS-451
The prod `harness-workers` fleet worker (serviceId
c2aa8a0b-350e-4b76-8541-3012dfac41d0, instance
7c48ee43-6df4-457b-b977-10f1f1ac1680) runs the shared `showcase-harness`
image (`imageOf: "harness"`) but had NO `prod` env entry in the
railway-envs SSOT. `expandImageConsumers` is env-aware — a consumer only
joins an env's redeploy scope if it declares that env — so a rebuilt
`showcase-harness:latest` bounced the prod control-plane but SILENTLY
SKIPPED the prod worker, leaving it pinned to a stale 2026-06-19 image.
That stale worker image carries a 1-demo `registry.json` for
`ms-agent-harness-dotnet` (only `beautiful-chat`), so the hourly
`e2e_demos` driver running on it produced only 1 of 39 `e2e:` rows in
prod PocketBase. The other 38 feature rows were absent → `resolveD3`
exists=false → `UI` badge omitted → broken D3 rung → D0.
Backfill the live prod worker as a `prod` env entry (real
serviceInstance ID), flip `gateIgnore` off, and set `gateValidated:
true` so the env-aware `imageOf` expansion now pulls the prod worker
into the prod redeploy scope on every `showcase-harness` rebuild.
Regenerate the emitted JSON + golden fixture and update the two
gate-count assertions accordingly.
The showcase-strands-typescript integration was staging-only: it had no
production Railway serviceInstance, so the prod D6 dashboard column showed
a uniform false-red (every cell errorClass=goto-error, backendUrl="") —
the probe navigated a bare relative path because the harness had no prod
health record / backendUrl to discover.
Provisions the prod serviceInstance (8a50728e-6119-43c4-b59c-d9535b6717a4,
domain showcase-strands-typescript-production.up.railway.app, healthcheck
/api/health, image pinned to the GHCR @sha256 digest, OPENAI_BASE_URL at
prod aimock) and brings the SSOT to the dual-env showcase-strands shape:
- railway-envs.ts: add the prod env entry with the real instanceId,
gateValidated:true, drop gateIgnore, remove the legacyJsonCompat
prod-domain placeholder.
- railway-envs.generated.json: regenerated (prod instanceId/domain, probe.prod
true, prod healthcheck; moved into the promote closure, tier 2).
- railway-envs.golden.json: regenerated to include the new prod (service,env)
pair (intentional behavior change, not a refactor regression).
- showcase_promote.yml: dropdown regenerated to list strands-typescript.
- verify-railway-image-refs.test.ts / redeploy-env.test.ts: update the
gateValidated/scope counts (39->40 gate targets, prod default 38->39) and the
now-stale staging-only comments.
RED->GREEN (live prod): BEFORE /api/health 404, prod PocketBase
health:strands-typescript totalItems:0, the 3 named D6 cells all
errorClass=goto-error backendUrl="". AFTER /api/health 200, prod PocketBase
health:strands-typescript present (status:200, valid url),
verify-railway-image-refs OK 80 instances.
Wire the strands-typescript showcase integration for staging deployment,
mirroring how the Python strands integration is deployed.
- manifest: flip deployed: true so the shell lists it in the integration menu
- railway-envs.ts: add showcase-strands-typescript SSOT entry (staging-only
for now: prod instance not yet provisioned, so it omits the prod env and is
gateIgnore'd until promoted dual-env); regenerate railway-envs.generated.json
- showcase_build.yml + showcase_build_check.yml: add the strands-typescript
build matrix entry, change-detection filter, and dispatch option (railway_id
is the new Railway service id)
- golden fixture + image-ref-gate inventory tests updated for the new service
Railway staging service showcase-strands-typescript provisioned
(showcase-strands-typescript-staging.up.railway.app, health /api/health,
OpenAI-via-aimock env). Prod is added later via the promote pipeline.
The pool-fleet migration is complete: the control-plane harness plus the
prod workers (deployed 2026-06-19, HARNESS_ROLE=worker, pool count 2) now
cover every probe dimension the interim `harness-legacy` fleet-migration
bridge was holding live, so `harness-legacy` is dead config.
This is the code-side cleanup only:
- Remove the `harness-legacy` entry from the railway-envs SSOT and the
now-dead `key === "harness-legacy"` special-case in computePromoteClosure.
- Regenerate railway-envs.generated.json (41 -> 40 services).
- Drop harness-legacy from the golden snapshot, the gateIgnore expectation
sets, the promote-notify fixtures, and the redeploy-env doc comments;
update the service-count assertions (41 -> 40).
- Fix the stale "STAGING-ONLY" harness-workers comment: prod workers are
live on Railway, though this SSOT entry still models the staging
instance only (no prod env backfilled here yet).
The live Railway `harness-legacy` service is torn down separately as a
follow-up infra step.
Flip the 12 starter-* services to behave like every other managed
showcase service: always-on and staging-probed.
- railway-envs SSOT: staging probe false -> true for all 12 starters
(prod was already on); refresh the stale "staging probe OFF" /
fail-loud-placeholder doc-comments to reflect baseline-driver verification.
- provision-starter-fleet: sleepApplication true -> false so the
provisioner creates always-on (not sleepable) Railway instances; update
the "the whole point — sleepable" header/inline comments + log lines.
- Regenerate railway-envs.generated.json (emit-railway-envs-json.ts) and
the golden fixture; update the SSOT probe + provisioner sleep assertions.
The pool-fleet worker's Railway service is named `harness-workers` (PLURAL),
but the SSOT keyed it `showcase-harness-worker` (singular). The image-ref gate
matches SSOT keys to Railway service names verbatim, so the gate reported
`harness-workers` as an untracked Railway service AND the stale singular key
matched nothing. Rename the SSOT key (and every test/fixture reference) to the
exact Railway name `harness-workers`.
It stays the staging-only, domainless, probe-disabled worker that runs the
shared `showcase-harness` image: serviceId c2aa8a0b-…, staging instance
362c1e37-…, ciBuilt:false, gateIgnore:true, no build slot (so no dispatchName),
single `staging` env with no domain. Add a focused test pinning that shape.
Counts are unchanged (29 services / 26 CI_BUILT) — this is a rename, not an
addition; both harness workers already existed on main.
Verified LOCALLY against Railway: verify-railway-image-refs reports
`54 env-scoped instances verified (2 skipped)` — 0 violations, 0 missing, 0
untracked (harness-workers reconciled, harness-workers + harness-legacy the 2
gateIgnore'd skips). emit --check zero drift, Ruby parity green (borrowed
.up.railway.app host is parity-excluded), full scripts suite + typecheck green.
Replace ServiceEntry's parallel prodInstanceId/stagingInstanceId/domains/probe/
repoNameOverride fields with a single environments: Record<string, {instanceId,
domain?, probe?, repoName?}> map plus a hoisted env-independent probeDriver.
EnvName becomes an open string backed by an ENV_ID_BY_NAME registry so
accessors resolve arbitrary env names; a single-env service (the staging-only
showcase-harness-worker) now simply omits the absent env instead of carrying a
placeholder ID/borrowed host.
Accessors instanceIdFor/domainFor/repoNameFor index environments[env]
(domainFor still throws on missing/scheme); add envsFor(name),
serviceEnvPairs(), and probeEnabled(name, env). Generalize the image-ref gate
(iterate each service's declared environments, resolve env-id via the
registry, sum/iterate missingByEnv over registry env names) and verify-deploy's
host->env reverse-map (envForTarget iterates environments).
Pure TS-internal: emit-railway-envs-json.ts projects the env-map back onto the
FROZEN legacy JSON shape (prodInstanceId/stagingInstanceId/domains/probe/
repoNameOverride) via a documented legacyJsonCompat shim for the two domainless
harness workers, so railway-envs.generated.json stays byte-identical and Ruby
(bin/railway) + workflow jq + the parity test are untouched. Verified: emit
--check zero drift, Ruby test_expected_domains_parity green, golden snapshot
toEqual proves byte-identical resolution for every real (service, env) pair,
full scripts vitest suite green, showcase scripts typecheck clean.
Serializes the fully-resolved {service -> env -> {instanceId, domain, probe,
driver, repoName}} projection for all 29 services x 2 envs via the public
accessors (instanceIdFor/domainFor/repoNameFor) + per-entry probe config,
frozen as a fixture. This is the behavior-preservation guard for the
forthcoming env-map (Option C) refactor: resolved values must stay
byte-identical before and after.
Delete the pre-generated starters/ directory tree (previously
synced from packages/ by generate-starters.ts). Add
extract-starter.ts which produces a starter tarball on demand
from any integration. Move shared starter template files to
showcase/shared/starter-template/.
Extract drift-comparison logic out of validate-pins.ts into
validate-pins-core.ts (pure module, CLI re-exports + remains thin
orchestrator) and extract the formatter out of
redirect-decommission-report.ts into redirect-decommission-core.ts.
Add vitest coverage for both cores with baseline fixtures so
showcase-ops ProbeDrivers can reuse the same logic without forking.
Drop legacy generate-status.ts — superseded by showcase-ops live
status feed.
The scrub and #4084 touched the same surface: #4084 re-added an `open:`
generative_ui profile listing `open-gen-ui`/`open-gen-ui-advanced`, and
re-added both features to `constrained-explicit.allowed`. Extending the
branch's scrub to both re-additions keeps the semantic consistent with
the schema (which already dropped `open` from the approaches enum).
- `showcase/shared/constraints.yaml`: drop `open-gen-ui` +
`open-gen-ui-advanced` from `constrained-explicit.allowed`; drop main's
re-added `open:` profile entirely.
- `showcase/packages/langgraph-python/manifest.yaml`: drop the now-orphan
`open-gen-ui` + `open-gen-ui-advanced` feature and demo entries
(validator confirmed they had no allowed approach left).
- Regenerated `showcase/shell/src/data/registry.json` + sibling
`shell-docs`/`shell-dojo` registries and `constraints.json` via
`pnpm --dir showcase/scripts generate-registry`. All 17 integrations
validate.
`feature-registry.json` intentionally still defines both features — the
original scrub commits (2b996c54d, 27f886e59) left it untouched, so the
demo source files on disk also stay. Follow-up deletion if desired is
out of scope for this merge.
Audits each showcase package's manifest-declared demos for matching
spec (tests/e2e/*.spec.ts) and qa/*.md coverage, and enforces a
monotonic demo-count baseline via fail-baseline.json.
Key design:
- PackageIssue tagged union (13 variants) cleanly separates MUST
errors from warnings; deriveMessage is the single renderer so new
variants cannot emit mismatched prose.
- ProbeResult tagged union (missing | ok | unreadable) driven by
statSync + errno inspection, distinguishing ENOENT from EACCES and
surfacing ENOTDIR as a misconfiguration rather than a silent miss.
- runParityImpl isolates each slug's audit in try/catch; a crash in
one slug surfaces as a crashed PackageIssue variant and forces
EXIT_INTERNAL without aborting siblings.
- runParity never throws for content errors. InvalidBaselineError
covers coerceBaseline failures; unknown errors route to
EXIT_INTERNAL via formatErrorChain (walks .cause with cycle
guard + depth cap).
- parseMainArgs rejects unrecognised flags and duplicate --baseline
with EXIT_INVALID_INPUT (2), mirroring audit.ts parseArgs
discipline.
- BASELINE_DEMO_COUNT default must match
fail-baseline.json.baselineDemoCount; enforced by
__tests__/baseline-sync.test.ts.
- Exit codes: 0 ok, 1 should-warnings-only, 2 invalid-input, 3
unreadable, 4 internal, 5 must-failure.
Tests cover every PackageIssue variant, every exit code
(in-process + subprocess), per-slug crash isolation, EACCES routing,
ENOTDIR classification, cascade suppression when tests/e2e or qa
dirs are unreadable, and baseline coercion edge cases (leading
zeros, negative, float, hex, non-numeric).
Compares framework dependency pins across showcase/packages/*/ and
the corresponding dojo examples/integrations/* trees, flagging drift
between the two and rejecting non-exact specs on the showcase side.
Key design:
- Parses package.json, requirements.txt, and pyproject.toml
(including Poetry's [tool.poetry.dependencies] and PEP 621
[project.dependencies] / optional-dependencies). Separate jsDeps
and pythonDeps maps prevent cross-ecosystem name collisions.
- isExactSpec enforces exact-version pins per ecosystem (npm: no
operators, workspace refs, or ranges; Python: ==X / ===X / ~=X
with PEP 440 body). Symmetric rejection of bare MAJOR-only forms.
- parseRequirementsTxt and parsePyprojectToml thin wrappers throw
when the detailed form produced skipped[] or dropped[] entries,
preventing silent data loss in simpler callers.
- canonicalizeDepMap canonicalises names per PEP 503 and surfaces
same-file collisions with differing specs as warnings.
- First-writer-wins at both file and package levels.
- UnreadableInputError carries an optional partialReport so an
infra failure mid-slug-loop preserves already-collected drift
findings for other slugs.
- Exit codes: 0 ok, 1 drift, 2 internal, 3 unreadable.
fail-baseline.json is the single source of truth for the CI ratchet
(validatePinsFailCount + validatePinsFailHash) and the demo-count
floor (baselineDemoCount, cross-checked against validate-parity.ts
in a dedicated sync test).
Test coverage spans every parser variant, EACCES routing via chmod
probe + fs spies, exit-code taxonomy subprocess tests, partial-report
preservation on mid-loop infra throws, and Poetry/PEP 503 edge cases
via committed fixture files under __tests__/fixtures/pins/.
Mark