Adds a CI-owned reconcile (every 15m) that compares each staging service's
deployed digest against GHCR :latest and re-runs the staging redeploy for
lagging services, alerting Slack. The run is green only when every in-scope
service is confirmed current; any unconfirmed service (lag, digest error,
dropped redeploy, empty scope, or a thrown redeploy) alerts and exits non-zero.
Exposes per-service redeploy records from redeploy-env for accurate per-service
remediation confirmation.
The strands-typescript SSOT entry is ciBuilt:true but staging-only (prod
instance not yet provisioned). redeploy-env.ts's default scope was the full
CI_BUILT_SERVICES set for BOTH envs, so a staging-only ciBuilt service would
wrongly enter the prod default scope and fail a manual `redeploy-env.ts prod`
(no prod instance). Filter the default scope by env declaration (explicit
--services stays unfiltered, preserving the contract-pin that an operator can
force a named service in an env it does not declare). imageOf expansion was
already env-aware; this extends the same invariant to the base scope.
Update the inventory-lock test counts for the new service (total 40->41,
CI_BUILT 38->39, staging default scope 39->40; prod default scope stays 38 as
the staging-only service is now correctly excluded).
The pool-fleet migration is complete: the control-plane harness plus the
prod workers (deployed 2026-06-19, HARNESS_ROLE=worker, pool count 2) now
cover every probe dimension the interim `harness-legacy` fleet-migration
bridge was holding live, so `harness-legacy` is dead config.
This is the code-side cleanup only:
- Remove the `harness-legacy` entry from the railway-envs SSOT and the
now-dead `key === "harness-legacy"` special-case in computePromoteClosure.
- Regenerate railway-envs.generated.json (41 -> 40 services).
- Drop harness-legacy from the golden snapshot, the gateIgnore expectation
sets, the promote-notify fixtures, and the redeploy-env doc comments;
update the service-count assertions (41 -> 40).
- Fix the stale "STAGING-ONLY" harness-workers comment: prod workers are
live on Railway, though this SSOT entry still models the staging
instance only (no prod env backfilled here yet).
The live Railway `harness-legacy` service is torn down separately as a
follow-up infra step.
assertEnvRegistryConsistent gains four clauses: (iv) a key present in
both ENV_IDS and ENV_ID_BY_NAME must carry the same env-id (ENV_IDS.prod
drifted to the staging id previously passed every clause while
resolveEnv("prod") silently returned staging); (v) every ENV_IDS env-id
must be carried by a canonical name (was only caught lazily in
resolveEnv); (vi) registry keys must be trim().toLowerCase()-normalized
(resolveEnv lowercases input, so a non-lowercase spelling is registered
but unreachable); (vii) no registry key may be an Object.prototype
property name.
expandImageConsumers' per-entry env skip-check becomes an own-property
test (Object.hasOwn) for uniformity with every other lookup in the file.
REDEPLOY_SUMMARY_JSON is trimmed before the set-but-empty branch so a
whitespace-only value hits the loud warn path instead of attempting a
JSON write against a garbage path.
- railway-envs: route envsFor/instanceIdFor/domainFor/probeEnabled (and
repoNameFor) through shared getEntry/getEnvCfg own-property helpers so
inherited Object.prototype keys on either axis produce the curated
error (or probeEnabled's contract false) instead of raw TypeErrors,
silent undefined, or a spurious probe=true.
- railway-envs: two new module-load invariants (synthetic-map
injectable): assertEnvRegistryConsistent (per-service env keys are
registered canonical names; ENV_ID_BY_NAME env-ids unique; every
canonical name has an ENV_IDS spelling) and
assertServiceAndInstanceIdsUnique (serviceId unique per entry,
instanceId globally unique).
- redeploy-env: invert the exit-code policy to fail-loud by default —
any env except the documented staging carve-out exits non-zero on
per-service failure (a future preview/canary env inherits fatal
semantics instead of silently swallowing failures).
- redeploy-env: sanitize per-service THROWN error messages through
sanitizeErrorBody before recording; flatten bare \r in the summary
table escape.
- redeploy-env: warn on set-but-empty REDEPLOY_SUMMARY_JSON; reject
flag-like --services CSV parts (both forms) and a flag-like first
argument (missing env); derive usage env lists from ENV_IDS.
- tests: prototype-key sweep across all accessors, invariant
positive/negative coverage, third-env exit-code pin, sanitization and
CLI-guard coverage, makeLiveRedeploy !res.ok and non-true mutation
branches.
Confirmation-CR bucket-(a) fixes for PR #5353 — one coherent concern:
env-name resolution has exactly ONE authority (the ENV_IDS /
ENV_ID_BY_NAME registries) and SSOT accessors fail loud instead of
silently returning wrong values.
- runRedeploy: resolve envId via ENV_ID_BY_NAME with an Object.hasOwn
guard + fail-loud throw listing the registered envs. Removes the
hardcoded `prod`/`staging` pair check and PRODUCTION/STAGING ternary
that contradicted the SSOT's documented open-env contract ("a new env
needs only a registry entry"); the registry lookup subsumes it.
- resolveEnv: derive resolution entirely from the registries (ENV_IDS
spellings -> env-id -> canonical ENV_ID_BY_NAME name) instead of its
own hardcoded synonym chain. Behavior identical for
prod/production/staging; still throws on unknowns, and now also
throws on a mis-wired registry (a spelling whose env-id has no
canonical name).
- serviceForDispatchName: fix the docstring's false "CI-built service"
claim — it does no ciBuilt filtering and tests pin the unfiltered
behavior (the non-CI-built webhooks resolves).
- repoNameFor: fail loud (consistent with instanceIdFor/domainFor)
instead of silently echoing the service name — the exact
silently-wrong-GHCR-name class this PR's hardening targets. Throws on
unknown service, on an env not registered in ENV_ID_BY_NAME
(unnormalized synonyms like "production"), and on a registered env
the service does not declare. Keeps the documented default (the
service name) for declared envs without an override.
Call-site enumeration confirming nothing relies on the old fallback:
- verify-railway-image-refs.ts:523 — iterates the entry's DECLARED
environments keys, registry-filtered, SSOT-matched service names
- __tests__/railway-envs.golden.test.ts:81 — iterates envsFor(name)
(declared envs only) over real SSOT keys
- railway-envs.test.ts repoNameFor cases — dual-env services,
prod/staging only
- __tests__/verify-railway-image-refs.test.ts:264 — FIVE_NEW keys,
all dual-env
- resolveTargetServices: throw when an explicitly-provided services
list resolves to zero entries (whitespace-only programmatic input)
instead of letting runRedeploy exit 0 having redeployed nothing; the
default undefined -> full CI-built scope is unchanged.
- makeLiveRedeploy: add signal: AbortSignal.timeout(30s) so a hung
Railway API records a per-service FAIL instead of stalling CI, and
pass GraphQL errors[].message through sanitizeErrorBody for
consistency with the HTTP-error path. Exported for direct unit tests.
Red-green: 11 new tests (open-env registry resolution incl. a
runtime-registered hypothetical env, repoNameFor negatives,
empty-resolution throw, abort-signal presence, GraphQL error
sanitization) all failed against the old code; full showcase/scripts
suite green (50 files, 1807 tests) + tsc --noEmit clean.
harness-workers runs the SAME showcase-harness GHCR image as the harness
scheduler but has ciBuilt:false (it has no build slot of its own), and the
CI staging redeploy scope was derived purely from ciBuilt — so a main-merge
rebuild of showcase-harness:latest only bounced the scheduler while the
workers silently kept running the stale image (PR #5352's worker-side fixes
never reached staging).
Model image consumption explicitly in the SSOT instead:
- railway-envs.ts: new optional `imageOf` field on ServiceEntry — the SSOT
key of the ciBuilt service whose image this entry runs. Set
`imageOf: "harness"` on harness-workers. New module-load invariant
`assertImageConsumersValid` (fail-loud, same style as
assertDispatchNamesUnique): imageOf must name an existing SSOT key, the
target must be ciBuilt, and the consumer itself must not be ciBuilt.
- redeploy-env.ts: new `expandImageConsumers(names, env)` applied inside
runRedeploy — the redeploy set becomes the resolved scope PLUS any
service whose imageOf points at a service already in scope. Env-aware:
a consumer only joins envs it declares, so the staging-only worker
never enters a prod redeploy (prod behavior unchanged).
No workflow change needed: showcase_build.yml keeps passing the
built-and-successful dispatch_names; the script expands them. Gate
behavior (gateIgnore / image-ref gate), the generated JSON
(emit --check passes byte-identical), and the promote dropdown are all
untouched. harness-legacy deliberately gets no imageOf (pinned pre-fleet
digest; must not follow rebuilds).
Adds optional REDEPLOY_SUMMARY_JSON path; when set, redeploy-env writes
a structured per-service record array {service,status,error?}. PR #5093's
exit-code contract is preserved (staging=0, prod=1-on-failure).
Consumed by showcase_deploy.yml to fail the workflow on staging per-service
errors without changing the script's exit semantics. Refs spec §3.
Mark