Layer (c) deploy-rollover config for the harness-workers fleet — pure
Railway config, no custom rolling-restart code. Declares overlapSeconds=45
(capacity floor: old deployment serves until new workers register+claim, so
no staleness dip) and drainingSeconds=180 (SIGTERM->SIGKILL window >=
PLATFORM_STOP_GRACE_MS so the layer-(b) 3s+90s composed worker-drain finishes)
for both prod and staging in the railway-envs SSOT, regenerates the JSON
snapshot, and extends the harness-workers drift gate so CI fails if either
field drifts between SSOT and snapshot. Documents both knobs, their rationale
(incl. that Railway's draining default is 0s = immediate SIGKILL), the
composition with drain layers a+b, and how to apply them (GraphQL
serviceInstanceUpdate / dashboard) in RAILWAY.md.
The harness-workers SSOT modeled only the top-level numReplicas, whose drift
gate watched a field that does not drive the live replica count. harness-workers
is single-region (us-west2); Railway derives the live count from
multiRegionConfig.us-west2.numReplicas.
- WorkerProvisioning gains effectiveReplicas (= multiRegionConfig.us-west2.
numReplicas), the authoritative field the drift gate now asserts. Top-level
numReplicas is retained as a documented mirror.
- Declared values reflect current reconciled reality (verified live via the
Railway GraphQL environment.config staged-config read): prod and staging both
at effectiveReplicas=6 (parity achieved by B-reconcile scaling prod 3 -> 6 in
both the top-level field and multiRegionConfig). BROWSER_POOL_MAX_CONTEXTS is
40 on both envs (verified live).
- Regenerated railway-envs.generated.json; drift-gate test asserts
effectiveReplicas (RED-GREEN proven). RAILWAY.md documents multiRegionConfig
as the effective knob and the achieved parity.
Add WorkerProvisioning interface and workerProvisioning field to the
harness-workers ServiceEntry in railway-envs.ts. Declares current live
reality: prod=3 replicas, staging=6 replicas, BROWSER_POOL_MAX_CONTEXTS=40
per worker (both envs).
Worker model: 1-worker-per-replica (Railway runs one process per
container, keyed on HOSTNAME). HARNESS_POOL_COUNT is informational
only — not a fork factor. Authoritative concurrency knob per worker is
BROWSER_POOL_MAX_CONTEXTS.
Comments flag: staging config-field drift (Railway field=2, live=6) as
a follow-up item; and the prod/staging parity decision (prod=3 vs
staging=6) as deliberately deferred.
Extends emit-railway-envs-json.ts to emit workerProvisioning into the
generated JSON snapshot. Adds drift-gate test
(harness-workers-provisioning.test.ts) that fails if SSOT numReplicas
diverges from the committed JSON snapshot — no live Railway API calls.
Red-green-red-green verified locally.
Updates RAILWAY.md with the 1-worker-per-replica model, declared
values, manual apply procedure, and drift-gate reference.
The existing tooling is verify-only for numReplicas; applying a replica
count change to Railway remains a manual operation (Railway Dashboard or
GraphQL API).
Adds a "Promoting a Staging-Only Integration to Production" section to
showcase/RAILWAY.md — the staging-first -> promote-later procedure that was
undocumented and caused the strands-typescript D6 false-red (PR #5705).
The procedure previously survived only as a comment inside the SSOT
(railway-envs.ts); there was no human-facing SOP. The new section is a
start-to-finish checklist grounded in the PR #5705 worked example:
- When it applies (gateValidated:false, gateIgnore:true, staging-only env map,
legacyJsonCompat prod placeholder).
- The critical gotcha up front: the promote pipeline (showcase_promote.yml /
bin/railway promote) only moves digests to a prod service that ALREADY
exists; it does NOT provision a new prod serviceInstance. Until that instance
exists, D6 false-reds the whole column (404 -> empty backendUrl ->
goto-error on every cell).
- Ordered steps: provision the prod serviceInstance out-of-band
(environmentStageChanges + environmentPatchCommitStaged, mirroring a peer
prod TS service); edit the SSOT (add prod env block, gateValidated:true, drop
gateIgnore, remove legacyJsonCompat); regenerate derived artifacts
(emit-railway-envs-json.ts, golden fixture, sync-promote-service-options.ts)
and run the gate (verify-railway-image-refs.ts + vitest); prod secrets via
the prod env var set / aimock (no inline secrets); verify GREEN (/api/health
200, prod PocketBase health record, D6 flips on the next hourly :40 tick).
Cross-links INTEGRATION-CHECKLIST.md §B (single-shot bring-up) both ways.
Leaves a precise TODO that §B.3 still names the stale showcase_deploy.yml for
the build matrix (the RAILWAY.md references were already corrected on main).
Re-tier the showcase docs tree to be an agent entry point: README.md
opens with a 'when X, see Y' fanout table that routes to the right
procedural doc; each procedural doc gets a one-line tagline answering
'what does this answer'.
Consolidation:
- DELETE showcase/RUNBOOK.md — operational content merged into DEBUGGING.md
(Integration Patterns, Docker Compose Environment, Production Debugging,
Anti-Patterns, Aimock Fixture Deployment, Dev Iteration Speed). The
--isolate mechanics + CLI rules were already duplicated in DEBUGGING.md.
- DELETE showcase/QA-COVERAGE.md — per-demo coverage matrix + starter hero
matrix + probe depth + infra locations + gaps folded into TESTING.md as
the 'Per-Demo Coverage Matrix' section.
Taglines added (no behavioral change to content): TESTING.md, DEBUGGING.md,
GOTCHAS.md, INTEGRATION-CHECKLIST.md, STYLING-GUIDE.md, FRONTEND-STRATEGY.md,
RAILWAY.md, bin/README.md, aimock/README.md, aimock/RAILWAY.md,
harness/README.md, harness/docs/rotation-drill.md.
Cross-link fixups: FRONTEND-STRATEGY.md (was QA-COVERAGE.md →
TESTING.md#per-demo-coverage-matrix), TESTING.md (removed dangling RUNBOOK
companion reference), README.md (rewritten as fanout entry + retained
from-scratch setup + dashboard SOPs below the fanout).
PARITY_NOTES.md × 12 left alone (per-slug context, not redundant).
(cherry picked from commit 75c9d9755c9118c8abc1fa52deda2012b768cab1)
(cherry picked from commit b64189bae0fe2c9e3a5e3ca440013deb4121f23b)
Documents fleet-wide Railway auto-update configuration (all 41 services),
the environmentPatchCommit API for enabling on new services, CI coupling
with explicit redeploy for health-check determinism, and known quirks
around polling frequency and API surface.
Mark