Hourly watchdog as a scheduled handler on the stateless Worker: queries
the DO GraphQL Analytics datasets (duration GB-s as the hibernation-
defeat detector, rows read/written as runaway detectors, request count),
thresholds derived from the validated workload model and env-overridable
with min(alert,kill) breach gating, Discord alerts in the release-notify
embed shape, and severe duration/rows breaches auto-tripping the kill
switch — KV trip strictly before Discord so a dead webhook can never
disable the brake. GraphQL failure alerts nothing and trips nothing.
Kill switch: KV flag (AUTH_CACHE, control:kill-switch), per-isolate 30s
cache, fail-open, checked before auth so every response — including
auth failures — carries X-Sync-Mode: poll while tripped; WS upgrades
refused before the DO is woken. Clients honor header presence at any
status, treat absence as cleared only on OK, close the socket, suppress
reconnects, and keep polling — product stays complete in poll mode.
Suspended clients now tear the socket down (idle needs no speed layer),
making the documented propagation bound structural.
Canary: two-device trickle script with bounded --flood for alert-chain
rehearsal. DEPLOY.md runbook covers provisioning, secrets, cron, canary
deployment, and the trip/degrade/recover verification procedure. The DO
itself is untouched — zero diff under src/do/.
Hub 68+15 tests, client sync suites 113, e2e trip/recover PASS.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Alex Newman