New @joelclaw/gateway-incident-latch reconstructs (source, anomalyId)
incident state from canonical gateway.decision.recorded receipts; Redis
out of the truth path. Stream tool rewrites incident-tagged decisions:
opens deliver + aggregate, identical repeats join, one material change
per PT day, resolution close-delivers once, reopens link follows.
Producer contract documented in docs/gateway.md + messaging skill.
Audited pulse replay: 15 DMs (12 transitions + 3 digests) from 42
inputs. 45 tests re-run green by steering; companion joelclaw-api
boundary change committed separately.
joelclaw gateway doctor prints PASS/FAIL for daemon (with crash-relaunch
detection against an operator restart marker), live gateway source
CLEAN/DIRTY (a running daemon plus dirty source is today's outage), and
transport state; --live proves delivery with a real probe that must
return a Telegram platformMessageId — confirmed telemetry alone fails.
Every FAIL prints its exact remediation command. gateway restart writes
the marker and ends with the doctor summary.
Panda's main diverged with two Joel-authored commits from mid-June that
never got pushed:
- 9bba6d72 fix o11y triage classifier JSON parsing + parse contract tests
- 70434b60 auto-capture #brain-joel Slack links into discovery/noted
The triage fix merged clean. The Slack capture was written against the
legacy channels/slack.ts that the messaging-v2 cutover replaced; its
feature block (URL extraction + discovery/noted emit, gated to Joel in
BRAIN_JOEL_CHANNEL_ID) is grafted into slack-runtime.ts, modernized to
use loadGatewayInngestEventConfig instead of hardcoded env fallbacks.
tsc clean; 60 tests green (triage contract tests + chat-sdk + inbound);
biome clean. Landing this makes panda's local main an ancestor of
origin/main so panda reconciles with a plain fast-forward.
joelclaw recall and voice recall now answer from Brain/observation
stores (19 tests). memory_observations archived to NAS (28,050 docs,
238MB, sha d2f3038d, verified) and dropped. Zombie functions
(reflect/review-promote/proposal-triage/batch-review) unregistered and
deleted with the MEM/FRIC suites (-6,306 lines). System log retired:
panda copy snapshotted (3,981 lines, sha 4e8e815d), slog write path /
system-logger / sync + backup + PDS mirror removed. Per decisions
decide-legacy-memory-retirement + decide-system-log-home.
Expired wake reservations now check Redis for the delivery event id;
actually-queued wakes flip to notified instead of firing again — closes
the crash-after-send window without a duplicate DM.
Dedicated Inngest cron (17 * * * *) on the host worker: bounded live
conversations.history sweep per allowlisted channel (cc-matt-p +
brain-joel), non-Joel human roots classified untagged/started/shipped
from Joel's root reactions — Slack stays the only store. Finding runs
write one sensitive observation page (no message bodies); wakes fire
only on state transitions via joelclaw notify --event-id with 7-day
Redis dedup. Funnel: scripts/work-state-pass-funnel.ts. Seeded proof
01KXRGVJD5JPS8JGH95JP7PJSN (page + one wake), live run proved zero
repeat wakes. Closes the slack-work-state brief — all steps done.
Scoped users.d policy (admin/writer/reader identities), hash-only
secrets, Flagg-local SSH tunnel plist, config templates, and the
runbook with verification + rollback. Verified against a disposable
ClickHouse 24.12 with positive and negative grant probes. Live apply
requires Joel present.
Step: .brain/projects/telegram-signal-system/clickhouse-access-control.svx
Satellite rig runbook/setup hardening, NAS_HOST pinned to LAN IP per
mount contract, agent-mail daemon uv resolution, CLAUDE.md + brain
areas/pipeline notes, and function registry wiring for the new
usage/webhook/backup functions.
"direct exposure and experience, documentation, or a runbook" -- the
three ways knowledge transfers, Observability Engineering, Ch. 11
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ClickHouse store with durable outbox + forward mode for satellite
workers (OTEL_STORE=forward -> flagg), clickhouse-otel capability
adapters for cli/sdk, backfill script, and cutover runbook. Raw OTEL
stops writing to Typesense; launchd/k8s env updated to match.
"For the observability domain, rows pertain to individual telemetry
events, and columns pertain to the fields or attributes of those
events." -- Observability Engineering, Ch. 16
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>