be7812c2da
Make o11y tier three escalations log only by default
2026-05-02 10:33:26 -07:00
31a7ed0ef3
Document root cause and durable recovery for docs API host port failures
2026-05-02 09:37:27 -07:00
faf900a197
Document how to diagnose and temporarily restore api docs search 502s caused by closed docs-api host port
2026-05-02 09:14:46 -07:00
2dce666d61
Fix video ingest event aliases and Inngest sync recovery
...
Recreate this work: diagnose video ingest events accepted without worker execution; align CLI/schema/skills around canonical pipeline/video.requested events while adding legacy alias Inngest functions for pipeline/video.download, pipeline/transcript.process, and content/summarize; upgrade the system-bus Inngest SDK past the self-hosted app-sync security gate; raise Bun serve idleTimeout so slow /api/inngest registration does not empty-reply under backlog pressure; rebuild the joelclaw CLI and verify aliases register.
2026-05-02 08:36:53 -07:00
c85d652a39
Make send --follow tolerate transient Inngest run lookup misses without crashing after event dispatch
2026-05-01 13:33:22 -07:00
b85b474bde
Stop sending Knowledge Watchdog drift alerts to Telegram
2026-04-30 17:04:09 -07:00
f717792226
Suppress meta system chatter from Telegram relay
2026-04-30 15:36:05 -07:00
9dd73a35a5
Stop paging Telegram for gateway session pressure
2026-04-30 15:34:38 -07:00
5b3444115f
Suppress noisy gateway channel degradation alerts
2026-04-30 15:32:22 -07:00
1e26b99a1a
Tighten gateway email relay quality gate
2026-04-30 15:30:15 -07:00
578d692961
Implement Slack important-channel passive intelligence routing
2026-04-30 15:17:10 -07:00
6bb733c10e
Index Slack search matches during channel backfill
...
Augment slack-channel-backfill with a bounded search.messages pass per channel so active thread replies are indexed even when their parent message is older than oldestTs and therefore absent from conversations.history. Keep history/replies ingestion for canonical pagination, then upsert search matches into slack_messages for daily-summary freshness.
2026-04-29 14:27:10 -07:00
3e869cbd55
Expand Slack backfill through thread replies
...
Teach slack-channel-backfill to call conversations.replies for top-level messages with replies, dedupe by Slack timestamp, and document why history-only backfill misses the same replies Slack search returns. This lets an agent hydrate Typesense slack_messages with the current daily-summary context instead of only root messages.
2026-04-29 14:25:43 -07:00
623c1956bc
Register host Slack backfill functions so Typesense can catch up
...
Add slackBackfillBatch and slackChannelBackfill to the host worker function registry and document that Slack backfill belongs on the host worker because it leases slack_user_token via the local secrets CLI. This lets an agent restart the host worker and send channel/slack.backfill.batch.requested to hydrate Typesense slack_messages when realtime Slack indexing misses ambient channel traffic.
2026-04-29 14:23:47 -07:00
4283cac079
Set gateway primary model to GPT-5.5 and document fallback contract
2026-04-25 11:57:05 -07:00
37a6321117
Treat Talon heartbeat misses as advisory and strip cron.heartbeat from signal digests
2026-04-24 17:04:35 -07:00
8e1cf4151d
Fix check-system-health so non-critical degradation stays warn-but-successful and document the health summary triage contract
2026-04-21 08:10:36 -07:00
0f95255c84
Restore the live joelclaw PDS operator surface by adding the missing pds CLI, canonical repo skill, and current health checks for the rebuilt service
2026-04-19 18:47:10 -07:00
c32beaa5b0
Restore bluesky-pds after a rebuild by documenting the Helm/account recovery flow and resolving PDS session auth through the repo handle before dual-write logs in
2026-04-19 18:16:26 -07:00
b646d1cadf
Auto-heal joelclaw knowledge search by recreating system_knowledge and retrying when Typesense says the collection is missing
2026-04-19 18:15:57 -07:00
9517bc242d
Default joelclaw otel emit provenance to role and hostname instead of unknown and record the docs-api secret needed after a rebuild
2026-04-19 16:54:35 -07:00
a7113d8265
Restore Typesense host access on NodePort 8108 so OTEL ingest and CLI observability stop hanging after the Colima rebuild
2026-04-19 16:41:36 -07:00
4c2e658592
Stop kube-operator-access from piggybacking Lima's ssh mux and cancel stale forwarded operator ports before launchd bootstrap
2026-04-19 16:30:29 -07:00
2b404923ab
Harden kubectl and talos access with a dedicated launchd-managed operator tunnel on stable local ports
2026-04-19 15:50:04 -07:00
573d842c40
Patch Colima recovery so a restart only counts after a stable verification window and failed recoveries suppress repeat force-cycles
2026-04-19 14:18:57 -07:00
9956802557
Reproduce the H1-usernet discriminator: extend infra/colima-proof.sh with a non-destructive recover-usernet verdict contract so pre/post snapshots can explicitly support, reject, or mark inconclusive the usernet root-cause hypothesis
2026-04-16 11:56:59 -07:00
2e770a83a7
Reproduce Colima substrate proof capture: add infra/colima-proof.sh and hook k8s-reboot-heal to snapshot failure edges, hold states, force-cycle boundaries, and post-invariant outcomes with incident artifacts and OTEL payloads
2026-04-16 11:24:07 -07:00
f445db5af9
Reproduce the stale Typesense forwarder fix: remove com.joel.typesense-portforward from the critical launchd set because Typesense is already exposed by joelclaw-controlplane-1 and the extra kubectl port-forward daemon only adds churn
2026-04-16 10:17:51 -07:00
d3b1fde4d6
Reproduce the duplicate-forwarder fix: remove com.joel.colima-tunnel from the critical launchd set and turn the old tunnel script into a clean-exit compatibility stub so it stops fighting Lima's own port forwarders
2026-04-16 10:07:30 -07:00
f93c1f0672
Reproduce the Colima timer fix: remove the 5-minute launchd StartInterval so com.joel.colima only starts the VM at boot instead of hammering colima start against a running instance
2026-04-16 09:47:27 -07:00
3394fa5b95
Reproduce the fast-but-earned healer escalation fix: add a rapid confirmation window and early host-path bailouts so real Colima collapse recovers faster without restoring single-tick panic cycles
2026-04-16 08:40:29 -07:00
4fcc853465
Reproduce the healer guardrail fix: persist Colima unhealthy streaks and cooldowns so k8s-reboot-heal stops single-tick warmup from force-cycling the VM, and document the new escalation contract
2026-04-16 08:34:54 -07:00
2f558b82e4
Reproduce the reboot-heal state fix: persist flannel and recovery markers across launchd ticks so Typesense is not re-warmed by repeated flannel restarts
2026-04-15 17:57:14 -07:00
fca46a276d
Reproduce the stale-tunnel reboot fix: make colima-tunnel restart on SSH port drift, harden k8s-reboot-heal against false Colima cycling, and make gateway status require real daemon reachability
2026-04-15 17:37:35 -07:00
4fa2bcee59
Reproduce the final reboot tunnel fix: stop forwarding 6443 from colima-tunnel and document that Caddy owns the port
2026-04-14 17:49:03 -07:00
f96157b683
Reproduce the reboot root-cause fix: add a repo-canonical colima tunnel launchd service, wire it into the critical installer, and make gateway CLI inspect system launchd services
2026-04-14 17:00:47 -07:00
5ff6a9a7df
Correct the boot-safe agent-mail runtime so launchd resolves joelhooks/mcp_agent_mail instead of hardcoding the legacy Dicklesworthstone checkout path
2026-04-12 09:03:34 -07:00
3384b45d9a
Replace the failed ADR-0239 bridge with boot-safe LaunchDaemons for critical host services and retire user/501 bootstrap
2026-04-12 08:57:27 -07:00
ca7a113b64
Harden Langfuse secret fallback so secrets lease error JSON cannot become base URLs or credentials
2026-04-12 08:38:23 -07:00
5fb084569f
Install a headless boot bridge for critical launchd services and track gateway/typesense/agent-mail plists in repo
2026-04-12 08:27:08 -07:00
ef5728a40c
Recover reboot health checks: ignore terminal pods in joelclaw status and document Colima 8/16 plus host-worker 3111 invariants
2026-04-12 08:11:22 -07:00
56913ee882
Implement ADR-0238 by keeping machine review fallbacks out of Joel-facing Todoist projects and limiting tasks/triage to human-facing task surfaces.
2026-04-08 15:48:52 -07:00
3722ccaec6
Suppress low-signal recovered gateway relay events, add a regression test, and update the gateway docs.
2026-04-07 21:11:29 -07:00
651e563b58
Treat Front unread queries as unreplied and bypass strict pagination decoding so Email Nag stops failing on _pagination.next null
2026-04-06 15:20:12 -07:00
bef5f63b74
Upsert the full channel_messages document during classification so Typesense keeps required timestamp sorting data and stops 400ing
2026-04-06 15:09:06 -07:00
51713a014e
Raise joelclaw run and event detail GraphQL defaults because live detail queries are taking more than a minute under load
2026-04-06 11:59:40 -07:00
855d79ed4b
Raise joelclaw runs GraphQL defaults because filtered self-hosted runs queries are still taking ~50s in practice
2026-04-06 11:55:18 -07:00
cfd17eebfa
Give gateway fallback time to save the session, raise joelclaw run/event detail GraphQL budgets, and stop self-healing investigator failed-run scans timing out
2026-04-06 11:40:12 -07:00
b75428b0bf
Degrade malformed channel-message-classify output to heuristic fallback instead of failing the workflow
2026-04-06 09:29:36 -07:00
2ab65429f4
Skip immutable Typesense id when patching channel_messages schema so channel-message-ingest stops flooding false failures
2026-04-06 09:18:12 -07:00
Joel Hooks