mirror of
https://github.com/thedotmack/claude-mem.git
synced 2026-09-20 04:23:02 +08:00
work/unbundle
35 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Madan Kumar
|
0bbef55a70 |
fix(sqlite): skip empty-title observations at capture (#3176)
Empty-title observations are malformed, low-signal rows that fill the recency-based recall window without adding any facts. Skip them in storeObservations so they stop accumulating in the observations table. The single-observation storeObservation wrapper now requires a non-empty title (throws instead of returning an undefined id), and its /api/memory/save caller falls back to the text-derived title for whitespace-only titles so a blank title never reaches the store. Refs #3163 |
||
|
Alex Newman
|
7bfc401627 |
fix(session): use latest user_prompts text on rehydration (#4047)
Cold initializeSession now loads the latest non-empty user_prompts.prompt_text when currentUserPrompt is absent, instead of falling back to sdk_sessions.user_prompt (first prompt only). Closes #4047. |
||
|
Alex Newman
|
7d3f0047c5 |
fix(sync): register a memory_session_id, do not re-register it (#4027)
ensureMemorySessionIdRegistered writes only when the stored id is NULL. Deliberate re-identification stays on updateMemorySessionId. Polarize the 8 suites that still treated ensure as the re-key path. Co-authored-by: Vorrawut Judasri <vorrawutjud@gmail.com> |
||
|
Po-Han Shih
|
e92a2d0acf |
fix(sqlite): populate session_summaries files_read and files_edited (#3517)
* fix(sqlite): populate session_summaries files_read and files_edited Roll up observation file lists when storing a session summary so the schema columns are no longer always NULL (false empty sessions). Fixes #3517 Co-authored-by: Cursor <cursoragent@cursor.com> * fix(observer): keep claimed file evidence on summary-only responses Greptile P1 on #3530: when XML has a summary but no observations, still merge Read/write file evidence into session_summaries columns. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Alex Newman
|
496dd0d0b8 |
fix(sqlite): keep ON UPDATE CASCADE through session_summaries rebuilds (#3980)
v7 could recreate session_summaries without ON UPDATE CASCADE after v21 had already been stamped, so parent-key rewrites failed for sessions that already had summary rows. Rebuild CREATE TABLE statements now include both cascade clauses, and v21 repairs from the live FK instead of the version row. Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com> |
||
|
L4XB
|
79208bd58a |
fix(sqlite): keep post-v7 columns through the session_summaries rebuild
The v7 rebuild that drops the UNIQUE constraint from session_summaries recreated the table from a fixed v7 column list. Fresh installs stamp every migration at once, so a database whose base schema already carried v11's discovery_tokens next to the v7 constraint lost the column in that rebuild, and the version-gated v11 guard never re-added it: every summary write failed with "no column named discovery_tokens" from then on while observations kept being captured. The rebuild now carries every live column outside its v7 list over (type and default included) and copies them along, and ensureDiscoveryTokensColumn re-checks the live tables regardless of the stamped version. Fixes #3890 |
||
|
Alex Newman
|
fd0ecf0233 |
feat(sqlite): durable tool_uses backup index + get_tool_uses disclosure (#3898)
Raw tool I/O had no durable home. `pending_messages` is the generation queue -- rows are claimed, summarized, and deleted -- so once an observation existed the original tool_input/tool_response were gone. Adds `tool_uses` (schema v51) as a by-reference side index for those bodies, written from the one ingest choke point both the PostToolUse hook route and the transcript-watch processor already share. The JSONL transcripts and `src/services/transcripts/*` remain the spine and are untouched. Schema (Receipt freeze 2026-09-06): UNIQUE(content_session_id, tool_use_id), nullable `or_generation_id` / `or_session_id` as join keys back to an OpenRouter spend line, and deliberately no cost_usd/micros -- this table carries tool identity, dollars stay on the OR stamp. No FK on session_db_id/observation_id: a FK there can abort the constructor migration chain (#3378), and observation_id is linked late by design. Write path dual-writes alongside -- never instead of -- the pending_messages enqueue, and swallows its own failures so an observation is never lost to a backup-index error. `toolUseId` now actually reaches the worker from hooks: it was missing from NormalizedHookInput and every hook adapter, so only the transcript path supplied it. ResponseProcessor links the batch's claimed ids to the first stored observation. Read path is progressive-disclosure layer 4: POST /api/tool-uses/batch requires explicit ids (never a full-table scan), GET /api/tool-uses returns a cheap index shape with size hints and never the payloads, and the `get_tool_uses` MCP tool is described as a last resort. claude-mem's own read tools are skipped by the writer -- without that, every call to get_tool_uses would store the bodies it just returned. Payloads are stored in-row with a 64 KB soft cap and a UTF-8-safe truncation marker; content_hash is computed over the original. Join contract for Receipt: RECEIPT-JOIN.md Claude-Session: https://claude.ai/code/session_01QgdJ6gExgBirDoAxknt94m Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Copilot
|
1c7aaff38d |
Add independent Cursor/Grok Bot marketplace plugins and host-observer install flow
Ship independent Cursor/Grok Bot marketplace plugins, host-observer probe, and Grok transcript discovery. Fixes Greptile P1s: source-specific manual sessions, probe-before-persist host observer, occupied-port rejection, and Grok Bot agent-data root discovery. |
||
|
Alex Newman
|
28c84255fe |
feat(telemetry): report observed model, source, and billing tier to PostHog (#3836)
PostHog only knew the observer model (the model claude-mem uses to write observations). It now also carries, per observed session: - observed_model: the model the user's IDE session is running, read from the transcript's last assistant entry on every Stop hook - observed_billing: closed enum (max | pro | team | enterprise | subscription | api_key | bedrock | vertex | foundry | unknown) detected in the hook process from Claude Code's env and ~/.claude.json oauthAccount.organizationType - ide and provider are now actually carried on observer_turn_rollup (the docs already claimed this; the rollup was dropping them) The Stop hook sends both fields to /api/sessions/summarize; the worker persists them on sdk_sessions (observed_model, observed_billing; schema version 50) and mirrors them onto the in-memory ActiveSession so every per-turn session_compressed record carries them into the rollup (last-seen semantics, 'unknown' when never learned). Privacy: only oauthAccount.organizationType, oauthAccount presence, and customApiKeyResponses.approved are read from .claude.json, projected immediately after parse; parse errors log only the error class name. Both new keys are added to the scrub whitelist and documented in telemetry.mdx. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019NoBPqpo7YaR3HKipExjAT |
||
|
Alex Newman
|
0b8288bf7c |
fix(worker): repair orphaned FK parents before migration rebuilds; adoption after db init (#3378) (#3390)
* test(sqlite): pin the #3378 background-init FK abort to the v7/v9 rebuild copies (red)
Pin-down proof on unfixed code: a legacy v4-era DB seeded (with
PRAGMA foreign_keys = OFF, mimicking historical data) with a child row
whose memory_session_id has no sdk_sessions parent aborts the
SessionStore constructor migration chain:
- orphaned observation -> SQLiteError: FOREIGN KEY constraint failed
at makeObservationsTextNullable (SessionStore.ts:1167), the
INSERT INTO observations_new SELECT copy; committed schema_versions
stop at [4,5,6,7,8], v9 absent.
- orphaned session_summary -> same error at
removeSessionSummariesUniqueConstraint (SessionStore.ts:1074), the
INSERT INTO session_summaries_new SELECT copy; committed
schema_versions stop at [4,5,6], v7 absent.
These two child-table rebuilds run with foreign_keys = ON (set per
connection in connection.ts:45) and, unlike the v21/v33/v34 rebuilds,
never disable it - so each orphaned row makes the copy INSERT violate
the freshly created FK, dbManager.initialize() rejects, and the worker
never reports ready. runOneTimeV12_4_3Cleanup cannot be the aborting
step: it swallows its own errors (CleanupV12_4_3.ts:70-76).
This test asserts the desired post-fix behavior (constructor completes,
orphans survive, stub parents created) and currently fails 4/4 with the
FK error above.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RSNZmfi4vSYjTeEWtwqKrW
* fix(worker): repair orphaned FK parents before migration rebuilds; adoption after db init (#3378)
Three root-cause fixes for the background-init half of #3378 (the
recycle-loop half shipped in 13.12.3 as
|
||
|
Alex Newman
|
4c1fea55ce |
fix(concepts): normalize concept tags at write time + v49 backfill (#3379) (#3389)
* fix(concepts): normalize concept tags at write time + v49 backfill (#3379) The context-injection query matches concepts exactly (WHERE value IN (...)), so tags stored as "keyword: description" — a shape the concept_guidance prompt's own "gotcha: traps or edge cases" format invited — were silently excluded from injection. Three-part root-cause fix, no query-side tolerance: - parser.ts: truncate each extracted concept at the first ':' and trim, then drop empties and the observation type (novel tags stay storable) - plugin/modes/{code,email-investigation,law-study,meme-tokens}.json: explicit OUTPUT RULE that each <concept> element is the bare keyword only - SessionStore.ts: migration v49 backfills existing rows, truncating each concepts array element at the first ':' (version-gated, idempotent) Known caveat (out of scope): inbound cloud sync can re-import malformed concepts from other clients that predate this fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RSNZmfi4vSYjTeEWtwqKrW * fix(concepts): v49 requeues corrected native rows and skips invalid JSON Addresses two P1 review findings on PR #3389 (v49 migration): - Corrected rows never re-synced: the backfill rewrote concepts but left synced_at/sync_rev untouched, so already-pushed rows were never re-selected by the push drain and the hub kept serving the malformed tags. Mirror requeuePromptSync: for NATIVE rows (origin_device_id IS NULL) bump sync_rev via incrementCanonicalDecimal and re-null synced_at inside the migration transaction; the next drain re-pushes the corrected row body at the higher rev. Replica rows are normalized locally only — their repair travels from their origin device (the plan's acknowledged inbound caveat). - Malformed JSON bricked boot: a non-JSON concepts value containing ':' passed the LIKE prefilter and made json_each throw, aborting the constructor migration chain (worker never initializes — the #3378 failure class). Add `AND json_valid(concepts)`; invalid rows are equally unreadable before and after v49 for every json_each reader, so skipping them changes no behavior. Tests: invalid-JSON row survives byte-identical with v49 recorded; native changed row → normalized + synced_at NULL + sync_rev bumped by exactly 1; replica changed row → normalized with sync fields untouched; colon-free row fully untouched including synced_at. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RSNZmfi4vSYjTeEWtwqKrW --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
b3eafc82b9 |
feat(sync): complete SyncHub-only client cutover
Retire legacy client adoption and per-kind sync state, establish the one-time launch boundary, and expose authenticated Hub health. Add bounded device metadata and a loopback-only two-client protocol-v2 matrix E2E. |
||
|
Alex Newman
|
1e3b2853ff | feat(sync): add canonical v2 projection pipeline | ||
|
Alex Newman
|
cbdf1ee3f4 |
feat(sync): client apply path + migration v41 (Phase 2)
Migration v41 adds origin identity columns (origin_device_id, origin_local_id, sync_rev) to observations/session_summaries/user_prompts with partial unique indexes, plus a sync_state table for the pull cursor and epoch. Narrows the v7 session_summaries rebuild predicate to table-level UNIQUE constraints only (PRAGMA origin 'u') — v41's new index would otherwise retrigger the rebuild every boot and destroy post-v7 column data; regression-tested. New SyncApply module applies remote hub ops in one transaction with the cursor advance: idempotent origin-keyed upserts pre-stamped synced_at (structural echo guard against the push drain), remote timestamps preserved, rev-guarded mutations (set_title/set_prompt_session/ remap_project), set_title convergence via sync_state title parking, epoch-mismatch cursor reset, FTS via existing triggers, post-commit fire-and-forget Chroma. Module header pins the Phase 3 push contract (body field mapping + rev minting rules). Avoids bun:sqlite .run().changes decisions (unreliable after RETURNING in the same tx). 22 new apply tests + v7 regression test; full suite green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
eed4f569cd |
test(cloud-sync): cover joined prompt drain, lane threshold, v40 once-only, and re-push hooks
- prompt mapping: session_db_id join pushes real memory_session_id/project; per-row fallback for NULL-memory-id sessions and orphan prompts; stamping still lands through the rowAlias-qualified drain - lane pick: >200 pending sends X-Sync-Lane: backfill on every page of that kind's drain while other kinds stay live; exactly 200 stays live - v40 requeue: records version 40 and never re-nulls stamps on later boots - requeuePromptSync: updateMemorySessionId/ensureMemorySessionIdRegistered requeue only the affected session's prompts, and only on an actual change Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
5af1b059be |
feat(cloud-sync): v40 self-repair — requeue all prompts through fixed mapper
Ships the prompt repair to every install instead of requiring the out-of-band SQL fix that was run for one account: schema migration v40 re-nulls user_prompts.synced_at once, so the next flush re-pushes the device's full prompt history through the fixed sdk_sessions-join mapper. The server upserts on (user_id, device_id, local_id) with a change guard, so corrected rows overwrite in place and unmapped rows (identical payload) cost nothing. Legacy standalone-client uploads had the same broken mapping, so v40 deliberately un-stamps the rows v39 just adopted — test expectations updated to match. CloudSync now picks a sync lane per drain: a backlog deeper than one page (fresh install, offline catch-up, v40 requeue) sends X-Sync-Lane: backfill so the server suppresses per-row realtime broadcasts and admission-gates the burst; small increments stay on the live lane for instant dashboard updates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0175gu1h5jbW2kHDARtg6gd4 |
||
|
Alex Newman
|
f7bb940f4f |
fix(sync): gate synced_at migration on column presence, guard null user_prompt
Community-edge DBs already consumed schema versions 36-38, so the version-gated migration silently no-opped there. The migration now keys on PRAGMA table_info per table (recorded as v39), stamping legacy rows only when columns are first created. Also guards three user_prompt.substring crash sites in SessionManager hit during worker-restart stale-session windows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
b3e79f8302 |
feat(sync): add synced_at columns, unsynced partial indexes, and legacy cursor stamping (schema v36)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Alex Newman
|
22e4325ce2 |
fix: harden sqlite migrations and settings writes
(cherry picked from commit
|
||
|
Alex Newman
|
348d9ee406 | feat(recovery): scope memories by platform source | ||
|
Alex Newman
|
edc5cf7d59 |
Ponytail audit (−10.4k lines) + worker-restart hardening + deterministic dependency closure (#3021)
* plan-10 Phase 1: ship deterministic plugin runtime dependency closure Approach A — commit & ship plugin/bun.lock so the plugin's runtime node_modules install is deterministic, fixing the recurring `Cannot find module 'zod/v3'` (#2730). - align generated plugin zod range to root (^4.4.3) in build-hooks.js - new scripts/gen-plugin-lockfile.cjs generates plugin/bun.lock as a build artifact after build-hooks.js writes plugin/package.json - track & ship plugin/bun.lock (.gitignore negation, .npmignore, files allowlist) - install with `bun install --frozen-lockfile --ignore-scripts` at runtime Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 2: fail loud at install time on a broken dependency closure Strengthen verifyCriticalModules to assert each dependency is actually importable via require.resolve (not merely a directory), and assert the worker-required zod subpaths resolve: zod/v3, zod/v4, zod/v4-mini. A partial/stale install now fails `npx claude-mem install` immediately instead of surfacing later as a Stop-hook `Cannot find module 'zod/v3'`. Bin-only packages (e.g. tree-sitter-cli, which has no bare-name entry point) fall back to resolving <dep>/package.json so a healthy install isn't falsely rejected. Adds tests/cli/verify-critical-modules.test.ts covering a missing zod/v3 subpath (throws), a complete zod (passes), and a bin-only dep (passes). Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 3: clean-room install + import smoke test (#2730 backstop) Add scripts/smoke-clean-room.cjs and a `smoke:clean-room` npm script. Against fresh temp dirs (never the repo's node_modules) it: - copies plugin/, runs `bun install --frozen-lockfile --ignore-scripts`, asserts zod, zod/v3, zod/v4, zod/v4-mini resolve, and boots the bundled worker asserting no `Cannot find module` — the direct #2730 regression guard; - `npm pack`s, installs the tarball into a second temp dir, and load-tests the published bin entrypoint, warning loudly on any declared main/exports target missing from the tarball (latent #2537 gap). Exits non-zero naming the missing module on any failure; cleans up all temp dirs and the tarball in a finally. Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10 Phase 4: gate CI and publish on the clean-room dependency closure - ci.yml: new `clean-room-deps` job (between build and the docker e2e job) runs a frozen-lockfile drift check on the committed plugin lockfile, then `npm run build` + `npm run smoke:clean-room`. The drift step catches a contributor who changed plugin deps without regenerating plugin/bun.lock. - npm-publish.yml: add setup-bun and run `npm run smoke:clean-room` between build and `npm publish`, so a broken runtime closure cannot be published on a tag push (ci.yml does not run on tags). Secrets block untouched. Refs #2783, #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10: doc recluster note + Phase 0 execution slice for #2730 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plans: backlog recluster (2026-06-04) — cross-cluster execution order + plan-13 doc Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan-10: gen-plugin-lockfile degrades gracefully when bun is absent The Windows build CI job has no bun on PATH; regenerating the lockfile there threw and failed the build. The committed plugin/bun.lock is already the deterministic closure, so skip regeneration (non-fatal) when bun is missing and a lockfile exists; fail loud only when neither is available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: rebuild plugin artifacts after merging main (v13.5.1) + plan-10 work Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: rebuild plugin artifacts after merging main v13.5.5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): daily upgrade pass — agent SDK 0.3.172, better-auth 1.6.16, posthog-node 5.36.15, dompurify 3.4.9 - Bump @anthropic-ai/claude-agent-sdk 0.2.141 -> 0.3.172 (tsc + full test suite green) - Remove deprecated @types/dompurify stub (dompurify ships its own types) - Add overrides.tmp ^0.2.7 to clear GHSA-52f5-9888-hmc6 / GHSA-ph9p-34f9-6g65 via np -> listr-input -> inquirer -> external-editor -> tmp chain - npm audit: 0 vulnerabilities; npm outdated: clean - package-lock.json is gitignored in this repo, so not committed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * plan: worker-restart single-source-of-truth — 7-phase fix for restart races Phased plan from the adversarially-verified diagnosis (wf_f07f3541-b05): kill the cache mirror, single verified restart initiator, self-replacing restart endpoint, unified spawn gate with lockfile, PID-file demotion, test data-dir isolation, soak verification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(restart): delete sync-script cache-mirror and HTTP restart trigger Phase 1 of plans/2026-06-10-worker-restart-single-source-of-truth.md. The installed-version cache mirror wrote version-N code into the version-(N-1) cache dir, manufacturing permanent version disagreement; the HTTP POST to /api/admin/restart raced the CLI restart that follows it in build-and-sync. Both are deleted; the CLI worker:restart in the marketplace copy is now the single restart initiator, and the sleep 1 between the two mechanisms is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): restart proves itself or exits 1 Phase 2 of plans/2026-06-10-worker-restart-single-source-of-truth.md. worker-service restart now captures the old worker pid, waits for the port with the same platform-scaled 15s budget as stop, spawns the marketplace copy of worker-service.cjs when present, then polls /api/health until the pid changes and the version matches this build's baked __DEFAULT_PACKAGE_VERSION__ — success is printed to stdout, deadline (platform-scaled 30s) exits 1 with the last observed health payload and the spawned script path. The --daemon generic start-failure path now exits 1 instead of masquerading as success; the three duplicate-suppression exits remain 0. New helper src/services/restart-verify.ts (worker-service.ts bootstraps on import, so the helper lives in an import-safe module) with 8 tests covering pid-flip success, stale pid, wrong version, unreachable timeout, 503-degraded acceptance, and null-oldPid version-only verification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): self-replacing worker — old worker spawns its successor Phase 3 of plans/2026-06-10-worker-restart-single-source-of-truth.md. /api/admin/restart was kill-only: hooks that POSTed it then raced the dying worker with their own lazy-spawn (the observed recycle ping-pong). Now the dying worker spawns its successor itself — after a re-entrancy- guarded, deadline-bounded (platform-scaled 10s) graceful shutdown, and only once its port is confirmed free; stop and signal shutdowns stay kill-only. The hook recycle path waits for that successor via /api/health polling (HOOK_READINESS_TIMEOUT_MS budget) and lazy-spawns only as a fallback, with a warn-only version re-check so a hook never recycles more than once per invocation. Shutdown sequence lives in import-safe src/services/worker-shutdown.ts (worker-service.ts bootstraps on import); registerSignalHandlers no longer pre-sets isShuttingDown — the supervisor's shutdownInitiated guard owns signal dedupe, and pre-setting would no-op the new entry guard. 13 new tests cover re-entrancy, deadline expiry/rejection, handoff ordering, kill-only reasons, successor-wait vs lazy-spawn fallback, and pre-graceful bookkeeping failures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): one spawn gate; CLI restart defers to the self-replacing worker Phase 4 of plans/2026-06-10-worker-restart-single-source-of-truth.md. Three uncoordinated spawn paths (hook lazy-spawn, MCP worker-spawner, CLI) with two different bun resolvers produced 3-launcher collisions within a single second. Now a wx-flag lockfile (<DATA_DIR>/spawn.lock, 60s mtime staleness with re-stat-before-unlink, owner-checked release) gates every external spawn: lock losers never fail — they skip the spawn and wait for the winner's worker. resolveBunRuntime is deleted in favor of ProcessManager's resolveWorkerRuntimePath (adds BUN_PATH, ~/.bun/bin, brew, which fallbacks), closing the kill-then-can't-respawn path; mcp-server prefers the marketplace worker script so stale cache dirs stop spawning stale workers. Integration fix surfaced by live verification: the CLI restart raced the Phase 3 self-replacement handoff (the successor re-binds the port in ~200ms, so waitForPortFree always timed out and restart exited 1 while the restart had actually succeeded). The CLI now verifies the worker's self-spawned successor directly, and only spawns — gate- wrapped, after the port frees — as the fallback when no worker was running, the shutdown POST was rejected, or no successor appeared. The dying worker's handoff is intentionally ungated: it spawns only after its own port closes, and hooks wait on it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(restart): demote the PID file — health and port are the liveness oracle Phase 5 of plans/2026-06-10-worker-restart-single-source-of-truth.md. The dying worker's shutdown cascade deleted the PID file unconditionally as its final act, clobbering the successor's freshly-written file; status then required portInUse AND pidInfo, so a healthy worker reported as "not running". Now every PID-file deletion is owner-guarded: the supervisor cascade deletes only its own pid (removeOwnedPidFile), and the CLI stop/restart-fallback, the restart handoff, and the daemon start-failure cleanup go through removePidFileIfOwner (owner-or-dead — a live successor's file always survives; corrupt files are left for the next boot's validator). status sources from GET /api/health alone (pid, version, uptime, workerPath; 503-degraded counts as running and now surfaces its queue detail), with port-in-use-but-unreachable and not-running fallbacks — all exit 0 as before. The --daemon duplicate gate checks the port first (ground truth) and the PID file second (advisory, for the freed-port-but-undeleted-file window); duplicate suppression stays exit 0. writePidFile/touchPidFile remain — the file is diagnostics, and the worker stays its only writer. Also fixes combined-run test pollution: spawn-gate and worker-utils timeout tests now eagerly import paths.js before setting a temp CLAUDE_MEM_DATA_DIR, so the import-time DATA_DIR const can't freeze on a deleted temp dir for suites loaded later in the same bun process. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: no test ever touches the real ~/.claude-mem again Phase 6 of plans/2026-06-10-worker-restart-single-source-of-truth.md. process-manager and graceful-shutdown tests wrote corrupt JSON and sentinel PIDs (2147483647) into the real ~/.claude-mem/worker.pid and drove the real supervisor.json cascade under a snapshot-restore that a killed run would skip — that pollution contaminated production logs and a prior diagnosis. Both files now set a temp CLAUDE_MEM_DATA_DIR at the top of the file before dynamically importing the code under test (ESM hoisting makes beforeEach too late), assert their paths landed outside the real dir, and derive PID_FILE from the same frozen paths module the code uses so test and code can never diverge under bun's shared module cache. The snapshot-restore scaffolding is deleted; zero assertions changed. tests/preload.ts gains a tripwire: when CLAUDE_MEM_DATA_DIR is unset it fills a per-run temp dir, so no test in any file can fall through to the real data dir. Fallout made explicit: worker-spawn child processes get an explicit temp dir; install-error-matrix restores rather than deletes the env var; settings-defaults-manager pins the unset-env default it was implicitly relying on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settings): bootstrap notices go to stderr, never stdout CI on PR #2894 caught the latent bug: on the first boot in a fresh data dir, SettingsDefaultsManager printed '[SETTINGS] Created settings file with defaults: ...' to stdout before the start command's JSON hook payload, corrupting the machine-readable contract every fresh install's first hook invocation relies on. The Phase 6 per-run temp data dir made the cold-dir case deterministic in CI, exposing it. Both informational notices (creation, nested-schema migration) now use console.warn — stderr — matching the function's existing failure-path idiom; two regression tests pin stdout silence on both paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(restart): address PR #2894 review — dedupe script resolver, skip futile port wait Both inline copies of the marketplace-first script-candidate list in worker-service.ts (restart fallback + successor handoff injection) now call the exported resolveWorkerScriptPath() ?? __filename, so the candidate list lives in one place. verifyRestartedWorker's failure result gains lastPollSawHealth; when the self-replacement handoff verification timed out while a live (but unverifiable) worker was still serving on the port, the CLI fallback now skips its port-free wait — the port cannot free while that worker lives, so the wait only burned its full platform-scaled budget before the same final verification ran anyway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): stale Claude CLI can no longer silently kill every observation findClaudeExecutable validated candidates with --version only, so an abandoned old install shadowing a current CLI in PATH (e.g. npm-global 2.0.42 next to the auto-updating native installer) passed validation, then died at every Observer spawn: the SDK passes --permission-mode dontAsk (hardened-options) which old CLIs reject with exit 1. Result: healthy worker, zero observations, no visible error (#2782 family; previously #1857/#2049/#1866/#2142 in the same class). - Probe every candidate (which -a + known install paths) with `--permission-mode dontAsk --version`: one spawn proves both flag compatibility and version, no API call (~150ms) - Prefer the newest capable version; PATH order only breaks ties - Explicit CLAUDE_CODE_PATH still wins but fails loud with version and remedy when too old, instead of dying silently at spawn - All-too-old throws an error naming each candidate, its version, and how to fix; resolution success logs at INFO with the chosen version - Cache successful resolution 15 min (resolver runs per SDK query); never cache failure so a CLI update is picked up without restart - SDK child keeps a 2KB stderr tail and includes it in the exit WARN, so "unknown option" deaths are diagnosable at default log level Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(audit): add ponytail audit master plan + SQLite removal companion Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-0): remove commit-hash fabrication verification Removes the commit-verification system whose second-system effects (greedy hex regex false-positives + compensating scaffolding) cost more than the edge case it caught. Deletes src/sdk/commit-verification.ts + its 2 tests, the verify-before-persist block + stripFabricatedHashesFromSummary in ResponseProcessor, and the now-dead fabricated_count/fabrication_* telemetry in buffer.ts/scrub.ts/npx-cli telemetry.ts. Consequence: PostHog Fabrication Rate tile goes dark (no other producer). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-1): remove duplicate SQLite stack, test the shipping path Two parallel SQLite systems lived in src/services/sqlite/. The worker only ever used SessionStore (which runs its own inline migrations); the second stack — ClaudeMemDatabase/MigrationRunner + a free-function CRUD API — was reachable only from tests. This removes the dead duplicate and rewrites its tests against the real SessionStore path. Deleted: Database.ts, migrations/runner.ts, index.ts, SchemaRepair.ts, all 6 CRUD barrels, transactions.ts, and the sessions/summaries/prompts/timeline/import submodules (18 files). Trimmed the 5 survivor leaf functions to their single live export. Removed dead SessionStore methods getSessionSummaryById (broken) and storeObservationsAndMarkComplete (0 callers). Replaced 9 old test files (coupled to the dead stack) with 7 focused session-store-*.test.ts suites + a rewritten cleanup-v12_4_3 test, all exercising the worker's real path. Net ~-3,262 src lines. tsc clean; full suite 2253 pass / 0 fail; dead-reference sweep zero. No production behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-6): drop dead storage methods, dedupe EnvManager, stdlib swaps Dead-code deletes (all zero-caller verified): sqlite ProjectsRepository.upsert + getByRootPath, ServerSessionsRepository.getByMemorySessionId, MemoryItemsRepository.getByLegacyObservationId, pg AgentEventsRepository.createMany + TeamsRepository.getMember; shared getWorkerSocketPath, createBackupFilename, and the deprecated ENV_FILE_PATH const. Refactors (behavior-preserving): EnvManager's three open-coded 5-key credential blocks collapse to one CREDENTIAL_KEYS loop (whitelist semantics kept, no Object.assign); parseEnvFile line-parser → util.parseEnv; fetchWithTimeout's manual setTimeout race → AbortSignal.timeout (caller timeout-message contract preserved). Net ~-106 lines. tsc clean; full suite 2253 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-4): sweep dead/redundant code from src/server Deletes (zero-caller verified): jobs/outbox.ts (canonical path is IngestEventsService/EndSessionService) + its test; the src/server/mcp/ surface (live MCP server is src/servers/mcp-server.ts) + its test; the always-inert ServerBetaProviderRegistry/EventBroadcaster boundaries (no Active variant exists) + their /v1/info payload keys; the unused isServerClassified guard; dead option sessionDebounceWindowMs (route + IngestEventsService). Dedupe: hoist the 5 byte-identical auth helpers shared by auth.ts/postgres-auth.ts into request-auth-helpers.ts; collapse resolveSummaryQueue/resolveEventQueue into one resolveQueue(lane). Kept (plan premise didn't verify): the ServerV1Routes runtime? option — a live smoke test sets and asserts it. Disabled* boundary subclasses kept (real Active counterparts chosen at runtime). Net ~-570 source lines. tsc clean; full suite 2242 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-7): dead-code sweep + dedup across services + npx-cli Deletes (zero-caller verified, tests removed in lockstep): ProcessManager getChildProcesses + parseElapsedTime (ps-reaper leftovers), ModeManager getObservationConcepts/validateType/getTypeLabel, AgentFormatter renderAgentFileHeader/renderAgentColumnKey/renderAgentContextIndex (return-[] inert, HeaderRenderer arms collapsed to Human-only), asyncHandler export, PriorMessages.userMessage (always ''), and 3 dead barrels (services/context, services/server, services/server/Middleware). Dedup: shared toError() helper (src/utils/to-error.ts) across 8 in-scope sites; asMs hoisted to telemetry/common; spawnPlugin() folds 4 spawnHidden blocks; countObserverSessionRows extracts the CleanupV12_4_3 count trio; IS_WINDOWS imported from paths.ts (was redeclared 3x); detectOsVersion inlined to os.release(). Kept (plan premise didn't verify): bun-resolver's spawnSync (npx-cli runs under Node — Bun.which would ReferenceError); context-generator.ts (a build entrypoint + test mock seam) — barrel still removed; Middleware.ts consumer Server.ts repointed to the real middleware. Net ~-279 lines. tsc clean; full suite 2232 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-5): delete orphans + dead UI/util code across cli/supervisor/ui Orphan deletes (zero-caller verified): src/bin/ (import-xml-observations, cleanup-duplicates — not wired to any entrypoint), src/adapters/ (abandoned parallel impl; real adapters are src/cli/adapters/), core/schemas/context-pack.ts + core/schemas/index.ts barrel. Companion tests removed in lockstep. Dead functions: 7 cursor-utils helpers (kept the 6 live registry/context writers), logger correlationId/sessionId/timing. YAGNI trims: gemini-cli metadata block (never read), useStats collapsed to fire-and-forget (+ dead Stats/WorkerStats/DatabaseStats types), useTheme resolvedTheme, useContextPreview.refresh from the public result; env-sanitizer ENV_PROXY_VARS folded into ENV_EXACT_MATCHES. Dedup: useGitHubStars + formatNumber inlined into their single consumer GitHubStarsButton (kept lowercase-k display — matches GitHub, Intl would uppercase); waitForExit extracted in process-registry and shared with shutdown's reapSession (placed low to avoid a circular import). Net ~-1085 lines (incl. test deletions). tsc + viewer typecheck clean; full suite 2181 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-2a): integrations/sync dead exports + scoped Chroma cleanup Dead-export deletes (zero-caller verified): syncContextToAgentsMd, updateWindsurfContextForProject, handleWindsurfCommand, detectClaudeCode (+ orphaned exec/promisify imports), getScriptExtension/detectPlatform, getVersionCheckAbsolutePath (+ companion plugin-distribution test edit). ChromaSyncState flush/resetCache/dirty (bump/replace persist eagerly, so dirty was always false). Scoped to the Chroma write path only (SQLite/search/telemetry copies stay live): dropped the created_at ISO field (kept created_at_epoch, the field actually read back) and the discovery_tokens param/field from ChromaSync's local Stored interfaces + the 3 live call sites. Stdlib/shrink: parseSemver/compareSemver → localeCompare numeric (ordering verified); OpenClaw roots hoisted to one const; bootstrapWatermarks max-loop → Math.max; Goose YAML builders collapsed to one withHeader flag. Defers the 5-installer consolidation to a separate pass. Net ~-187 lines. tsc clean; full suite 2181 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-2b): delete dead fetchAndInjectOpenCodeContext Orphaned when slice-2a removed syncContextToAgentsMd (its only caller chain). Confirmed zero callers. Kept fetchRealContextFromWorker (live at :323). Per feasibility analysis, the plan's 5-installer table-driven consolidation (headline -2400) was descoped: the 5 installers write 5 different config schemas and 4 different context formats with thin test coverage, so a 5->1 collapse would relocate divergence into callbacks and risk untested IDE-config write paths for a line-count-only win. Plan gate ('only if behavior genuinely identical') not met. tsc clean; full suite 2181 pass / 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(slice-3): worker dead-code sweep + provider/search consolidation Search pipeline: delete the unreachable half built speculatively (TimelineBuilder, ResultFormatter instance methods [kept the one live static], search/filters/, SearchStrategy.canHandle/name + BaseSearchStrategy, HybridSearchStrategy no-op search(), dead SearchOrchestrator wrappers, ChromaQueryResult) + companion tests. Dedup the 5 genuinely-identical chroma->recency->hydrate->FTS paths into one hybridSemanticHydrate helper and the 3 timeline renderers into renderTimeline() (search()/decisions()/changes()/howItWorks() left as-is — different shape). Providers: extract OpenAICompatibleProvider base from the Gemini/OpenRouter twins, preserving every per-provider divergence via flags/abstracts (truncation guard, token estimation, empty-response handling, endpointClass, the Gemini-only RPM throttle). Dedup parseRetryAfterMs into retry.ts. http/agents/session: delete the IngestEventBus machinery + ingestPrompt, the read-only POST /api/processing dup, FallbackErrorHandler.shouldFallbackToClaude, RateLimitStore.getAll/clear, SessionCompletionHandler.completeByDbId, SessionCleanupHelper, 3 dead agent types, and no-op empty-passthrough validateBody; collapse the 6 CorpusRoutes 404 blocks into one helper. Kept (false-dead, caller verified): SSEEventPayload (live via WorkerRef), RateLimitStore.get (live via shouldAbortForQuota). Both adversarial reviews confirmed behavior preserved. Net ~-2,640 lines. tsc clean; full suite 2080 pass / 0 fail; dead-ref sweep zero. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(plugin): regenerate distribution artifacts after ponytail audit Rebuilds the worker/mcp/server-beta/transcript-watcher/context-generator bundles and the viewer UI from the post-audit source (Slices 0-7). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(env): require Node >=20.12.0 to match util.parseEnv usage EnvManager adopted stdlib util.parseEnv (added in Node 20.12.0) during the ponytail audit, but engines.node still advertised >=20.0.0. On Node 20.0-20.11 parseEnv is undefined: loadClaudeMemEnv() silently returns {} (credentials never load) and saveClaudeMemEnv() throws. Declare the real floor in both the npm package and the generated plugin manifest (via its generator in build-hooks.js so rebuilds stay consistent). Greptile PR #3021 P1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
3759d99010 |
batch: land five reviewed fixes from @rodboev (#2848 #2850 #2851 #2852 #2860) (#2862)
* fix(context): suppress native Codex transcript AGENTS writes (closes #2837) * fix(sqlite): preserve legacy NULL observation hashes during migration 29 (closes #2812) * Keep Codex hook output within the supported schema (closes #2844) * Honor persisted worker request timeouts in MCP calls (closes #2822) * Limit Codex AGENTS suppression to native transcript watches * Preserve migrated hashes through legacy FK rebuilds * Keep settings-file defaults from inheriting stale env state * Keep welcome-hint settings overrides request-local * Keep Bun test state from masking settings fallbacks * Keep sticky test mocks from bleeding into later routes * Keep generated worker bundle aligned after baseline carry-over * Keep settings-file defaults from inheriting stale env state * Keep welcome-hint settings overrides request-local * Keep Bun test state from masking settings fallbacks * Keep sticky test mocks from bleeding into later routes * Keep generated hooks aligned after baseline carry-over * Keep settings-file defaults from inheriting stale env state * Keep welcome-hint settings overrides request-local * Keep Bun test state from masking settings fallbacks * Keep sticky test mocks from bleeding into later routes * Keep generated bundles aligned after baseline carry-over * Keep settings-file defaults from inheriting stale env state * Keep welcome-hint settings overrides request-local * Keep Bun test state from masking settings fallbacks * Keep sticky test mocks from bleeding into later routes * Keep generated scripts aligned after baseline carry-over * Keep worker recycle tests isolated from module cache * Preserve native Codex watch filtering while narrowing AGENTS suppression * Keep migration backfills out of user hash namespaces * fix(sqlite): bound stored prompt payloads to stop prompt-table bloat (closes #2793) * fix(sqlite): normalize prompt dedupe against the stored payload * Keep settings-file defaults from inheriting stale env state * Keep welcome-hint settings overrides request-local * Keep Bun test state from masking settings fallbacks * Keep sticky test mocks from bleeding into later routes * Keep worker recycle tests isolated from module cache * Keep baseline fallback fixes consistent with regenerated assets * Keep parser mode tests isolated and prompt storage observable * Keep parser mode tests isolated and prompt storage observable * Keep parser mode tests isolated and prompt storage observable * build: regenerate plugin bundles for combined batch * test: restore module mocks after codex-context processor tests The module-level mock of utils/project-name leaked into tests/utils/project-name.test.ts when the full suite ran in one process. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Rod Boev <rod.boev@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Glucksberg
|
667a5ae6b5 | Prevent duplicate user prompts (#2515) | ||
|
Alex Newman
|
61fe70a203 |
refactor: remove retry/observation queue, replace with SessionMessageBuffer
Removes the BullMQ-based observation queue engine, SessionQueueProcessor, PendingMessageStore, ObservationQueueEngine, and RestartGuard. Replaces the retry-queue path with an in-memory SessionMessageBuffer + confirmClaimedMessages contract on SessionManager. Net large deletion; observation pipeline no longer depends on a persistent retry queue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
36b0929fae |
Server-beta: Postgres storage + independent runtime + BullMQ queue (Phases 1–3) (#2351)
* Add server beta runtime foundation * Address server beta review findings * Resolve server beta review comments * Tighten server beta review follow-ups * Harden server beta auth and search * Avoid unnecessary FTS rebuilds * Block scoped keys from creating projects * Release BullMQ claims best effort on close * Address server beta review blockers * Reset BullMQ claims best effort * Add Postgres observation storage foundation * feat(server-beta): add independent runtime service Introduce src/server/runtime/ as a self-contained server-beta runtime that owns its lifecycle, Postgres bootstrap, and HTTP boundary without depending on WorkerService. ServerBetaService wraps the existing Server class, exposes /healthz and /v1/info with runtime="server-beta", and persists state to dedicated paths (.server-beta.pid|.port|.runtime.json). The four boundary managers (queue, generation worker, provider registry, event broadcaster) are intentionally disabled in this phase and report their status through /v1/info; later phases activate them. Adds plans/2026-05-07-finish-bullmq-branch-ship-plan.md to track the remaining work for this branch. Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): route CLI lifecycle and bundle separate runtime scripts/build-hooks.js now produces plugin/scripts/server-beta-service.cjs as a separate Node CJS bundle, alongside the existing worker-service bundle. The server-beta runtime is now installable independently. src/npx-cli/commands/server.ts routes start|stop|restart|status to the server-beta lifecycle instead of the legacy worker. The worker keeps its own start|stop|restart|status under the worker namespace; the two runtimes can be operated independently. src/services/worker-service.ts adds a server-* command parser branch that delegates to the sibling server-beta-service.cjs bundle so direct worker-service invocations still route to the right runtime. tests/npx-cli-server-namespace.test.ts updated to expect server-beta lifecycle routing. Includes rebuilt plugin/scripts/*.cjs bundles produced by build-and-sync. Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): add BullMQ job queue primitives Introduce src/server/jobs/ as the queue-side primitives that Phase 3 of the server-beta runtime needs to operate. types.ts defines a discriminated union over the four job kinds (event, event-batch, summary, reindex) and maps each to a per-kind BullMQ queue name and deterministic-ID prefix. job-id.ts builds deterministic, colon-free BullMQ jobIds from (kind, team, project, source). The colon ban exists because BullMQ uses ':' as a Redis key separator internally; embedding ':' in jobIds breaks scan and state lookups. ServerJobQueue.ts is a thin wrapper over BullMQ Queue + Worker that enforces autorun:false, default concurrency 1, and an attached error listener — all per BullMQ docs requirements. Test seams accept queue and worker factories so unit tests do not need Redis. outbox.ts publishes through the Postgres ObservationGenerationJob repository as canonical history. enqueueOutbox writes the row first, then publishes to BullMQ; if BullMQ throws, the row is transitioned to failed and a failed event is appended. reconcileOnStartup re-enqueues queued + processing rows after a restart, replacing terminal BullMQ jobs that may still be holding the deterministic ID slot. markCompleted and markFailed wrap transitionStatus and append the matching event row. Includes 20 unit tests covering deterministic ID stability, colon-free output, queue lifecycle, error-listener attachment, double-start refusal, idempotent enqueue, BullMQ failure rollback, startup reconciliation, max-attempts skipping, and completion / failure / retry transitions. Phase 3 commit 1 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server-beta): activate queue boundary in runtime service Wire ActiveServerBetaQueueManager into the server-beta runtime graph. The active manager owns one ServerJobQueue per generation kind (event, event-batch, summary, reindex) and surfaces lane metadata through boundary health. Selection is opt-in and fail-fast: if CLAUDE_MEM_QUEUE_ENGINE is set to bullmq the active manager is constructed (and any Redis/config error throws — no silent fallback to SQLite, per Phase 3 anti-pattern guard). For any other engine the disabled boundary remains so worker-era and test setups stay compatible. Widens ServerBetaBoundaryHealth.status to a discriminated union ('disabled' | 'active' | 'errored') with optional details. The disabled adapter still emits status='disabled', which keeps the existing server-beta-service test green. ServerBetaService receives the manager through a new optional queueManager field on CreateServerBetaServiceOptions so test graphs and Phase 4 wiring can inject custom managers. Adds tests/server/runtime/active-queue-manager.test.ts covering bullmq guard, active health shape, per-kind queue access, close behavior, and post-close errored health. Phase 3 commit 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(server-beta): cap /v1/events/batch at 500 events Prevents unbounded array DoS surface flagged in PR review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Alex Newman
|
65f2fd8cdd |
fix: harden startup and schema repair contracts
Reliability patch covering startup path resolution, install marker compatibility, export CLI request contracts, schema repair safety, hard-stop retry-loop handling, and the PR babysit status helper. |
||
|
Alex Newman
|
9e2973059a |
UX redesign: installer + provider rename + /learn-codebase + welcome card + SessionStart hint (#2255)
* feat(ux): claude-mem UX improvements with installer enhancements Squashed PR #2156 commits for clean rebase onto main: - feat(installer): add provider selection, model prompt, worker auto-start - refactor: rename *Agent provider classes to *Provider - feat: add /learn-codebase skill and viewer welcome card - feat(worker): inject welcome hint when project has zero observations - fix(pr-2156): address greptile review comments - fix(pr-2156): address coderabbit review comments - fix(pr-2156): persist CLAUDE_MEM_PROVIDER for non-claude in non-TTY mode - fix(pr-2156): file-backed settings reads in installer + env-first SKILL doc Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build: rebuild plugin artifacts after rebase onto v12.4.7 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(skills): strip claude-mem internals from learn-codebase The learn-codebase skill, install next-step copy, WelcomeCard, and welcome-hint previously walked the primary agent through worker endpoints and synthetic observation payloads. The PostToolUse hook already captures every Read/Edit the agent makes — the agent should have no awareness that the memory layer exists. Collapse the skill to one instruction ("read every source file in full") and rephrase touchpoints to describe only what the user observes (Claude reading files), not what happens behind the scenes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): preflight version mismatch + settings-aware port resolution Two related fixes for build-and-sync's worker restart step: 1. Read CLAUDE_MEM_WORKER_PORT from ~/.claude-mem/settings.json the same way the worker does, instead of computing the default port from the uid alone. Previously, users with a custom port saw a misleading "Worker not running" message because the restart POST hit the wrong port and got ECONNREFUSED. 2. Add a preflight check that aborts the sync when the running worker's reported version does not match the version we are about to build. Claude Code's plugin loader pins the worker to a specific cache version per session, so syncing into a newer cache directory has no effect until the user runs `claude plugin update thedotmack/claude-mem` to bump the pin. The preflight surfaces this explicitly with the exact command to run; --force bypasses it for intentional cases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(learn-codebase): note sed for partial reads of large files Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: strip comments codebase-wide Removed prose comments from all tracked source. Preserved directives (@ts-ignore, eslint-disable, biome-ignore, prettier-ignore, triple-slash references, webpack magic, shebangs). Deleted two tests that asserted on comment text rather than runtime behavior. Net: 401 files, -14,587 / +389 lines, -10.4% bytes. Verified: typecheck passes, build passes, test count unchanged from baseline (22 pre-existing fails, all unrelated). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(installer): move runtime setup into npx, eliminate hook dead air Smart-install ran 3 times during a fresh install — the worst run was silent, fired by Claude Code's Setup hook after `claude plugin install`, producing ~30s of dead air that looked like the plugin was hung. This change makes `npx claude-mem install` the single place heavy work happens, with a visible spinner. Hooks become runtime-only. - New `src/npx-cli/install/setup-runtime.ts` module: ensureBun, ensureUv, installPluginDependencies, read/writeInstallMarker, isInstallCurrent. Marker schema preserved exactly ({version, bun, uv, installedAt}) so ContextBuilder and BranchManager readers keep working. - `npx claude-mem install`: ungated copy/register/enable for every IDE, inserts a "Setting up runtime" task with honest "first install can take ~30s" spinner. The claude-code shell-out to `claude plugin install` is removed — npx already populated everything Claude reads. - New `npx claude-mem repair` command for post-`claude plugin update` recovery, force-reinstalls runtime. - Setup hook now runs `plugin/scripts/version-check.js` (29ms wall) instead of smart-install. Mismatch prints "run: npx claude-mem repair" on stderr. Always exits 0 (non-blocking, per CLAUDE.md exit-code strategy). - SessionStart loses the smart-install entry; 2 hooks remain (worker start, context fetch). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(installer): delete smart-install sources, retarget tests - Delete scripts/smart-install.js + plugin/scripts/smart-install.js (both are source files kept in sync manually; both must go). - Delete tests/smart-install.test.ts (covered surface is gone). - tests/plugin-scripts-line-endings: drop smart-install.js entry. - tests/infrastructure/plugin-distribution: retarget two assertions at version-check.js (the new Setup hook script). - New tests/setup-runtime.test.ts: 9 tests covering marker read/write, isInstallCurrent semantics. Marker schema invariant verified. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(installer): describe npx-driven setup + version-check Setup hook Sweep public docs and architecture notes to reflect the new flow: npx installer does Bun/uv setup with a visible spinner; Setup hook runs sub-100ms version-check.js; users hit `npx claude-mem repair` after a `claude plugin update`. - docs/architecture-overview.md: hook lifecycle table + npx flow paragraph - docs/public/configuration.mdx: tree + hook config example - docs/public/development.mdx: build output line - docs/public/hooks-architecture.mdx: full rewrite of pre-hook section, timing table, performance table - docs/public/architecture/{overview,hooks,worker-service}.mdx: tree comments, JSON config example, Bun requirement section docs/reports/* untouched (historical incident reports). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): mergeSettings writes via USER_SETTINGS_PATH Greptile P1 (#2156): `settingsFilePath()` only resolved `process.env.CLAUDE_MEM_DATA_DIR`, while `getSetting()` reads via `USER_SETTINGS_PATH` which `resolveDataDir()` populates from BOTH the env var AND a `CLAUDE_MEM_DATA_DIR` entry persisted in `~/.claude-mem/settings.json`. Result: a user with the data dir saved in settings.json but not exported in their shell would have provider/model settings silently written to `~/.claude-mem/settings.json` while `getSetting()` read from `/custom/path/settings.json` — read/write split. Drop `settingsFilePath()` and the now-unused `homedir` import; reuse the already-imported `USER_SETTINGS_PATH` constant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): parse --provider, --model, --no-auto-start install flags Greptile P1 (#2156): InstallOptions has fields `provider`, `model`, `noAutoStart`, but the install case in the npx-cli switch only parsed `--ide`. The other three flags were silently dropped — `npx claude-mem install --provider gemini` was a no-op. Extract a `parseInstallOptions(argv)` helper, share it between the bare `npx claude-mem` and `npx claude-mem install` paths, and validate `--provider` against the allowed set. Update help text accordingly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): pipe runtime-setup output, always show IDE multiselect Two issues caught in a docker test of the installer: 1. The bun.sh installer, uv installer, and `bun install` were using stdio: 'inherit', dumping their stdout/stderr through clack's spinner region — visible as raw "downloading uv 0.11.8…" / "Checked 58 installs across 38 packages…" text streaming under the spinner. Switch to stdio: 'pipe' and surface captured stderr only on failure (via a shared describeExecError() helper that includes stdout when stderr is empty). Spinner stays clean on the happy path. 2. promptForIDESelection() silently picked claude-code when no IDEs were detected, never showing the user the multiselect. On a fresh machine with no IDEs present yet (e.g. our docker test container), the user never got to choose. Now: always show the full IDE list when interactive; mark detected ones with [detected] hints and pre-select them; show a warn line if zero are detected explaining they should pick what they plan to use. Non-TTY callers still get the silent claude-code default at the call site (unchanged). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): skip marketplace work for claude-code-only, offer to install Claude Code Two related UX fixes from a docker test: **Delay between "Saved Claude model=…" and "Plugin files copied OK"** After dropping the needsManualInstall gate, every install was unconditionally running `copyPluginToMarketplace` (which copied the entire root node_modules tree — thousands of files, dozens of seconds) and `runNpmInstallInMarketplace` (npm install --production) even when only claude-code was selected. Neither is needed for claude-code: that path uses the plugin cache dir + the installed_plugins.json + enabledPlugins flag, all of which we already write. - Drop `node_modules` from `copyPluginToMarketplace`'s allowed-entries list; the dependency-install task populates it on the destination side anyway. - Re-introduce `needsMarketplace = selectedIDEs.some(id => id !== 'claude-code')` scoped *only* to `copyPluginToMarketplace`, `runNpmInstallInMarketplace`, and the pre-install `shutdownWorkerAndWait` (also pointless for claude-code- only flows since we're not overwriting the worker's running cache dir source). All other tasks (cache copy, register, enable, runtime setup) stay unconditional. **Claude Code missing → silent install of an IDE that isn't there** When the user picked claude-code on a machine without it (e.g. a fresh container), the install completed but `claude` was unavailable and the only hint was a generic warn line. Replace with an explicit pre-flight prompt: Claude Code is not installed. Claude-mem works best in Claude Code, but also works with the IDEs below. ? Install Claude Code now? ◆ Yes — install Claude Code (recommended) ◯ No — pick another IDE below ◯ Cancel installation If the user picks "Yes", run `curl -fsSL https://claude.ai/install.sh | bash` (or the PowerShell equivalent on Windows), then re-detect IDEs and proceed with claude-code pre-selected. If the install fails or the user picks "No", the multiselect still appears with claude-code visible (just unmarked [detected]), so they can opt in or pick another IDE. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install): detect Claude Code via `claude` CLI, not ~/.claude dir The directory `~/.claude` can exist (e.g. mounted in Docker, or created by tooling) without Claude Code actually being installed. Detect the `claude` command in PATH instead so the installer correctly offers to install Claude Code when missing. * docs(learn-codebase): add reviewer note explaining the cost tradeoff The skill intentionally reads every file in full to build a cognitive cache that pays off across the rest of the project. Add a brief note so reviewers (human or bot) understand the tradeoff before flagging the unbounded read as a cost issue. * fix: address Greptile P1 feedback on welcome hint and learn-codebase - SearchRoutes: skip welcome hint when caller passes ?full=true so explicit full-context requests aren't intercepted by the hint. - learn-codebase: replace `sed` instruction with the Read tool's offset/limit parameters, since Bash is gated in Claude Code by default. * feat(install): ASCII-animated logo splash on interactive install Plays a ~1s bloom animation of the claude-mem sunburst logomark when the installer starts in an interactive terminal — geometrically rendered via 12 ray curves around a center disc, in the brand orange. The wordmark and tagline type on alongside the final frame. Auto-skipped on non-TTY, in CI, when NO_COLOR or CLAUDE_MEM_NO_BANNER is set, or when the terminal is too narrow. Inspired by ghostty +boo. * feat(banner): replace rotation frames with angular-sector bloom generator Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): replace rotation frames with angular-sector bloom generator Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): three-act choreography renderer with radial gradient and diff redraw Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): update preview script to support small/medium/hero tier selection Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(docker): add COLORTERM=truecolor to test-installer sandbox Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(install): auto-apply PATH for Claude Code with spinner UX The Claude Code install.sh prints a Setup notes block telling users to manually edit "your shell config file" to add ~/.local/bin to PATH — which left fresh installs unable to launch claude from the command line. After a successful install, detect ~/.local/bin/claude on disk and, if the dir is missing from PATH, append the right export line to .zshrc / .bash_profile / .bashrc / fish config (idempotent, marked with a comment). Also updates process.env.PATH for the current install run. Wraps the curl|bash install in a clack spinner (interactive only) so the ~4 minute native-build download doesn't look frozen — output is captured silently and dumped on failure for debuggability. Non-interactive mode keeps inherited stdio for CI logs. Verified end-to-end in the test-installer docker sandbox: spinner animates, .bashrc gets the export, fresh login shell resolves claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): video-frame ASCII renderer with three-act choreography Generator switched from a single Jimp-rendered logo to pre-extracted video frames concatenated with \x01 separators and gzip-deflated, ported from ghostty's boo wire format. Renderer rewritten around three acts (ignite → stagger bloom → text reveal + breathe) with adaptive sizing, radial gradient, and diff-based redraw. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(onboarding): unify install / SessionStart / viewer around one first-success moment Three surfaces now point at the same north-star moment — open the viewer, do anything in Claude Code, watch an observation appear within seconds — with the same verbatim timing and privacy lines, and a single canonical "how it works" explainer instead of three diverging copies. - Canonical explainer at src/services/worker/onboarding-explainer.md served via GET /api/onboarding/explainer; mirrored into plugin/skills/how-it-works/SKILL.md - SessionStart welcome hint rewritten as third-person status (no imperatives Claude tries to execute), pinned with a default-value regression test - Post-install Next Steps reframed as "two paths": passive default + optional /learn-codebase front-load; drops /mem-search and /knowledge-agent from this surface; adds verbatim timing + privacy lines and /how-it-works link - /api/stats response gains firstObservationAt for the viewer stat row - Viewer WelcomeCard branches on observationCount === 0: empty state shows live worker-connection dot + "waiting for activity"; has-data state shows observations · projects · since [date] and two example prompts. v2 dismiss key - jimp added to package.json to fix pre-existing banner-frame build break Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(banner): play unconditionally; only honor CLAUDE_MEM_NO_BANNER The 128-col / TTY / CI / NO_COLOR gates silently swallowed the banner in narrower terminals, CI logs, and any non-TTY pipe — including Docker runs where -it should preserve the experience but column width was the wrong gate. Remove the implicit gates; keep the explicit opt-out only. If a frame wraps in a narrow terminal, that's better than the banner not playing at all. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * revert(banner): restore 15:33 gating logic per user request Reverts |
||
|
Alex Newman
|
94d592f212 |
perf: streamline worker startup and consolidate database connections (#2122)
* docs: pathfinder refactor corpus + Node 20 preflight
Adds the PATHFINDER-2026-04-22 principle-driven refactor plan (11 docs,
cross-checked PASS) plus the exploratory PATHFINDER-2026-04-21 corpus
that motivated it. Bumps engines.node to >=20.0.0 per the ingestion-path
plan preflight (recursive fs.watch). Adds the pathfinder skill.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: land PATHFINDER Plan 01 — data integrity
Schema, UNIQUE constraints, self-healing claim, Chroma upsert fallback.
- Phase 1: fresh schema.sql regenerated at post-refactor shape.
- Phase 2: migrations 23+24 — rebuild pending_messages without
started_processing_at_epoch; UNIQUE(session_id, tool_use_id);
UNIQUE(memory_session_id, content_hash) on observations; dedup
duplicate rows before adding indexes.
- Phase 3: claimNextMessage rewritten to self-healing query using
worker_pid NOT IN live_worker_pids; STALE_PROCESSING_THRESHOLD_MS
and the 60-s stale-reset block deleted.
- Phase 4: DEDUP_WINDOW_MS and findDuplicateObservation deleted;
observations.insert now uses ON CONFLICT DO NOTHING.
- Phase 5: failed-message purge block deleted from worker-service
2-min interval; clearFailedOlderThan method deleted.
- Phase 6: repairMalformedSchema and its Python subprocess repair
path deleted from Database.ts; SQLite errors now propagate.
- Phase 7: Chroma delete-then-add fallback gated behind
CHROMA_SYNC_FALLBACK_ON_CONFLICT env flag as bridge until
Chroma MCP ships native upsert.
- Phase 8: migration 19 no-op block absorbed into fresh schema.sql.
Verification greps all return 0 matches. bun test tests/sqlite/
passes 63/63. bun run build succeeds.
Plan: PATHFINDER-2026-04-22/01-data-integrity.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: land PATHFINDER Plan 02 — process lifecycle
OS process groups replace hand-rolled reapers. Worker runs until
killed; orphans are prevented by detached spawn + kill(-pgid).
- Phase 1: src/services/worker/ProcessRegistry.ts DELETED. The
canonical registry at src/supervisor/process-registry.ts is the
sole survivor; SDK spawn site consolidated into it via new
createSdkSpawnFactory/spawnSdkProcess/getSdkProcessForSession/
ensureSdkProcessExit/waitForSlot helpers.
- Phase 2: SDK children spawn with detached:true + stdio:
['ignore','pipe','pipe']; pgid recorded on ManagedProcessInfo.
- Phase 3: shutdown.ts signalProcess teardown uses
process.kill(-pgid, signal) on Unix when pgid is recorded;
Windows path unchanged (tree-kill/taskkill).
- Phase 4: all reaper intervals deleted — startOrphanReaper call,
staleSessionReaperInterval setInterval (including the co-located
WAL checkpoint — SQLite's built-in wal_autocheckpoint handles
WAL growth without an app-level timer), killIdleDaemonChildren,
killSystemOrphans, reapOrphanedProcesses, reapStaleSessions, and
detectStaleGenerator. MAX_GENERATOR_IDLE_MS and MAX_SESSION_IDLE_MS
constants deleted.
- Phase 5: abandonedTimer — already 0 matches; primary-path cleanup
via generatorPromise.finally() already lives in worker-service
startSessionProcessor and SessionRoutes ensureGeneratorRunning.
- Phase 6: evictIdlestSession and its evict callback deleted from
SessionManager. Pool admission gates backpressure upstream.
- Phase 7: SDK-failure fallback — SessionManager has zero matches
for fallbackAgent/Gemini/OpenRouter. Failures surface to hooks
via exit code 2 through SessionRoutes error mapping.
- Phase 8: ensureWorkerRunning in worker-utils.ts rewritten to
lazy-spawn — consults isWorkerPortAlive (which gates
captureProcessStartToken for PID-reuse safety via commit
|
||
|
Alex Newman
|
6250a194dd |
Merge branch 'pr-1472' into integration/validation-batch
# Conflicts: # plugin/scripts/context-generator.cjs # plugin/scripts/mcp-server.cjs # plugin/scripts/worker-service.cjs # plugin/ui/viewer-bundle.js # src/cli/handlers/context.ts # src/services/sqlite/SessionStore.ts # src/services/sqlite/migrations/runner.ts # src/services/worker-service.ts # src/shared/SettingsDefaultsManager.ts |
||
|
79475432@qq.com
|
9cfa57d498 |
fix: use null-byte delimiter in observation content hash to prevent collisions
Fields concatenated without separators allowed different tuples to produce identical hashes (e.g. session="ab", title="cd" vs session="abc", title="d"). This could cause legitimate observations to be silently deduplicated. Join with \x00 so field boundaries are unambiguous. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
huakson
|
4f6fb9e614 |
fix: address platform source review feedback
Tighten platform source persistence so legacy callers cannot silently relabel existing sessions, repair migration 24 when schema_versions drifts from the real schema, and polish the follow-up UI/error-handler review nits. - only backfill platform_source when it is blank and raise on explicit source conflicts for an existing session - make migration 24 verify both the sdk_sessions column and its index before treating it as applied - expose platform_source from the functional session getters and add regression tests for source preservation and schema drift recovery - add the required APPROVED OVERRIDE annotation for centralized HTTP error translation - keep mobile source pills on a single horizontal row |
||
|
Alex Newman
|
c6f932988a |
Fix 30+ root-cause bugs across 10 triage phases (#1214)
* MAESTRO: fix ChromaDB core issues — Python pinning, Windows paths, disable toggle, metadata sanitization, transport errors - Add --python version pinning to uvx args in both local and remote mode (fixes #1196, #1206, #1208) - Convert backslash paths to forward slashes for --data-dir on Windows (fixes #1199) - Add CLAUDE_MEM_CHROMA_ENABLED setting for SQLite-only fallback mode (fixes #707) - Sanitize metadata in addDocuments() to filter null/undefined/empty values (fixes #1183, #1188) - Wrap callTool() in try/catch for transport errors with auto-reconnect (fixes #1162) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix data integrity — content-hash deduplication, project name collision, empty project guard, stuck isProcessing - Add SHA-256 content-hash deduplication to observations INSERT (store.ts, transactions.ts, SessionStore.ts) - Add content_hash column via migration 22 with backfill and index - Fix project name collision: getCurrentProjectName() now returns parent/basename - Guard against empty project string with cwd-derived fallback - Fix stuck isProcessing: hasAnyPendingWork() resets processing messages older than 5 minutes - Add 12 new tests covering all four fixes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix hook lifecycle — stderr suppression, output isolation, conversation pollution prevention - Suppress process.stderr.write in hookCommand() to prevent Claude Code showing diagnostic output as error UI (#1181). Restores stderr in finally block for worker-continues case. - Convert console.error() to logger.warn()/error() in hook-command.ts and handlers/index.ts so all diagnostics route to log file instead of stderr. - Verified all 7 handlers return suppressOutput: true (prevents conversation pollution #598, #784). - Verified session-complete is a recognized event type (fixes #984). - Verified unknown event types return no-op handler with exit 0 (graceful degradation). - Added 10 new tests in tests/hook-lifecycle.test.ts covering event dispatch, adapter defaults, stderr suppression, and standard response constants. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix worker lifecycle — restart loop coordination, stale transport retry, ENOENT shutdown race - Add PID file mtime guard to prevent concurrent restart storms (#1145): isPidFileRecent() + touchPidFile() coordinate across sessions - Add transparent retry in ChromaMcpManager.callTool() on transport error — reconnects and retries once instead of failing (#1131) - Wrap getInstalledPluginVersion() with ENOENT/EBUSY handling (#1042) - Verified ChromaMcpManager.stop() already called on all shutdown paths Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix Windows platform support — uvx.cmd spawn, PowerShell $_ elimination, windowsHide, FTS5 fallback - Route uvx spawn through cmd.exe /c on Windows since MCP SDK lacks shell:true (#1190, #1192, #1199) - Replace all PowerShell Where-Object {$_} pipelines with WQL -Filter server-side filtering (#1024, #1062) - Add windowsHide: true to all exec/spawn calls missing it to prevent console popups (#1048) - Add FTS5 runtime probe with graceful fallback when unavailable on Windows (#791) - Guard FTS5 table creation in migrations, SessionSearch, and SessionStore with try/catch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix skills/ distribution — build-time verification and regression tests (#1187) Add post-build verification in build-hooks.js that fails if critical distribution files (skills, hooks, plugin manifest) are missing. Add 10 regression tests covering skill file presence, YAML frontmatter, hooks.json integrity, and package.json files field. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix MigrationRunner schema initialization (#979) — version conflict between parallel migration systems Root cause: old DatabaseManager migrations 1-7 shared schema_versions table with MigrationRunner's 4-22, causing version number collisions (5=drop tables vs add column, 6=FTS5 vs prompt tracking, 7=discovery_tokens vs remove UNIQUE). initializeSchema() was gated behind maxApplied===0, so core tables were never created when old versions were present. Fixes: - initializeSchema() always creates core tables via CREATE TABLE IF NOT EXISTS - Migrations 5-7 check actual DB state (columns/constraints) not just version tracking - Crash-safe temp table rebuilds (DROP IF EXISTS _new before CREATE) - Added missing migration 21 (ON UPDATE CASCADE) to MigrationRunner - Added ON UPDATE CASCADE to FK definitions in initializeSchema() - All changes applied to both runner.ts and SessionStore.ts Tests: 13 new tests in migration-runner.test.ts covering fresh DB, idempotency, version conflicts, crash recovery, FK constraints, and data integrity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix 21 test failures — stale mocks, outdated assertions, missing OpenClaw guards Server tests (12): Added missing workerPath and getAiStatus to ServerOptions mocks after interface expansion. ChromaSync tests (3): Updated to verify transport cleanup in ChromaMcpManager after architecture refactor. OpenClaw (2): Added memory_ tool skipping and response truncation to prevent recursive loops and oversized payloads. MarkdownFormatter (2): Updated assertions to match current output. SettingsDefaultsManager (1): Used correct default key for getBool test. Logger standards (1): Excluded CLI transcript command from background service check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix Codex CLI compatibility (#744) — session_id fallbacks, unknown platform tolerance, undefined guard Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix Cursor IDE integration (#838, #1049) — adapter field fallbacks, tolerant session-init validation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix /api/logs OOM (#1203) — tail-read replaces full-file readFileSync Replace readFileSync (loads entire file into memory) with readLastLines() that reads only from the end of the file in expanding chunks (64KB → 10MB cap). Prevents OOM on large log files while preserving the same API response shape. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix Settings CORS error (#1029) — explicit methods and allowedHeaders in CORS config Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: add session custom_title for agent attribution (#1213) — migration 23, endpoint + store support Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: prevent CLAUDE.md/AGENTS.md writes inside .git/ directories (#1165) Add .git path guard to all 4 write sites to prevent ref corruption when paths resolve inside .git internals. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix plugin disabled state not respected (#781) — early exit check in all hook entry points Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix UserPromptSubmit context re-injection on every turn (#1079) — contextInjected session flag Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * MAESTRO: fix stale AbortController queue stall (#1099) — lastGeneratorActivity tracking + 30s timeout Three-layer fix: 1. Added lastGeneratorActivity timestamp to ActiveSession, updated by processAgentResponse (all agents), getMessageIterator (queue yields), and startGeneratorWithProvider (generator launch) 2. Added stale generator detection in ensureGeneratorRunning — if no activity for >30s, aborts stale controller, resets state, restarts 3. Added AbortSignal.timeout(30000) in deleteSession to prevent indefinite hang when awaiting a stuck generator promise Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Alex Newman
|
4df9f61347 |
refactor: implement in-process worker architecture for hooks (#722)
* fix: stop generating empty CLAUDE.md files - Return empty string instead of "No recent activity" when no observations exist - Skip writing CLAUDE.md files when formatted content is empty - Remove redundant "auto-generated by claude-mem" HTML comment - Clean up 98 existing empty CLAUDE.md files across the codebase - Update tests to expect empty string for empty input Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * build assets * refactor: implement in-process worker architecture for hooks Replaces spawn-based worker startup with in-process architecture: - Hook processes now become the worker when port 37777 is free - Eliminates Windows spawn issues (NO SPAWN rule) - SessionStart chains: smart-install && stop && context Key changes: - worker-service.ts: hook case starts WorkerService in-process - hook-command.ts: skipExit option prevents process.exit() when hosting worker - hooks.json: single chained command replaces separate start/hook commands - worker-utils.ts: ensureWorkerRunning() returns boolean, doesn't block - handlers: graceful fallback when worker unavailable All 761 tests pass. Manual verification confirms hook stays alive as worker. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * context * a * MAESTRO: Mark PR #722 test verification task complete All 797 tests passed (3 skipped, 0 failed) after merge conflict resolution. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * MAESTRO: Mark PR #722 build verification task complete * MAESTRO: Mark PR #722 code review task complete Code review verified: - worker-service.ts hook case starts WorkerService in-process - hook-command.ts has skipExit option - hooks.json uses single chained command - worker-utils.ts ensureWorkerRunning() returns boolean Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * MAESTRO: Mark PR #722 conflict resolution push task complete Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Alex Newman
|
e1ab73decc |
feat: Live Context System with Distributed CLAUDE.md Generation (#556)
* docs: add folder index generator plan RFC for auto-generating folder-level CLAUDE.md files with observation timelines. Includes IDE symlink support and root CLAUDE.md integration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: implement folder index generator (Phase 1) Add automatic CLAUDE.md generation for folders containing observed files. This enables IDE context providers to access relevant memory observations. Core modules: - FolderDiscovery: Extract folders from observation file paths - FolderTimelineCompiler: Compile chronological timeline per folder - ClaudeMdGenerator: Write CLAUDE.md with tag-based content replacement - FolderIndexOrchestrator: Coordinate regeneration on observation save Integration: - Event-driven regeneration after observation save in ResponseProcessor - HTTP endpoints for folder discovery, timeline, and manual generation - Settings for enabling/configuring folder index behavior The <claude-mem-context> tag wrapping ensures: - Manual CLAUDE.md content is preserved - Auto-generated content won't be recursively observed - Clean separation between user and system content 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add updateFolderClaudeMd function to CursorHooksInstaller Adds function to update CLAUDE.md files for folders touched by observations. Uses existing /api/search/by-file endpoint, preserves content outside <claude-mem-context> tags, and writes atomically via temp file + rename. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: hook updateFolderClaudeMd into ResponseProcessor Calls updateFolderClaudeMd after observation save to update folder-level CLAUDE.md files. Uses fire-and-forget pattern with error logging. Extracts file paths from saved observations and workspace path from registry. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add timeline formatting for folder CLAUDE.md files Implements formatTimelineForClaudeMd function that transforms API response into compact markdown table format. Converts emojis to text labels, handles ditto marks for timestamps, and groups under "Recent" header. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: remove old folder-index implementation Deletes redundant folder-index services that were replaced by the simpler updateFolderClaudeMd approach in CursorHooksInstaller.ts. Removed: - src/services/folder-index/ directory (5 files) - FolderIndexRoutes.ts - folder-index settings from SettingsDefaultsManager - folder-index route registration from worker-service 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add worktree-aware project filtering for unified timelines Detect git worktrees and show both parent repo and worktree observations in the session start timeline. When running in a worktree, the context now includes observations from both projects, interleaved chronologically. - Add detectWorktree() utility to identify worktree directories - Add getProjectContext() to return parent + worktree projects - Update context hook to pass multi-project queries - Add queryObservationsMulti() and querySummariesMulti() for IN clauses - Maintain backward compatibility with single-project queries 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * fix: restructure logging to prove session correctness and reduce noise Add critical logging at each stage of the session lifecycle to prove the session ID chain (contentSessionId → sessionDbId → memorySessionId) stays aligned. New logs include CREATED, ENQUEUED, CLAIMED, MEMORY_ID_CAPTURED, STORING, and STORED. Move intermediate migration and backfill progress logs to DEBUG level to reduce noise, keeping only essential initialization and completion logs at INFO level. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * refactor: extract folder CLAUDE.md utils to shared location Moves folder CLAUDE.md utilities from CursorHooksInstaller to a new shared utils file. Removes Cursor registry dependency - file paths from observations are already absolute, no workspace lookup needed. New file: src/utils/claude-md-utils.ts - replaceTaggedContent() - preserves user content outside tags - writeClaudeMdToFolder() - atomic writes with tag preservation - formatTimelineForClaudeMd() - API response to compact markdown - updateFolderClaudeMdFiles() - orchestrates folder updates 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: trigger folder CLAUDE.md updates when observations are saved The folder CLAUDE.md update was previously only triggered in syncAndBroadcastSummary, but summaries run with observationCount=0 (observations are saved separately). Moved the update logic to syncAndBroadcastObservations where file paths are available. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all the claudes * test: add unit tests for claude-md-utils pure functions Add 11 tests covering replaceTaggedContent and formatTimelineForClaudeMd: - replaceTaggedContent: empty content, tag replacement, appending, partial tags - formatTimelineForClaudeMd: empty input, parsing, ditto marks, session IDs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add integration tests for file operation functions Add 9 tests for writeClaudeMdToFolder and updateFolderClaudeMdFiles: - writeClaudeMdToFolder: folder creation, content preservation, nested dirs, atomic writes - updateFolderClaudeMdFiles: empty skip, fetch/write, deduplication, error handling 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add unit tests for timeline-formatting utilities Add 14 tests for extractFirstFile and groupByDate functions: - extractFirstFile: relative paths, fallback to files_read, null handling, invalid JSON - groupByDate: empty arrays, date grouping, chronological sorting, item preservation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: rebuild plugin scripts with merged features * docs: add project-specific CLAUDE.md with architecture and development notes * fix: exclude project root from auto-generated CLAUDE.md updates Skip folders containing .git directory when auto-updating subfolder CLAUDE.md files. This ensures: 1. Root CLAUDE.md remains user-managed and untouched by the system 2. SessionStart context injection stays pristine throughout the session 3. Subfolder CLAUDE.md files continue to receive live context updates 4. Cleaner separation between user-authored root docs and auto-generated folder indexes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: prevent crash from resuming stale SDK sessions on worker restart When the worker restarts, it was incorrectly passing the `resume` parameter to INIT prompts (lastPromptNumber=1) when a memorySessionId existed from a previous SDK session. This caused "Claude Code process exited with code 1" crashes because the SDK tried to resume into a session that no longer exists. Root cause: The resume condition only checked `hasRealMemorySessionId` but did not verify that this was a CONTINUATION prompt (lastPromptNumber > 1). Fix: Add `session.lastPromptNumber > 1` check to the resume condition: - Before: `...(hasRealMemorySessionId && { resume: session.memorySessionId })` - After: `...(hasRealMemorySessionId && session.lastPromptNumber > 1 && { resume: ... })` Also added: - Enhanced debug logging that warns when skipping resume for INIT prompts - Unit tests in tests/sdk-agent-resume.test.ts (9 test cases) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: properly handle Chroma MCP connection errors Previously, ensureCollection() caught ALL errors from chroma_get_collection_info and assumed they meant "collection doesn't exist", triggering unnecessary collection creation attempts. Connection errors like "Not connected" or "MCP error -32000: Connection closed" would cascade into failed creation attempts. Similarly, queryChroma() would silently return empty results when the MCP call failed, masking the underlying connection problem. Changes: - ensureCollection(): Detect connection errors and re-throw immediately instead of attempting collection creation - queryChroma(): Wrap MCP call in try-catch and throw connection errors instead of returning empty results - Both methods reset connection state (connected=false, client=null) on connection errors so subsequent operations can attempt to reconnect 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * pushed * fix: scope regenerate-claude-md.ts to current working directory Critical bug fix: The script was querying ALL observations from the database across ALL projects ever recorded (1396+ folders), then attempting to write CLAUDE.md files everywhere including other projects, non-existent paths, and ignored directories. Changes: - Use git ls-files to discover folders (respects .gitignore automatically) - Filter database query to current project only (by folder name) - Use relative paths for database queries (matches storage format) - Add --clean flag to remove auto-generated CLAUDE.md files - Add fallback directory walker for non-git repos Now correctly scopes to 26 folders with observations instead of 1396+. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs and adjustments * fix: cleanup mode strips tags instead of deleting files blindly The cleanup mode was incorrectly deleting entire files that contained <claude-mem-context> tags. The correct behavior (per original design): 1. Strip the <claude-mem-context>...</claude-mem-context> section 2. If empty after stripping → delete the file 3. If has remaining content → save the stripped version Now properly preserves user content in CLAUDE.md files while removing only the auto-generated sections. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * deleted some files * chore: regenerate folder CLAUDE.md files with fixed script Regenerated 23 folder CLAUDE.md files using the corrected script that: - Scopes to current working directory only - Uses git ls-files to respect .gitignore - Filters by project name 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Update CLAUDE.md files for January 5, 2026 - Regenerated and staged 23 CLAUDE.md files with a mix of new and modified content. - Fixed cleanup mode to properly strip tags instead of deleting files blindly. - Cleaned up empty CLAUDE.md files from various directories, including ~/.claude and ~/Scripts. - Conducted dry-run cleanup that identified a significant reduction in auto-generated CLAUDE.md files. - Removed the isAutoGeneratedClaudeMd function due to incorrect file deletion behavior. * feat: use settings for observation limit in batch regeneration script Replace hard-coded limit of 10 with configurable CLAUDE_MEM_CONTEXT_OBSERVATIONS setting (default: 50). This allows users to control how many observations appear in folder CLAUDE.md files. Changes: - Import SettingsDefaultsManager and load settings at script startup - Use OBSERVATION_LIMIT constant derived from settings at both call sites - Remove stale default parameter from findObservationsByFolder function 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: use settings for observation limit in event-driven updates Replace hard-coded limit of 10 in updateFolderClaudeMdFiles with configurable CLAUDE_MEM_CONTEXT_OBSERVATIONS setting (default: 50). Changes: - Import SettingsDefaultsManager and os module - Load settings at function start (once, not in loop) - Use limit from settings in API call 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: Implement configurable observation limits and enhance search functionality - Added configurable observation limits to batch regeneration scripts. - Enhanced SearchManager to handle folder queries and normalize parameters. - Introduced methods to check for direct child files in observations and sessions. - Updated SearchOptions interface to include isFolder flag for filtering. - Improved code quality with comprehensive reviews and anti-pattern checks. - Cleaned up auto-generated CLAUDE.md files across various directories. - Documented recent changes and improvements in CLAUDE.md files. * build asset * Project Context from Claude-Mem auto-added (can be auto removed at any time) * CLAUDE.md updates * fix: resolve CLAUDE.md files to correct directory in worktree setups When using git worktrees, CLAUDE.md files were being written relative to the worker's process.cwd() instead of the actual project directory. This fix threads the project's cwd from message processing through to the file writing utilities, ensuring CLAUDE.md files are created in the correct project directory regardless of where the worker was started. Changes: - Add projectRoot parameter to updateFolderClaudeMdFiles for path resolution - Thread projectRoot through ResponseProcessor call chain - Track lastCwd from messages in SDKAgent, GeminiAgent, OpenRouterAgent - Add tests for relative/absolute path handling with projectRoot 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * more project context updates * context updates * fix: preserve actual dates in folder CLAUDE.md generation Previously, formatTimelineForClaudeMd used today's date for all observations because the API only returned time (e.g., "4:30 PM") without date information. This caused all historical observations to appear as if they happened today. Changes: - SearchManager.findByFile now groups results by date with headers (e.g., "### Jan 4, 2026") matching formatSearchResults behavior - formatTimelineForClaudeMd now parses these date headers and uses the correct date when constructing epochs for date grouping The timeline dates are critical for claude-mem context - LLMs need accurate temporal context to understand when work happened. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * build: update worker assets with date parsing fix 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * claude-mem context: Fixed critical date parsing bug in PR #556 * fix: address PR #556 review items - Use getWorkerHost() instead of hard-coded 127.0.0.1 in claude-md-utils - Add error message and stack details to FOLDER_INDEX logging - Add 5 new tests for worktree/projectRoot path resolution 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Refactor CLAUDE documentation across multiple components and tests - Updated CLAUDE.md files in src/ui/viewer, src/ui/viewer/constants, src/ui/viewer/hooks, tests/server, tests/worker/agents, and plans to reflect recent changes and improvements. - Removed outdated entries and consolidated recent activities for clarity. - Enhanced documentation for hooks, settings, and pagination implementations. - Streamlined test suite documentation for server and worker agents, indicating recent test audits and cleanup efforts. - Adjusted plans to remove obsolete entries and focus on current implementation strategies. * docs: comprehensive v9.0 documentation audit and updates - Add usage/folder-context to docs.json navigation (was documented but hidden!) - Update introduction.mdx with v9.0 release notes (Live Context, Worktree Support, Windows Fixes) - Add CLAUDE_MEM_WORKER_HOST setting to configuration.mdx - Add Folder Context Files section with link to detailed docs - Document worktree support in folder-context.mdx - Update terminology from "mem-search skill" to "MCP tools" throughout active docs - Update Search Pipeline in architecture/overview.mdx - Update usage/getting-started.mdx with MCP tools terminology - Update usage/claude-desktop.mdx title and terminology - Update hooks-architecture.mdx reference 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add recent activity log for worker CLI with detailed entries * chore: update CLAUDE.md context files 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add brainstorming report for CLAUDE.md distribution architecture --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Alex Newman
|
f21ea97c39 |
refactor: decompose monolith into modular architecture with comprehensive test suite (#538)
* fix: prevent memory_session_id from equaling content_session_id The bug: memory_session_id was initialized to contentSessionId as a "placeholder for FK purposes". This caused the SDK resume logic to inject memory agent messages into the USER's Claude Code transcript, corrupting their conversation history. Root cause: - SessionStore.createSDKSession initialized memory_session_id = contentSessionId - SDKAgent checked memorySessionId !== contentSessionId but this check only worked if the session was fetched fresh from DB The fix: - SessionStore: Initialize memory_session_id as NULL, not contentSessionId - SDKAgent: Simple truthy check !!session.memorySessionId (NULL = fresh start) - Database migration: Ran UPDATE to set memory_session_id = NULL for 1807 existing sessions that had the bug Also adds [ALIGNMENT] logging across the session lifecycle to help debug session continuity issues: - Hook entry: contentSessionId + promptNumber - DB lookup: contentSessionId → memorySessionId mapping proof - Resume decision: shows which memorySessionId will be used for resume - Capture: logs when memorySessionId is captured from first SDK response UI: Added "Alignment" quick filter button in LogsModal to show only alignment logs for debugging session continuity. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: improve error handling in worker-service.ts - Fix GENERIC_CATCH anti-patterns by logging full error objects instead of just messages - Add [ANTI-PATTERN IGNORED] markers for legitimate cases (cleanup, hot paths) - Simplify error handling comments to be more concise - Improve httpShutdown() error discrimination for ECONNREFUSED - Reduce LARGE_TRY_BLOCK issues in initialization code Part of anti-pattern cleanup plan (132 total issues) * refactor: improve error logging in SearchManager.ts - Pass full error objects to logger instead of just error.message - Fixes PARTIAL_ERROR_LOGGING anti-patterns (10 instances) - Better debugging visibility when Chroma queries fail Part of anti-pattern cleanup (133 remaining) * refactor: improve error logging across SessionStore and mcp-server - SessionStore.ts: Fix error logging in column rename utility - mcp-server.ts: Log full error objects instead of just error.message - Improve error handling in Worker API calls and tool execution Part of anti-pattern cleanup (133 remaining) * Refactor hooks to streamline error handling and loading states - Simplified error handling in useContextPreview by removing try-catch and directly checking response status. - Refactored usePagination to eliminate try-catch, improving readability and maintaining error handling through response checks. - Cleaned up useSSE by removing unnecessary try-catch around JSON parsing, ensuring clarity in message handling. - Enhanced useSettings by streamlining the saving process, removing try-catch, and directly checking the result for success. * refactor: add error handling back to SearchManager Chroma calls - Wrap queryChroma calls in try-catch to prevent generator crashes - Log Chroma errors as warnings and fall back gracefully - Fixes generator failures when Chroma has issues - Part of anti-pattern cleanup recovery * feat: Add generator failure investigation report and observation duplication regression report - Created a comprehensive investigation report detailing the root cause of generator failures during anti-pattern cleanup, including the impact, investigation process, and implemented fixes. - Documented the critical regression causing observation duplication due to race conditions in the SDK agent, outlining symptoms, root cause analysis, and proposed fixes. * fix: address PR #528 review comments - atomic cleanup and detector improvements This commit addresses critical review feedback from PR #528: ## 1. Atomic Message Cleanup (Fix Race Condition) **Problem**: SessionRoutes.ts generator error handler had race condition - Queried messages then marked failed in loop - If crash during loop → partial marking → inconsistent state **Solution**: - Added `markSessionMessagesFailed()` to PendingMessageStore.ts - Single atomic UPDATE statement replaces loop - Follows existing pattern from `resetProcessingToPending()` **Files**: - src/services/sqlite/PendingMessageStore.ts (new method) - src/services/worker/http/routes/SessionRoutes.ts (use new method) ## 2. Anti-Pattern Detector Improvements **Problem**: Detector didn't recognize logger.failure() method - Lines 212 & 335 already included "failure" - Lines 112-113 (PARTIAL_ERROR_LOGGING detection) did not **Solution**: Updated regex patterns to include "failure" for consistency **Files**: - scripts/anti-pattern-test/detect-error-handling-antipatterns.ts ## 3. Documentation **PR Comment**: Added clarification on memory_session_id fix location - Points to SessionStore.ts:1155 - Explains why NULL initialization prevents message injection bug ## Review Response Addresses "Must Address Before Merge" items from review: ✅ Clarified memory_session_id bug fix location (via PR comment) ✅ Made generator error handler message cleanup atomic ❌ Deferred comprehensive test suite to follow-up PR (keeps PR focused) ## Testing - Build passes with no errors - Anti-pattern detector runs successfully - Atomic cleanup follows proven pattern from existing methods 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix: FOREIGN KEY constraint and missing failed_at_epoch column Two critical bugs fixed: 1. Missing failed_at_epoch column in pending_messages table - Added migration 20 to create the column - Fixes error when trying to mark messages as failed 2. FOREIGN KEY constraint failed when storing observations - All three agents (SDK, Gemini, OpenRouter) were passing session.contentSessionId instead of session.memorySessionId - storeObservationsAndMarkComplete expects memorySessionId - Added null check and clear error message However, observations still not saving - see investigation report. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * Refactor hook input parsing to improve error handling - Added a nested try-catch block in new-hook.ts, save-hook.ts, and summary-hook.ts to handle JSON parsing errors more gracefully. - Replaced direct error throwing with logging of the error details using logger.error. - Ensured that the process exits cleanly after handling input in all three hooks. * docs: add monolith refactor report with system breakdown Comprehensive analysis of codebase identifying: - 14 files over 500 lines requiring refactoring - 3 critical monoliths (SessionStore, SearchManager, worker-service) - 80% code duplication across agent files - 5-phase refactoring roadmap with domain-based architecture * docs: update monolith report post session-logging merge - SessionStore grew to 2,011 lines (49 methods) - highest priority - SearchManager reduced to 1,778 lines (improved) - Agent files reduced by ~45 lines combined - Added trend indicators and post-merge observations - Core refactoring proposal remains valid * refactor(sqlite): decompose SessionStore into modular architecture Extract the 2011-line SessionStore.ts monolith into focused, single-responsibility modules following grep-optimized progressive disclosure pattern: New module structure: - sessions/ - Session creation and retrieval (create.ts, get.ts, types.ts) - observations/ - Observation storage and queries (store.ts, get.ts, recent.ts, files.ts, types.ts) - summaries/ - Summary storage and queries (store.ts, get.ts, recent.ts, types.ts) - prompts/ - User prompt management (store.ts, get.ts, types.ts) - timeline/ - Cross-entity timeline queries (queries.ts) - import/ - Bulk import operations (bulk.ts) - migrations/ - Database migrations (runner.ts) New coordinator files: - Database.ts - ClaudeMemDatabase class with re-exports - transactions.ts - Atomic cross-entity transactions - Named re-export facades (Sessions.ts, Observations.ts, etc.) Key design decisions: - All functions take `db: Database` as first parameter (functional style) - Named re-exports instead of index.ts for grep-friendliness - SessionStore retained as backward-compatible wrapper - Target file size: 50-150 lines (60% compliance) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(agents): extract shared logic into modular architecture Consolidate duplicate code across SDKAgent, GeminiAgent, and OpenRouterAgent into focused utility modules. Total reduction: 500 lines (29%). New modules in src/services/worker/agents/: - ResponseProcessor.ts: Atomic DB transactions, Chroma sync, SSE broadcast - ObservationBroadcaster.ts: SSE event formatting and dispatch - SessionCleanupHelper.ts: Session state cleanup and stuck message reset - FallbackErrorHandler.ts: Provider error detection for fallback logic - types.ts: Shared interfaces (WorkerRef, SSE payloads, StorageResult) Bug fix: SDKAgent was incorrectly using obs.files instead of obs.files_read and hardcoding files_modified to empty array. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(search): extract search strategies into modular architecture Decompose SearchManager into focused strategy pattern with: - SearchOrchestrator: Coordinates strategy selection and fallback - ChromaSearchStrategy: Vector semantic search via ChromaDB - SQLiteSearchStrategy: Filter-only queries for date/project/type - HybridSearchStrategy: Metadata filtering + semantic ranking - ResultFormatter: Markdown table formatting for results - TimelineBuilder: Chronological timeline construction - Filter modules: DateFilter, ProjectFilter, TypeFilter SearchManager now delegates to new infrastructure while maintaining full backward compatibility with existing public API. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(context): decompose context-generator into modular architecture Extract 660-line monolith into focused components: - ContextBuilder: Main orchestrator (~160 lines) - ContextConfigLoader: Configuration loading - TokenCalculator: Token budget calculations - ObservationCompiler: Data retrieval and query building - MarkdownFormatter/ColorFormatter: Output formatting - Section renderers: Header, Timeline, Summary, Footer Maintains full backward compatibility - context-generator.ts now delegates to new ContextBuilder while preserving public API. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(worker): decompose worker-service into modular infrastructure Split 2000+ line monolith into focused modules: Infrastructure: - ProcessManager: PID files, signal handlers, child process cleanup - HealthMonitor: Port checks, health polling, version matching - GracefulShutdown: Coordinated cleanup on exit Server: - Server: Express app setup, core routes, route registration - Middleware: Re-exports from existing middleware - ErrorHandler: Centralized error handling with AppError class Integrations: - CursorHooksInstaller: Full Cursor IDE integration (registry, hooks, MCP) WorkerService now acts as thin coordinator wiring all components together. Maintains full backward compatibility with existing public API. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Refactor session queue processing and database interactions - Implement claim-and-delete pattern in SessionQueueProcessor to simplify message handling and eliminate duplicate processing. - Update PendingMessageStore to support atomic claim-and-delete operations, removing the need for intermediate processing states. - Introduce storeObservations method in SessionStore for simplified observation and summary storage without message tracking. - Remove deprecated methods and clean up session state management in worker agents. - Adjust response processing to accommodate new storage patterns, ensuring atomic transactions for observations and summaries. - Remove unnecessary reset logic for stuck messages due to the new queue handling approach. * Add duplicate observation cleanup script Script to clean up duplicate observations created by the batching bug where observations were stored once per message ID instead of once per observation. Includes safety checks to always keep at least one copy. Usage: bun scripts/cleanup-duplicates.ts # Dry run bun scripts/cleanup-duplicates.ts --execute # Delete duplicates bun scripts/cleanup-duplicates.ts --aggressive # Ignore time window 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(sqlite): add comprehensive test suite for SQLite repositories Add 44 tests across 5 test files covering: - Sessions: CRUD operations and schema validation - Observations: creation, retrieval, filtering, and ordering - Prompts: persistence and association with observations - Summaries: generation tracking and session linkage - Transactions: context management and rollback behavior 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(worker): add comprehensive test suites for worker agent modules Add test coverage for response-processor, observation-broadcaster, session-cleanup-helper, and fallback-error-handler agents. Fix type import issues across search module (use `import type` for type-only imports) and update worker-service main module detection for ESM/CJS compatibility. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(search): add comprehensive test suites for search module Add test coverage for the refactored search architecture: - SearchOrchestrator: query coordination and caching - ResultFormatter: pagination, sorting, and field mapping - SQLiteSearchStrategy: database search operations - ChromaSearchStrategy: vector similarity search - HybridSearchStrategy: combined search with score fusion 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(context): add comprehensive test suites for context-generator modules Add test coverage for the modular context-generator architecture: - context-builder.test.ts: Tests for context building and result assembly - observation-compiler.test.ts: Tests for observation compilation with privacy tags - token-calculator.test.ts: Tests for token budget calculations - formatters/markdown-formatter.test.ts: Tests for markdown output formatting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(infrastructure): add comprehensive test suites for worker infrastructure modules Add test coverage for graceful-shutdown, health-monitor, and process-manager modules extracted during the worker-service refactoring. All 32 tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(server): add comprehensive test suites for server modules Add test coverage for Express server infrastructure: - error-handler.test.ts: Tests error handling middleware including validation errors, database errors, and async error handling - server.test.ts: Tests server initialization, middleware configuration, and route mounting for all API endpoints 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore(package): add test scripts for modular test suites Add npm run scripts to simplify running tests: - test: run all tests - test:sqlite, test:agents, test:search, test:context, test:infra, test:server 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * build assets * feat(tests): add detailed failure analysis reports for session ID refactor, validation, and store tests - Created reports for session ID refactor test failures, highlighting 8 failures due to design mismatches. - Added session ID usage validation report detailing 10 failures caused by outdated assumptions in tests. - Documented session store test failures, focusing on foreign key constraint violations in 2 tests. - Compiled a comprehensive test suite report summarizing overall test results, including 28 failing tests across various categories. * fix(tests): align session ID tests with NULL-based initialization Update test expectations to match implementation where memory_session_id starts as NULL (not equal to contentSessionId) per architecture decision that memory_session_id must NEVER equal contentSessionId. Changes: - session_id_refactor.test.ts: expect NULL initial state, add updateMemorySessionId() calls - session_id_usage_validation.test.ts: update placeholder detection to check !== null - session_store.test.ts: add updateMemorySessionId() before storeObservation/storeSummary 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(tests): update GeminiAgent tests with correct field names and mocks - Rename deprecated fields: claudeSessionId → contentSessionId, sdkSessionId → memorySessionId, pendingProcessingIds → pendingMessages - Add missing required ActiveSession fields - Add storeObservations mock (plural) for ResponseProcessor compatibility - Fix settings mock to use correct CLAUDE_MEM_GEMINI_RATE_LIMITING_ENABLED key - Add await to rejects.toThrow assertion 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(tests): add logger imports and fix coverage test exclusions Phase 3 of test suite fixes: - Add logger imports to 34 high-priority source files (SQLite, worker, context) - Exclude CLI-facing files from console.log check (worker-service.ts, integrations/*Installer.ts) as they use console.log intentionally for interactive user output 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: update SESSION_ID_ARCHITECTURE for NULL-based initialization Update documentation to reflect that memory_session_id starts as NULL, not as a placeholder equal to contentSessionId. This matches the implementation decision that memory_session_id must NEVER equal contentSessionId to prevent injecting memory messages into user transcripts. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore(deps): update esbuild and MCP SDK - esbuild: 0.25.12 → 0.27.2 (fixes minifyIdentifiers issue) - @modelcontextprotocol/sdk: 1.20.1 → 1.25.1 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * build assets and updates * chore: remove bun.lock and add to gitignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |