skills/thedotmack__claude-mem
1
0
Fork 0
mirror of https://github.com/thedotmack/claude-mem.git synced 2026-09-20 04:23:02 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,669 Commits 597 Branches 365 Tags
cursor/fix-localhost-worker-host-1789
Commit Graph

154 Commits

Author SHA1 Message Date
Alex Newman 9841dd4120 chore: bump version to 13.24.11 (#4004) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
 2026-09-10 22:25:56 -07:00
Alex Newman 6f9a73c8ca chore: bump version to 13.24.10 (#3993) 
Package-facing patch so 13.24.10 ships #3991 (content flush batch 200→40 + requestTimeout settings), which landed on main after v13.24.9 was tagged.
 2026-09-10 21:20:08 -07:00
Alex Newman 47c06ff9a1 fix(observer): surface quota cooldown on observer-health and session-start (#3986) 
Quota cooldown is an intentional pause, not a failure, so observer-health
stayed green and the session-start banner stayed silent while the worker
queued work. Mirror the breaker into observer-health.json and show a
pause notice on the existing health/banner path without changing
queue-accept or drain-on-clear.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
 2026-09-10 20:49:31 -07:00
Cursor Agent 4890e14c4b chore: bump version to 13.24.7 
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
 2026-09-11 01:05:17 +00:00
Cursor Agent 9811b0eb78 chore: bump version to 13.24.6 
Co-authored-by: Alex Newman <thedotmack@users.noreply.github.com>
 2026-09-11 00:27:49 +00:00
Alex Newman fd0ecf0233 feat(sqlite): durable tool_uses backup index + get_tool_uses disclosure (#3898) 
Raw tool I/O had no durable home. `pending_messages` is the generation
queue -- rows are claimed, summarized, and deleted -- so once an
observation existed the original tool_input/tool_response were gone.

Adds `tool_uses` (schema v51) as a by-reference side index for those
bodies, written from the one ingest choke point both the PostToolUse hook
route and the transcript-watch processor already share. The JSONL
transcripts and `src/services/transcripts/*` remain the spine and are
untouched.

Schema (Receipt freeze 2026-09-06): UNIQUE(content_session_id,
tool_use_id), nullable `or_generation_id` / `or_session_id` as join keys
back to an OpenRouter spend line, and deliberately no cost_usd/micros --
this table carries tool identity, dollars stay on the OR stamp. No FK on
session_db_id/observation_id: a FK there can abort the constructor
migration chain (#3378), and observation_id is linked late by design.

Write path dual-writes alongside -- never instead of -- the
pending_messages enqueue, and swallows its own failures so an observation
is never lost to a backup-index error. `toolUseId` now actually reaches
the worker from hooks: it was missing from NormalizedHookInput and every
hook adapter, so only the transcript path supplied it. ResponseProcessor
links the batch's claimed ids to the first stored observation.

Read path is progressive-disclosure layer 4: POST /api/tool-uses/batch
requires explicit ids (never a full-table scan), GET /api/tool-uses
returns a cheap index shape with size hints and never the payloads, and
the `get_tool_uses` MCP tool is described as a last resort. claude-mem's
own read tools are skipped by the writer -- without that, every call to
get_tool_uses would store the bodies it just returned.

Payloads are stored in-row with a 64 KB soft cap and a UTF-8-safe
truncation marker; content_hash is computed over the original.

Join contract for Receipt: RECEIPT-JOIN.md


Claude-Session: https://claude.ai/code/session_01QgdJ6gExgBirDoAxknt94m

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-06 18:25:26 -07:00
Alex Newman 60310f86bf fix(quota): claim the cmem gateway re-probe instead of reading the clock 
The gateway fallback said "allow one normal gateway request as a probe", but
nothing made it one. shouldUseCmemFallback is a subtraction and a less-than,
and ensureGeneratorRunning gates per session, so 28-69 live sessions are 28-69
independent callers.

It is worse than the in-memory case. The marker lives on disk
(CLAUDE_MEM_PRO_FALLBACK_AT in settings.json, loaded uncached), so every
process parses the same ISO string and computes the same expiry instant -- the
herd was handed a shared clock to synchronize on. Each member's failure then
does a read-modify-write of the user's whole settings.json to re-arm the
marker; the write is atomic but the sequence is not, so concurrent settings
edits can be clobbered.

The re-probe is now claimed through the breaker's existing machinery under a
DISTINCT key, 'cmem-gateway'. Distinct because tryAdmitQuotaProbe takes the
cooldown per call: this path's 15 minutes against the provider breaker's 30
would otherwise let two callers reach contradictory answers about whether the
same breaker is armed, and provider switches (which this branch makes routine)
make the collision reachable in both directions -- a fresh funded key born into
an armed breaker and withheld for a refusal it never earned.

getSelectedProvider stays read-only for diagnostics and status; only the new
selectProviderForGenerator claims. Every path out of ensureGeneratorRunning
that does not actually send releases the claim, including the one where a
generator is already running, so a claim cannot outlive the call that took it.

Also regenerates the plugin bundles rather than hand-merging them, which is
what the five .cjs conflicts in this consolidation always needed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWmJDjFM6BNmGaLJ7Xbrsy
 2026-08-31 12:18:30 -07:00
Alex Newman 2319050541 fix(installer): harden signed-in provider transitions 2026-08-26 17:27:20 -07:00
Alex Newman 90aa9a1ebc build: regenerate plugin bundles 2026-08-19 01:38:44 -07:00
Alex Newman 3bf1914503 fix(worker): carry the gateway error envelope end-to-end — stop retrying quota, log once, tell the user what to do (#3601) 
* docs: add observer error-path implementation plan

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BPptieMC1vzUNGtUErU3UM

* fix(worker): carry the gateway error envelope, stop retrying quota errors, log once

The OpenRouter classifier discarded the upstream body (which contains the
remedy — e.g. "Key limit exceeded … Manage it using <url>"), treated 402 as
unrecoverable, never used the request id it was handed, and each failure
fanned out to five log lines plus two pointless retries.

- provider-errors.ts: ClassifiedProviderError gains optional code/action/url/
  requestId; describeProviderError() is the single renderer
  (message — action url (req id)).
- OpenRouterProvider.ts: classifyOpenRouterError parses the cmem.ai gateway
  envelope {code,message,action,url,request_id} first (six taxonomy codes →
  worker kinds), otherwise legacy classification now includes the upstream
  body in every message, maps 402 and "key limit exceeded"/"limit exceeded"/
  "negative credit" bodies to quota_exhausted (bare "limit exceeded" excluded
  on 429 so real rate limits stay retryable), and sets requestId. The
  200-with-error path now forwards bodyText + requestId.
- error-classification.ts (server mirror): same marker list, no imports.
- Log once: init/message-loop/handleSessionError log at debug for classified
  errors; SessionRoutes emits one "Observer failed" error line with kind/code/
  requestId and the rendered message string (no errorSink re-fire).
- Tests: envelope + legacy cases assert message/code/action/url/requestId;
  new retry-policy.test.ts pins isRetryableKind; server mirror pinned.

Plan: plans/2026-08-16-observer-error-path.md (Phase 2)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BPptieMC1vzUNGtUErU3UM

* feat(worker): observer-health ledger and session-start warning carry the structured error

Users don't read worker logs; they open Claude Code. The session-start
health warning from #3538 said "check ~/.claude-mem/settings.json", which is
wrong for a Pro user whose allowance is exhausted.

- observer-health.ts: ObserverHealthState gains lastErrorCode/Action/Url/
  RequestId (null by default; old ledgers read back unchanged);
  recordObserverFailure accepts string | ObserverFailureDetail — objects
  populate the new fields (message/action scrubbed, url/requestId as-is);
  renderObserverHealthWarning emits "What to do:", "Link:", "Request id:"
  and drops the generic settings.json remedy when an action exists.
  scrubErrorMessage also masks cm_pro_ tokens.
- SessionRoutes.ts: classified errors record {message, code, action, url,
  requestId} in the ledger — the same words as the "Observer failed" log line.
- tests/observer-health.test.ts: round-trip, backward compat, render with/
  without action, scrub.

Plan: plans/2026-08-16-observer-error-path.md (Phase 3)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BPptieMC1vzUNGtUErU3UM

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-16 11:25:07 -07:00
Alex Newman a38d51169b feat(worker): alert the user at session start when observations stop flowing (#3538) 
* feat(worker): surface observer outages at session start via observer-health ledger

Generator failures now record to a file-backed ledger
(~/.claude-mem/observer-health.json); successful observation stores reset
it. Session-start context injection prepends a loud outage warning (with
the provider's last error, which often embeds the remedy URL) once 3+
consecutive failures are newer than the last success — including on the
empty-state and missing-DB paths. Prevents a repeat of the silent
17-hour 2026-08-09 provider-quota outage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(observer-health): lock the ledger, widen credential redaction, plain-language warning

Greptile P1s on #3538:

- Credential redaction leaked provider secrets into injected context.
  scrubErrorMessage only caught `sk-…` and `Bearer …`, so `api_key=…`,
  JSON `"apiKey": "…"`, `Authorization: Basic …` and query-string secrets
  were persisted in observer-health.json and rendered verbatim at session
  start. Redaction now covers the common key/value, JSON, authorization and
  query-string shapes, skips purely numeric values so `max_tokens: 200000`
  style diagnostics survive, and runs again at render time so a ledger
  written by an older build cannot leak.

- Concurrent failure writes could suppress the outage warning. The ledger's
  read-modify-write was unlocked, so simultaneous writers overwrote each
  other's increments and consecutiveFailures stayed under the threshold.
  Serialized with a `wx` lockfile (same idiom as worker-spawn-gate.ts),
  stale-breaking at 5s and failing open after a 2s wait — a racy count beats
  no count. Regression test drives three synchronized writer processes and
  requires the threshold to be reached; it fails ~2 in 3 runs without the lock.

Also rewrites the warning for humans (it is read by the user, not just the
agent) with a describeDuration helper, and rebuilds the plugin artifacts —
the committed context-generator.cjs was stale and shipped without the
observer-health injection.

* test(worker): restore the real ModeManager after session-manager-project mocks it

bun's mock.module is process-global and mock.restore() does not undo it, so
this file's partial ModeManager stub (no class prototype, no loadMode) leaked
into every later test file in the run. When readdir order put it before
tests/server/server-boot.test.ts and the server-runtime smoke, loadServerMode()
hit `modeManager.loadMode is not a function` and CI went red — which is exactly
what happened on this PR while main stayed green.

Applies the snapshot + afterAll re-register pattern already used in
agent-formatter, SearchManager.timeline-anchor, and response-processor tests;
session-manager-project was the last ModeManager mock without it.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-16 10:42:12 -07:00
Alex Newman 9326fae06a feat: add custom mode creator skill (#3487) 2026-08-03 14:24:10 -07:00
Alex Newman ce921328db chore: bump version to 13.13.0 2026-08-02 10:14:45 -07:00
Alex Newman 2db6a1f506 build: regenerate plugin bundles from merged sources 
Bundle conflicts during the PR merge sweep were resolved to main's side;
this rebuild brings every shipped artifact back in sync with src/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RSNZmfi4vSYjTeEWtwqKrW
 2026-07-22 23:57:24 -07:00
Alex Newman b3eafc82b9 feat(sync): complete SyncHub-only client cutover 
Retire legacy client adoption and per-kind sync state, establish the one-time launch boundary, and expose authenticated Hub health. Add bounded device metadata and a loopback-only two-client protocol-v2 matrix E2E.
 2026-07-20 09:30:22 -07:00
Alex Newman 1e3b2853ff feat(sync): add canonical v2 projection pipeline 2026-07-20 05:29:49 -07:00
Alex Newman d3bf40434f chore(build): regenerate plugin bundle for advisory websocket 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-18 03:51:10 -07:00
Alex Newman b77f38a1fd chore(build): regenerate plugin bundle for two-lane sync 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-18 03:02:52 -07:00
Alex Newman d03392f8da build: rebuild plugin bundles with fixed prompt mapper and v40 self-repair 
Regenerated from src including 7c8c6ebc7 (joined drain pushes real
memory_session_id/project on prompts) and 5af1b059b (v40 requeue +
backfill lane). Deployed live: v40 re-queued all 20,226 local prompts
and the worker drained them to zero through the backfill lane with no
errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-12 13:08:00 -07:00
Alex Newman c275b340c8 fix(worker): graft worker-script resolver identity + IPv6 + worktree gitdir 
Grafted from the two plan branches, minus their regressions:
- worker-utils: cache-first resolveWorkerScriptPath so hook/MCP/CLI/restart
  launches converge on one worker bundle (stops cache-vs-marketplace version
  skew); + CLAUDE_MEM_WORKER_SCRIPT_PATH override; + formatHostForUrl IPv6
  bracketing in buildWorkerUrl.
- HealthMonitor: IPv6-bracket the worker host in health/readiness fetches.
- worktree: resolve a relative gitdir against the .git file's dir before
  matching worktrees/, so a relative 'gitdir:' entry is not misread as a
  non-worktree.

Excluded: c78b8bf8's stale-socket recovery (dead code — SO_REUSEADDR means
TIME_WAIT never raises EADDRINUSE) and the fd1b5221 submodule re-keying.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-05 15:14:02 -07:00
Alex Newman 22e4325ce2 fix: harden sqlite migrations and settings writes 
(cherry picked from commit b20024b772)
 2026-07-05 15:06:27 -07:00
Alex Newman 2a0a407340 chore: remove over-engineering findings from repo-wide ponytail audit (wave 1) 
Cuts verified dead code, duplication, and speculative abstractions across
14 disjoint areas of the tree: sqlite layer, worker HTTP routes, search
pipeline, server, chroma sync, viewer UI, npx-cli, MCP server, shared/utils,
telemetry/infra, integrations, scripts, tests, and stale plans/evals docs.

Also unifies the Chroma search pipeline onto SearchOrchestrator/strategies
and ports dual-project (merged_into_project) scoping plus dateRange
filtering into ChromaSearchStrategy, which corpus builds need but had
silently lost.

Full audit trail: 65-agent verification workflow wf_160a7862-1a6, 14-agent
execution wf_c7e48c0f-164, 6-agent fixup wf_865f86a4-c6b.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-07-02 20:49:27 -07:00
Alex Newman 39dd77d9e3 fix: eliminate all 331 error-handling anti-patterns detected by scanner (#3119) 
Drive scripts/anti-pattern-test/detect-error-handling-antipatterns.ts to
zero issues across the entire src/ tree (331 -> 0):

- NO_LOGGING_IN_CATCH (124): every silent catch now logs the full error
  object via logger/console, or carries an explicit documented override
- PROMISE_EMPTY_CATCH (6): empty .catch(() => {}) handlers in the SDK
  cleanup paths now log the swallowed cleanup failure
- GENERIC_CATCH (78): catches discriminate error types via instanceof
  Error normalization instead of blind `as Error` casts
- LARGE_TRY_BLOCK (115): oversized try blocks narrowed to the throwing
  statements or extracted into named helpers, semantics-preserving
- CATCH_AND_CONTINUE_CRITICAL_PATH (6) + ERROR_STRING_MATCHING (2):
  fixed or documented with [ANTI-PATTERN IGNORED] justifications

297 real fixes, 38 documented overrides (each with a specific technical
reason and explicit recovery). No behavior changes: swallowed errors
remain non-throwing, only visible now.

Verified: detector exits 0, tsc --noEmit clean (both tsconfigs),
2292 tests pass / 0 fail. Includes rebuilt plugin artifacts.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-02 13:09:09 -07:00
Alex Newman 29af028403 fix(providers): remove client-side context truncation ("second system") 
The OpenAICompatibleProvider.truncateHistory() sliding-window dropped
conversation messages based on a hardcoded 20-message cap and a 100k
"safety" token limit — a client-side context manager layered on top of
the provider's own context window. In practice it fired on message count
alone (dropping messages at ~12k tokens, nowhere near the token limit)
and silently corrupted conversation history, mislabeled as "runaway cost"
prevention.

Rip it out entirely. The full history is now sent to the provider, which
owns its own context window.

Removed:
- truncateHistory() + requireNonEmptyToTruncate from OpenAICompatibleProvider
- truncateHistoryForOpenRouter / truncateHistoryForGemini wrappers + constants
- CLAUDE_MEM_{GEMINI,OPENROUTER}_MAX_CONTEXT_MESSAGES / _MAX_TOKENS settings,
  their defaults, and SettingsRoutes validation
- truncation-specific tests; docs + openclaw installer references

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-30 20:52:54 -07:00
Alex Newman aad0d87c55 Merge remote-tracking branch 'origin/main' into release/recovery-2026-06-24 
# Conflicts:
#	plugin/scripts/context-generator.cjs
#	plugin/scripts/mcp-server.cjs
#	plugin/scripts/server-service.cjs
#	plugin/scripts/transcript-watcher.cjs
#	plugin/scripts/worker-service.cjs
#	src/cli/handlers/session-init.ts
#	src/servers/mcp-server.ts
#	src/services/sync/ChromaSync.ts
#	tests/hooks/server-client.test.ts
 2026-06-29 16:01:54 -07:00
Alex Newman 348d9ee406 feat(recovery): scope memories by platform source 2026-06-29 15:24:59 -07:00
Alex Newman 108c5c65a1 Merge remote-tracking branch 'origin/main' into mutual-aardvark 
# Conflicts:
#	plugin/scripts/context-generator.cjs
#	plugin/scripts/mcp-server.cjs
#	plugin/scripts/server-service.cjs
#	plugin/scripts/transcript-watcher.cjs
#	plugin/scripts/worker-service.cjs
#	plugin/ui/viewer-bundle.js
#	src/server/routes/v1/ServerV1PostgresRoutes.ts
#	src/server/runtime/ServerService.ts
#	src/server/runtime/create-server-service.ts
#	src/server/runtime/types.ts
#	tests/server/server-service.test.ts
 2026-06-29 15:10:51 -07:00
Alex Newman 8702c8f331 fix(chroma): prewarm uvx and harden shutdown 2026-06-29 10:04:51 -07:00
Alex Newman 4557920565 Merge remote-tracking branch 'origin/main' into mutual-aardvark 
# Conflicts:
#	package.json
#	plugin/scripts/context-generator.cjs
#	plugin/scripts/mcp-server.cjs
#	plugin/scripts/server-service.cjs
#	plugin/scripts/worker-service.cjs
#	plugin/ui/viewer-bundle.js
#	src/server/runtime/create-server-service.ts
#	tests/sdk/parse-summary.test.ts
#	tests/sdk/parser.test.ts
 2026-06-27 21:20:51 -07:00
Alex Newman edc5cf7d59 Ponytail audit (−10.4k lines) + worker-restart hardening + deterministic dependency closure (#3021) 
* plan-10 Phase 1: ship deterministic plugin runtime dependency closure

Approach A — commit & ship plugin/bun.lock so the plugin's runtime
node_modules install is deterministic, fixing the recurring
`Cannot find module 'zod/v3'` (#2730).

- align generated plugin zod range to root (^4.4.3) in build-hooks.js
- new scripts/gen-plugin-lockfile.cjs generates plugin/bun.lock as a
  build artifact after build-hooks.js writes plugin/package.json
- track & ship plugin/bun.lock (.gitignore negation, .npmignore, files allowlist)
- install with `bun install --frozen-lockfile --ignore-scripts` at runtime

Refs #2783, #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10 Phase 2: fail loud at install time on a broken dependency closure

Strengthen verifyCriticalModules to assert each dependency is actually
importable via require.resolve (not merely a directory), and assert the
worker-required zod subpaths resolve: zod/v3, zod/v4, zod/v4-mini.
A partial/stale install now fails `npx claude-mem install` immediately
instead of surfacing later as a Stop-hook `Cannot find module 'zod/v3'`.

Bin-only packages (e.g. tree-sitter-cli, which has no bare-name entry
point) fall back to resolving <dep>/package.json so a healthy install
isn't falsely rejected.

Adds tests/cli/verify-critical-modules.test.ts covering a missing zod/v3
subpath (throws), a complete zod (passes), and a bin-only dep (passes).

Refs #2783, #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10 Phase 3: clean-room install + import smoke test (#2730 backstop)

Add scripts/smoke-clean-room.cjs and a `smoke:clean-room` npm script.
Against fresh temp dirs (never the repo's node_modules) it:
- copies plugin/, runs `bun install --frozen-lockfile --ignore-scripts`,
  asserts zod, zod/v3, zod/v4, zod/v4-mini resolve, and boots the bundled
  worker asserting no `Cannot find module` — the direct #2730 regression guard;
- `npm pack`s, installs the tarball into a second temp dir, and load-tests
  the published bin entrypoint, warning loudly on any declared main/exports
  target missing from the tarball (latent #2537 gap).

Exits non-zero naming the missing module on any failure; cleans up all
temp dirs and the tarball in a finally.

Refs #2783, #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10 Phase 4: gate CI and publish on the clean-room dependency closure

- ci.yml: new `clean-room-deps` job (between build and the docker e2e job)
  runs a frozen-lockfile drift check on the committed plugin lockfile, then
  `npm run build` + `npm run smoke:clean-room`. The drift step catches a
  contributor who changed plugin deps without regenerating plugin/bun.lock.
- npm-publish.yml: add setup-bun and run `npm run smoke:clean-room` between
  build and `npm publish`, so a broken runtime closure cannot be published
  on a tag push (ci.yml does not run on tags). Secrets block untouched.

Refs #2783, #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10: doc recluster note + Phase 0 execution slice for #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plans: backlog recluster (2026-06-04) — cross-cluster execution order + plan-13 doc

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10: gen-plugin-lockfile degrades gracefully when bun is absent

The Windows build CI job has no bun on PATH; regenerating the lockfile there
threw and failed the build. The committed plugin/bun.lock is already the
deterministic closure, so skip regeneration (non-fatal) when bun is missing
and a lockfile exists; fail loud only when neither is available.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: rebuild plugin artifacts after merging main (v13.5.1) + plan-10 work

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: rebuild plugin artifacts after merging main v13.5.5

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(deps): daily upgrade pass — agent SDK 0.3.172, better-auth 1.6.16, posthog-node 5.36.15, dompurify 3.4.9

- Bump @anthropic-ai/claude-agent-sdk 0.2.141 -> 0.3.172 (tsc + full test suite green)
- Remove deprecated @types/dompurify stub (dompurify ships its own types)
- Add overrides.tmp ^0.2.7 to clear GHSA-52f5-9888-hmc6 / GHSA-ph9p-34f9-6g65
  via np -> listr-input -> inquirer -> external-editor -> tmp chain
- npm audit: 0 vulnerabilities; npm outdated: clean
- package-lock.json is gitignored in this repo, so not committed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* plan: worker-restart single-source-of-truth — 7-phase fix for restart races

Phased plan from the adversarially-verified diagnosis (wf_f07f3541-b05):
kill the cache mirror, single verified restart initiator, self-replacing
restart endpoint, unified spawn gate with lockfile, PID-file demotion,
test data-dir isolation, soak verification.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(restart): delete sync-script cache-mirror and HTTP restart trigger

Phase 1 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
The installed-version cache mirror wrote version-N code into the
version-(N-1) cache dir, manufacturing permanent version disagreement;
the HTTP POST to /api/admin/restart raced the CLI restart that follows
it in build-and-sync. Both are deleted; the CLI worker:restart in the
marketplace copy is now the single restart initiator, and the sleep 1
between the two mechanisms is gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(restart): restart proves itself or exits 1

Phase 2 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
worker-service restart now captures the old worker pid, waits for the
port with the same platform-scaled 15s budget as stop, spawns the
marketplace copy of worker-service.cjs when present, then polls
/api/health until the pid changes and the version matches this build's
baked __DEFAULT_PACKAGE_VERSION__ — success is printed to stdout,
deadline (platform-scaled 30s) exits 1 with the last observed health
payload and the spawned script path. The --daemon generic start-failure
path now exits 1 instead of masquerading as success; the three
duplicate-suppression exits remain 0.

New helper src/services/restart-verify.ts (worker-service.ts bootstraps
on import, so the helper lives in an import-safe module) with 8 tests
covering pid-flip success, stale pid, wrong version, unreachable
timeout, 503-degraded acceptance, and null-oldPid version-only
verification.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(restart): self-replacing worker — old worker spawns its successor

Phase 3 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
/api/admin/restart was kill-only: hooks that POSTed it then raced the
dying worker with their own lazy-spawn (the observed recycle ping-pong).
Now the dying worker spawns its successor itself — after a re-entrancy-
guarded, deadline-bounded (platform-scaled 10s) graceful shutdown, and
only once its port is confirmed free; stop and signal shutdowns stay
kill-only. The hook recycle path waits for that successor via
/api/health polling (HOOK_READINESS_TIMEOUT_MS budget) and lazy-spawns
only as a fallback, with a warn-only version re-check so a hook never
recycles more than once per invocation.

Shutdown sequence lives in import-safe src/services/worker-shutdown.ts
(worker-service.ts bootstraps on import); registerSignalHandlers no
longer pre-sets isShuttingDown — the supervisor's shutdownInitiated
guard owns signal dedupe, and pre-setting would no-op the new entry
guard. 13 new tests cover re-entrancy, deadline expiry/rejection,
handoff ordering, kill-only reasons, successor-wait vs lazy-spawn
fallback, and pre-graceful bookkeeping failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(restart): one spawn gate; CLI restart defers to the self-replacing worker

Phase 4 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
Three uncoordinated spawn paths (hook lazy-spawn, MCP worker-spawner,
CLI) with two different bun resolvers produced 3-launcher collisions
within a single second. Now a wx-flag lockfile (<DATA_DIR>/spawn.lock,
60s mtime staleness with re-stat-before-unlink, owner-checked release)
gates every external spawn: lock losers never fail — they skip the
spawn and wait for the winner's worker. resolveBunRuntime is deleted in
favor of ProcessManager's resolveWorkerRuntimePath (adds BUN_PATH,
~/.bun/bin, brew, which fallbacks), closing the kill-then-can't-respawn
path; mcp-server prefers the marketplace worker script so stale cache
dirs stop spawning stale workers.

Integration fix surfaced by live verification: the CLI restart raced
the Phase 3 self-replacement handoff (the successor re-binds the port
in ~200ms, so waitForPortFree always timed out and restart exited 1
while the restart had actually succeeded). The CLI now verifies the
worker's self-spawned successor directly, and only spawns — gate-
wrapped, after the port frees — as the fallback when no worker was
running, the shutdown POST was rejected, or no successor appeared. The
dying worker's handoff is intentionally ungated: it spawns only after
its own port closes, and hooks wait on it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(restart): demote the PID file — health and port are the liveness oracle

Phase 5 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
The dying worker's shutdown cascade deleted the PID file
unconditionally as its final act, clobbering the successor's
freshly-written file; status then required portInUse AND pidInfo, so a
healthy worker reported as "not running". Now every PID-file deletion
is owner-guarded: the supervisor cascade deletes only its own pid
(removeOwnedPidFile), and the CLI stop/restart-fallback, the restart
handoff, and the daemon start-failure cleanup go through
removePidFileIfOwner (owner-or-dead — a live successor's file always
survives; corrupt files are left for the next boot's validator).

status sources from GET /api/health alone (pid, version, uptime,
workerPath; 503-degraded counts as running and now surfaces its queue
detail), with port-in-use-but-unreachable and not-running fallbacks —
all exit 0 as before. The --daemon duplicate gate checks the port
first (ground truth) and the PID file second (advisory, for the
freed-port-but-undeleted-file window); duplicate suppression stays
exit 0. writePidFile/touchPidFile remain — the file is diagnostics,
and the worker stays its only writer.

Also fixes combined-run test pollution: spawn-gate and worker-utils
timeout tests now eagerly import paths.js before setting a temp
CLAUDE_MEM_DATA_DIR, so the import-time DATA_DIR const can't freeze on
a deleted temp dir for suites loaded later in the same bun process.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: no test ever touches the real ~/.claude-mem again

Phase 6 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
process-manager and graceful-shutdown tests wrote corrupt JSON and
sentinel PIDs (2147483647) into the real ~/.claude-mem/worker.pid and
drove the real supervisor.json cascade under a snapshot-restore that a
killed run would skip — that pollution contaminated production logs and
a prior diagnosis. Both files now set a temp CLAUDE_MEM_DATA_DIR at the
top of the file before dynamically importing the code under test (ESM
hoisting makes beforeEach too late), assert their paths landed outside
the real dir, and derive PID_FILE from the same frozen paths module the
code uses so test and code can never diverge under bun's shared module
cache. The snapshot-restore scaffolding is deleted; zero assertions
changed.

tests/preload.ts gains a tripwire: when CLAUDE_MEM_DATA_DIR is unset it
fills a per-run temp dir, so no test in any file can fall through to
the real data dir. Fallout made explicit: worker-spawn child processes
get an explicit temp dir; install-error-matrix restores rather than
deletes the env var; settings-defaults-manager pins the unset-env
default it was implicitly relying on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(settings): bootstrap notices go to stderr, never stdout

CI on PR #2894 caught the latent bug: on the first boot in a fresh data
dir, SettingsDefaultsManager printed '[SETTINGS] Created settings file
with defaults: ...' to stdout before the start command's JSON hook
payload, corrupting the machine-readable contract every fresh install's
first hook invocation relies on. The Phase 6 per-run temp data dir made
the cold-dir case deterministic in CI, exposing it. Both informational
notices (creation, nested-schema migration) now use console.warn —
stderr — matching the function's existing failure-path idiom; two
regression tests pin stdout silence on both paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(restart): address PR #2894 review — dedupe script resolver, skip futile port wait

Both inline copies of the marketplace-first script-candidate list in
worker-service.ts (restart fallback + successor handoff injection) now
call the exported resolveWorkerScriptPath() ?? __filename, so the
candidate list lives in one place. verifyRestartedWorker's failure
result gains lastPollSawHealth; when the self-replacement handoff
verification timed out while a live (but unverifiable) worker was still
serving on the port, the CLI fallback now skips its port-free wait —
the port cannot free while that worker lives, so the wait only burned
its full platform-scaled budget before the same final verification ran
anyway.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): stale Claude CLI can no longer silently kill every observation

findClaudeExecutable validated candidates with --version only, so an
abandoned old install shadowing a current CLI in PATH (e.g. npm-global
2.0.42 next to the auto-updating native installer) passed validation,
then died at every Observer spawn: the SDK passes --permission-mode
dontAsk (hardened-options) which old CLIs reject with exit 1. Result:
healthy worker, zero observations, no visible error (#2782 family;
previously #1857/#2049/#1866/#2142 in the same class).

- Probe every candidate (which -a + known install paths) with
  `--permission-mode dontAsk --version`: one spawn proves both flag
  compatibility and version, no API call (~150ms)
- Prefer the newest capable version; PATH order only breaks ties
- Explicit CLAUDE_CODE_PATH still wins but fails loud with version and
  remedy when too old, instead of dying silently at spawn
- All-too-old throws an error naming each candidate, its version, and
  how to fix; resolution success logs at INFO with the chosen version
- Cache successful resolution 15 min (resolver runs per SDK query);
  never cache failure so a CLI update is picked up without restart
- SDK child keeps a 2KB stderr tail and includes it in the exit WARN,
  so "unknown option" deaths are diagnosable at default log level

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(audit): add ponytail audit master plan + SQLite removal companion

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-0): remove commit-hash fabrication verification

Removes the commit-verification system whose second-system effects (greedy hex
regex false-positives + compensating scaffolding) cost more than the edge case
it caught. Deletes src/sdk/commit-verification.ts + its 2 tests, the
verify-before-persist block + stripFabricatedHashesFromSummary in
ResponseProcessor, and the now-dead fabricated_count/fabrication_* telemetry
in buffer.ts/scrub.ts/npx-cli telemetry.ts.

Consequence: PostHog Fabrication Rate tile goes dark (no other producer).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-1): remove duplicate SQLite stack, test the shipping path

Two parallel SQLite systems lived in src/services/sqlite/. The worker only ever
used SessionStore (which runs its own inline migrations); the second stack —
ClaudeMemDatabase/MigrationRunner + a free-function CRUD API — was reachable only
from tests. This removes the dead duplicate and rewrites its tests against the
real SessionStore path.

Deleted: Database.ts, migrations/runner.ts, index.ts, SchemaRepair.ts, all 6 CRUD
barrels, transactions.ts, and the sessions/summaries/prompts/timeline/import
submodules (18 files). Trimmed the 5 survivor leaf functions to their single live
export. Removed dead SessionStore methods getSessionSummaryById (broken) and
storeObservationsAndMarkComplete (0 callers).

Replaced 9 old test files (coupled to the dead stack) with 7 focused
session-store-*.test.ts suites + a rewritten cleanup-v12_4_3 test, all exercising
the worker's real path.

Net ~-3,262 src lines. tsc clean; full suite 2253 pass / 0 fail; dead-reference
sweep zero. No production behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-6): drop dead storage methods, dedupe EnvManager, stdlib swaps

Dead-code deletes (all zero-caller verified): sqlite ProjectsRepository.upsert +
getByRootPath, ServerSessionsRepository.getByMemorySessionId,
MemoryItemsRepository.getByLegacyObservationId, pg AgentEventsRepository.createMany
+ TeamsRepository.getMember; shared getWorkerSocketPath, createBackupFilename, and
the deprecated ENV_FILE_PATH const.

Refactors (behavior-preserving): EnvManager's three open-coded 5-key credential
blocks collapse to one CREDENTIAL_KEYS loop (whitelist semantics kept, no
Object.assign); parseEnvFile line-parser → util.parseEnv; fetchWithTimeout's manual
setTimeout race → AbortSignal.timeout (caller timeout-message contract preserved).

Net ~-106 lines. tsc clean; full suite 2253 pass / 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-4): sweep dead/redundant code from src/server

Deletes (zero-caller verified): jobs/outbox.ts (canonical path is
IngestEventsService/EndSessionService) + its test; the src/server/mcp/ surface
(live MCP server is src/servers/mcp-server.ts) + its test; the always-inert
ServerBetaProviderRegistry/EventBroadcaster boundaries (no Active variant exists)
+ their /v1/info payload keys; the unused isServerClassified guard; dead option
sessionDebounceWindowMs (route + IngestEventsService).

Dedupe: hoist the 5 byte-identical auth helpers shared by auth.ts/postgres-auth.ts
into request-auth-helpers.ts; collapse resolveSummaryQueue/resolveEventQueue into
one resolveQueue(lane).

Kept (plan premise didn't verify): the ServerV1Routes runtime? option — a live
smoke test sets and asserts it. Disabled* boundary subclasses kept (real Active
counterparts chosen at runtime).

Net ~-570 source lines. tsc clean; full suite 2242 pass / 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-7): dead-code sweep + dedup across services + npx-cli

Deletes (zero-caller verified, tests removed in lockstep): ProcessManager
getChildProcesses + parseElapsedTime (ps-reaper leftovers), ModeManager
getObservationConcepts/validateType/getTypeLabel, AgentFormatter
renderAgentFileHeader/renderAgentColumnKey/renderAgentContextIndex (return-[] inert,
HeaderRenderer arms collapsed to Human-only), asyncHandler export,
PriorMessages.userMessage (always ''), and 3 dead barrels (services/context,
services/server, services/server/Middleware).

Dedup: shared toError() helper (src/utils/to-error.ts) across 8 in-scope sites;
asMs hoisted to telemetry/common; spawnPlugin() folds 4 spawnHidden blocks;
countObserverSessionRows extracts the CleanupV12_4_3 count trio; IS_WINDOWS imported
from paths.ts (was redeclared 3x); detectOsVersion inlined to os.release().

Kept (plan premise didn't verify): bun-resolver's spawnSync (npx-cli runs under
Node — Bun.which would ReferenceError); context-generator.ts (a build entrypoint +
test mock seam) — barrel still removed; Middleware.ts consumer Server.ts repointed
to the real middleware.

Net ~-279 lines. tsc clean; full suite 2232 pass / 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-5): delete orphans + dead UI/util code across cli/supervisor/ui

Orphan deletes (zero-caller verified): src/bin/ (import-xml-observations,
cleanup-duplicates — not wired to any entrypoint), src/adapters/ (abandoned parallel
impl; real adapters are src/cli/adapters/), core/schemas/context-pack.ts +
core/schemas/index.ts barrel. Companion tests removed in lockstep.

Dead functions: 7 cursor-utils helpers (kept the 6 live registry/context writers),
logger correlationId/sessionId/timing.

YAGNI trims: gemini-cli metadata block (never read), useStats collapsed to
fire-and-forget (+ dead Stats/WorkerStats/DatabaseStats types), useTheme
resolvedTheme, useContextPreview.refresh from the public result; env-sanitizer
ENV_PROXY_VARS folded into ENV_EXACT_MATCHES.

Dedup: useGitHubStars + formatNumber inlined into their single consumer
GitHubStarsButton (kept lowercase-k display — matches GitHub, Intl would uppercase);
waitForExit extracted in process-registry and shared with shutdown's reapSession
(placed low to avoid a circular import).

Net ~-1085 lines (incl. test deletions). tsc + viewer typecheck clean;
full suite 2181 pass / 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-2a): integrations/sync dead exports + scoped Chroma cleanup

Dead-export deletes (zero-caller verified): syncContextToAgentsMd,
updateWindsurfContextForProject, handleWindsurfCommand, detectClaudeCode (+ orphaned
exec/promisify imports), getScriptExtension/detectPlatform, getVersionCheckAbsolutePath
(+ companion plugin-distribution test edit). ChromaSyncState flush/resetCache/dirty
(bump/replace persist eagerly, so dirty was always false).

Scoped to the Chroma write path only (SQLite/search/telemetry copies stay live):
dropped the created_at ISO field (kept created_at_epoch, the field actually read back)
and the discovery_tokens param/field from ChromaSync's local Stored interfaces + the
3 live call sites.

Stdlib/shrink: parseSemver/compareSemver → localeCompare numeric (ordering verified);
OpenClaw roots hoisted to one const; bootstrapWatermarks max-loop → Math.max;
Goose YAML builders collapsed to one withHeader flag.

Defers the 5-installer consolidation to a separate pass. Net ~-187 lines.
tsc clean; full suite 2181 pass / 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-2b): delete dead fetchAndInjectOpenCodeContext

Orphaned when slice-2a removed syncContextToAgentsMd (its only caller chain).
Confirmed zero callers. Kept fetchRealContextFromWorker (live at :323).

Per feasibility analysis, the plan's 5-installer table-driven consolidation
(headline -2400) was descoped: the 5 installers write 5 different config schemas
and 4 different context formats with thin test coverage, so a 5->1 collapse would
relocate divergence into callbacks and risk untested IDE-config write paths for a
line-count-only win. Plan gate ('only if behavior genuinely identical') not met.

tsc clean; full suite 2181 pass / 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(slice-3): worker dead-code sweep + provider/search consolidation

Search pipeline: delete the unreachable half built speculatively (TimelineBuilder,
ResultFormatter instance methods [kept the one live static], search/filters/,
SearchStrategy.canHandle/name + BaseSearchStrategy, HybridSearchStrategy no-op
search(), dead SearchOrchestrator wrappers, ChromaQueryResult) + companion tests.
Dedup the 5 genuinely-identical chroma->recency->hydrate->FTS paths into one
hybridSemanticHydrate helper and the 3 timeline renderers into renderTimeline()
(search()/decisions()/changes()/howItWorks() left as-is — different shape).

Providers: extract OpenAICompatibleProvider base from the Gemini/OpenRouter twins,
preserving every per-provider divergence via flags/abstracts (truncation guard,
token estimation, empty-response handling, endpointClass, the Gemini-only RPM
throttle). Dedup parseRetryAfterMs into retry.ts.

http/agents/session: delete the IngestEventBus machinery + ingestPrompt, the
read-only POST /api/processing dup, FallbackErrorHandler.shouldFallbackToClaude,
RateLimitStore.getAll/clear, SessionCompletionHandler.completeByDbId,
SessionCleanupHelper, 3 dead agent types, and no-op empty-passthrough validateBody;
collapse the 6 CorpusRoutes 404 blocks into one helper.

Kept (false-dead, caller verified): SSEEventPayload (live via WorkerRef), RateLimitStore.get
(live via shouldAbortForQuota). Both adversarial reviews confirmed behavior preserved.

Net ~-2,640 lines. tsc clean; full suite 2080 pass / 0 fail; dead-ref sweep zero.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* build(plugin): regenerate distribution artifacts after ponytail audit

Rebuilds the worker/mcp/server-beta/transcript-watcher/context-generator bundles
and the viewer UI from the post-audit source (Slices 0-7).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(env): require Node >=20.12.0 to match util.parseEnv usage

EnvManager adopted stdlib util.parseEnv (added in Node 20.12.0) during
the ponytail audit, but engines.node still advertised >=20.0.0. On Node
20.0-20.11 parseEnv is undefined: loadClaudeMemEnv() silently returns {}
(credentials never load) and saveClaudeMemEnv() throws. Declare the real
floor in both the npm package and the generated plugin manifest (via its
generator in build-hooks.js so rebuilds stay consistent).

Greptile PR #3021 P1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-21 13:12:41 -07:00
Alex Newman 580621c936 PostHog telemetry overhaul: per-session rollups, unified instrumentation, redacted error tracking (#3010) 
* feat(telemetry): add unified instrument() layer fanning out to logger + telemetry

Introduces src/services/telemetry/instrument.ts as the single
instrumentation entry point: always writes the local log line, then
(consent-gated, swallow-all) routes scrubbed props to the telemetry
sink. Logger stays telemetry-free (no cycle). Migrates 3 exemplar
call sites (SessionRoutes, BaseRouteHandler, ResponseProcessor) from
duplicated log+capture pairs to single instrument() calls.

Phase 1 of the PostHog telemetry overhaul (plans/2026-06-19-posthog-telemetry-overhaul.md).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(telemetry): emit per-session session_compressed rollups at session end

Replaces the 5-minute time-window session rollup with a per-session
accumulator keyed by sessionDbId. Flushes one observer_turn_rollup per
session on session_end (deleteSession/removeSessionImmediate),
worker_shutdown (drained in shutdownTelemetry before the PostHog client
closes), and a periodic safety_flush for over-cap sessions (window_seq
bump + re-arm). Adds rollup_reason/window_seq to the scrub whitelist.
context_injected stays a time-window rollup (hook-level, no sessionDbId).

Fixes a shutdown-drain ordering bug where worker_shutdown rollups were
dropped because isShutdown latched before the drain; adds a
shutdownTelemetry() integration test covering it.

Phase 2 of the PostHog telemetry overhaul.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(telemetry): capture redacted error data via PostHog Error Tracking

Adds $exception capture with an allow-then-redact error scrubber
(error-scrub.ts): home dir, absolute paths, DB connection-string creds,
URL userinfo, emails, API tokens (sk-/phc-/ghp-/AWS AKIA/JWT), hex and
IPv4 are stripped; message capped 500 / stack ~2KB. captureException is
consent-gated, profile-less ($process_person_profile:false), and
fingerprint rate-limited (1/min/fingerprint, bounded LRU map). Manual
errors route via a no-cycle logger error sink (logger.setErrorSink).

Worker enables enableExceptionAutocapture, with errorBeforeSend fully
redacting autocaptured $exception_list payloads (deletes source
context_line/pre_context/post_context, redacts value + filename) so the
SDK's on-disk source-context capture can never exfiltrate code. A
sentinel marks manual captures so the limiter runs exactly once and
occurrence_count ships intact. Redaction inputs are hard-capped at 8KB
with bounded regexes to prevent ReDoS on the worker.

New kill-switch CLAUDE_MEM_TELEMETRY_ERRORS=0 disables error capture
independently of analytics.

Phase 3 of the PostHog telemetry overhaul.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(telemetry): canonicalize rollup naming and document the new model

Fixes a stale session_compressed_rollup comment to the canonical live
event name (observer_turn_rollup). Rewrites docs/public/telemetry.mdx to
reflect the overhaul: per-session observer_turn_rollup with
rollup_reason/window_seq, the new redacted $exception error tracking
(what is kept vs redacted, rate-limiting, consent gate, one-way-door
note), the CLAUDE_MEM_TELEMETRY_ERRORS opt-out, the unified instrument()
logging model, and a note that PostHog session replay is N/A for a
headless Node worker. Reconciles "What is NEVER collected" honestly now
that redacted error messages are collected. No raw session_compressed or
context_injected events remain; only rollups are emitted.

Phase 5 of the PostHog telemetry overhaul.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* build(telemetry): regenerate plugin artifacts with telemetry overhaul

Rebuilds the tracked plugin/ bundle so the published plugin emits the
new per-session observer_turn_rollup (rollup_reason/window_seq), the
redacted $exception error tracking, and the unified instrument() layer.
The committed artifacts were stale (last built before Phases 1-5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-20 00:02:36 -07:00
Alex Newman c0b96288a7 fix(restart): worker-restart single source of truth — self-replacing worker, spawn gate, verified restarts (#2894) 
* plan-10 Phase 1: ship deterministic plugin runtime dependency closure

Approach A — commit & ship plugin/bun.lock so the plugin's runtime
node_modules install is deterministic, fixing the recurring
`Cannot find module 'zod/v3'` (#2730).

- align generated plugin zod range to root (^4.4.3) in build-hooks.js
- new scripts/gen-plugin-lockfile.cjs generates plugin/bun.lock as a
  build artifact after build-hooks.js writes plugin/package.json
- track & ship plugin/bun.lock (.gitignore negation, .npmignore, files allowlist)
- install with `bun install --frozen-lockfile --ignore-scripts` at runtime

Refs #2783, #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10 Phase 2: fail loud at install time on a broken dependency closure

Strengthen verifyCriticalModules to assert each dependency is actually
importable via require.resolve (not merely a directory), and assert the
worker-required zod subpaths resolve: zod/v3, zod/v4, zod/v4-mini.
A partial/stale install now fails `npx claude-mem install` immediately
instead of surfacing later as a Stop-hook `Cannot find module 'zod/v3'`.

Bin-only packages (e.g. tree-sitter-cli, which has no bare-name entry
point) fall back to resolving <dep>/package.json so a healthy install
isn't falsely rejected.

Adds tests/cli/verify-critical-modules.test.ts covering a missing zod/v3
subpath (throws), a complete zod (passes), and a bin-only dep (passes).

Refs #2783, #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10 Phase 3: clean-room install + import smoke test (#2730 backstop)

Add scripts/smoke-clean-room.cjs and a `smoke:clean-room` npm script.
Against fresh temp dirs (never the repo's node_modules) it:
- copies plugin/, runs `bun install --frozen-lockfile --ignore-scripts`,
  asserts zod, zod/v3, zod/v4, zod/v4-mini resolve, and boots the bundled
  worker asserting no `Cannot find module` — the direct #2730 regression guard;
- `npm pack`s, installs the tarball into a second temp dir, and load-tests
  the published bin entrypoint, warning loudly on any declared main/exports
  target missing from the tarball (latent #2537 gap).

Exits non-zero naming the missing module on any failure; cleans up all
temp dirs and the tarball in a finally.

Refs #2783, #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10 Phase 4: gate CI and publish on the clean-room dependency closure

- ci.yml: new `clean-room-deps` job (between build and the docker e2e job)
  runs a frozen-lockfile drift check on the committed plugin lockfile, then
  `npm run build` + `npm run smoke:clean-room`. The drift step catches a
  contributor who changed plugin deps without regenerating plugin/bun.lock.
- npm-publish.yml: add setup-bun and run `npm run smoke:clean-room` between
  build and `npm publish`, so a broken runtime closure cannot be published
  on a tag push (ci.yml does not run on tags). Secrets block untouched.

Refs #2783, #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10: doc recluster note + Phase 0 execution slice for #2730

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plans: backlog recluster (2026-06-04) — cross-cluster execution order + plan-13 doc

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* plan-10: gen-plugin-lockfile degrades gracefully when bun is absent

The Windows build CI job has no bun on PATH; regenerating the lockfile there
threw and failed the build. The committed plugin/bun.lock is already the
deterministic closure, so skip regeneration (non-fatal) when bun is missing
and a lockfile exists; fail loud only when neither is available.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: rebuild plugin artifacts after merging main (v13.5.1) + plan-10 work

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: rebuild plugin artifacts after merging main v13.5.5

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(deps): daily upgrade pass — agent SDK 0.3.172, better-auth 1.6.16, posthog-node 5.36.15, dompurify 3.4.9

- Bump @anthropic-ai/claude-agent-sdk 0.2.141 -> 0.3.172 (tsc + full test suite green)
- Remove deprecated @types/dompurify stub (dompurify ships its own types)
- Add overrides.tmp ^0.2.7 to clear GHSA-52f5-9888-hmc6 / GHSA-ph9p-34f9-6g65
  via np -> listr-input -> inquirer -> external-editor -> tmp chain
- npm audit: 0 vulnerabilities; npm outdated: clean
- package-lock.json is gitignored in this repo, so not committed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* plan: worker-restart single-source-of-truth — 7-phase fix for restart races

Phased plan from the adversarially-verified diagnosis (wf_f07f3541-b05):
kill the cache mirror, single verified restart initiator, self-replacing
restart endpoint, unified spawn gate with lockfile, PID-file demotion,
test data-dir isolation, soak verification.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(restart): delete sync-script cache-mirror and HTTP restart trigger

Phase 1 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
The installed-version cache mirror wrote version-N code into the
version-(N-1) cache dir, manufacturing permanent version disagreement;
the HTTP POST to /api/admin/restart raced the CLI restart that follows
it in build-and-sync. Both are deleted; the CLI worker:restart in the
marketplace copy is now the single restart initiator, and the sleep 1
between the two mechanisms is gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(restart): restart proves itself or exits 1

Phase 2 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
worker-service restart now captures the old worker pid, waits for the
port with the same platform-scaled 15s budget as stop, spawns the
marketplace copy of worker-service.cjs when present, then polls
/api/health until the pid changes and the version matches this build's
baked __DEFAULT_PACKAGE_VERSION__ — success is printed to stdout,
deadline (platform-scaled 30s) exits 1 with the last observed health
payload and the spawned script path. The --daemon generic start-failure
path now exits 1 instead of masquerading as success; the three
duplicate-suppression exits remain 0.

New helper src/services/restart-verify.ts (worker-service.ts bootstraps
on import, so the helper lives in an import-safe module) with 8 tests
covering pid-flip success, stale pid, wrong version, unreachable
timeout, 503-degraded acceptance, and null-oldPid version-only
verification.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(restart): self-replacing worker — old worker spawns its successor

Phase 3 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
/api/admin/restart was kill-only: hooks that POSTed it then raced the
dying worker with their own lazy-spawn (the observed recycle ping-pong).
Now the dying worker spawns its successor itself — after a re-entrancy-
guarded, deadline-bounded (platform-scaled 10s) graceful shutdown, and
only once its port is confirmed free; stop and signal shutdowns stay
kill-only. The hook recycle path waits for that successor via
/api/health polling (HOOK_READINESS_TIMEOUT_MS budget) and lazy-spawns
only as a fallback, with a warn-only version re-check so a hook never
recycles more than once per invocation.

Shutdown sequence lives in import-safe src/services/worker-shutdown.ts
(worker-service.ts bootstraps on import); registerSignalHandlers no
longer pre-sets isShuttingDown — the supervisor's shutdownInitiated
guard owns signal dedupe, and pre-setting would no-op the new entry
guard. 13 new tests cover re-entrancy, deadline expiry/rejection,
handoff ordering, kill-only reasons, successor-wait vs lazy-spawn
fallback, and pre-graceful bookkeeping failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(restart): one spawn gate; CLI restart defers to the self-replacing worker

Phase 4 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
Three uncoordinated spawn paths (hook lazy-spawn, MCP worker-spawner,
CLI) with two different bun resolvers produced 3-launcher collisions
within a single second. Now a wx-flag lockfile (<DATA_DIR>/spawn.lock,
60s mtime staleness with re-stat-before-unlink, owner-checked release)
gates every external spawn: lock losers never fail — they skip the
spawn and wait for the winner's worker. resolveBunRuntime is deleted in
favor of ProcessManager's resolveWorkerRuntimePath (adds BUN_PATH,
~/.bun/bin, brew, which fallbacks), closing the kill-then-can't-respawn
path; mcp-server prefers the marketplace worker script so stale cache
dirs stop spawning stale workers.

Integration fix surfaced by live verification: the CLI restart raced
the Phase 3 self-replacement handoff (the successor re-binds the port
in ~200ms, so waitForPortFree always timed out and restart exited 1
while the restart had actually succeeded). The CLI now verifies the
worker's self-spawned successor directly, and only spawns — gate-
wrapped, after the port frees — as the fallback when no worker was
running, the shutdown POST was rejected, or no successor appeared. The
dying worker's handoff is intentionally ungated: it spawns only after
its own port closes, and hooks wait on it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(restart): demote the PID file — health and port are the liveness oracle

Phase 5 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
The dying worker's shutdown cascade deleted the PID file
unconditionally as its final act, clobbering the successor's
freshly-written file; status then required portInUse AND pidInfo, so a
healthy worker reported as "not running". Now every PID-file deletion
is owner-guarded: the supervisor cascade deletes only its own pid
(removeOwnedPidFile), and the CLI stop/restart-fallback, the restart
handoff, and the daemon start-failure cleanup go through
removePidFileIfOwner (owner-or-dead — a live successor's file always
survives; corrupt files are left for the next boot's validator).

status sources from GET /api/health alone (pid, version, uptime,
workerPath; 503-degraded counts as running and now surfaces its queue
detail), with port-in-use-but-unreachable and not-running fallbacks —
all exit 0 as before. The --daemon duplicate gate checks the port
first (ground truth) and the PID file second (advisory, for the
freed-port-but-undeleted-file window); duplicate suppression stays
exit 0. writePidFile/touchPidFile remain — the file is diagnostics,
and the worker stays its only writer.

Also fixes combined-run test pollution: spawn-gate and worker-utils
timeout tests now eagerly import paths.js before setting a temp
CLAUDE_MEM_DATA_DIR, so the import-time DATA_DIR const can't freeze on
a deleted temp dir for suites loaded later in the same bun process.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: no test ever touches the real ~/.claude-mem again

Phase 6 of plans/2026-06-10-worker-restart-single-source-of-truth.md.
process-manager and graceful-shutdown tests wrote corrupt JSON and
sentinel PIDs (2147483647) into the real ~/.claude-mem/worker.pid and
drove the real supervisor.json cascade under a snapshot-restore that a
killed run would skip — that pollution contaminated production logs and
a prior diagnosis. Both files now set a temp CLAUDE_MEM_DATA_DIR at the
top of the file before dynamically importing the code under test (ESM
hoisting makes beforeEach too late), assert their paths landed outside
the real dir, and derive PID_FILE from the same frozen paths module the
code uses so test and code can never diverge under bun's shared module
cache. The snapshot-restore scaffolding is deleted; zero assertions
changed.

tests/preload.ts gains a tripwire: when CLAUDE_MEM_DATA_DIR is unset it
fills a per-run temp dir, so no test in any file can fall through to
the real data dir. Fallout made explicit: worker-spawn child processes
get an explicit temp dir; install-error-matrix restores rather than
deletes the env var; settings-defaults-manager pins the unset-env
default it was implicitly relying on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(settings): bootstrap notices go to stderr, never stdout

CI on PR #2894 caught the latent bug: on the first boot in a fresh data
dir, SettingsDefaultsManager printed '[SETTINGS] Created settings file
with defaults: ...' to stdout before the start command's JSON hook
payload, corrupting the machine-readable contract every fresh install's
first hook invocation relies on. The Phase 6 per-run temp data dir made
the cold-dir case deterministic in CI, exposing it. Both informational
notices (creation, nested-schema migration) now use console.warn —
stderr — matching the function's existing failure-path idiom; two
regression tests pin stdout silence on both paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(restart): address PR #2894 review — dedupe script resolver, skip futile port wait

Both inline copies of the marketplace-first script-candidate list in
worker-service.ts (restart fallback + successor handoff injection) now
call the exported resolveWorkerScriptPath() ?? __filename, so the
candidate list lives in one place. verifyRestartedWorker's failure
result gains lastPollSawHealth; when the self-replacement handoff
verification timed out while a live (but unverifiable) worker was still
serving on the port, the CLI fallback now skips its port-free wait —
the port cannot free while that worker lives, so the wait only burned
its full platform-scaled budget before the same final verification ran
anyway.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 19:59:34 -07:00
Alex Newman be43fd18b8 feat(telemetry): deep instrumentation — observation types, token economics, compression metrics, IDE on lifecycle events 
context_injected now carries the full injection story computed alongside
rendering (ContextInjectStats): observation/session counts, timeline
depth, per-type buckets, tokens_injected, tokens_saved_vs_naive,
mode/provider/search_strategy. Capture moved from middleware into the
handler so stats ride the event; empty-state injections are not counted.

session_compressed gains hook (init|ingest|summarize, stamped at prompt
dispatch), compression_ms (dispatch → response), model id, ide (session
platform source), per-type buckets + dominant observation_type, and REAL
token usage: Claude from message usage (incl. cache), Gemini from
usageMetadata, OpenRouter from usage — never the 70/30 estimates;
providers leave lastUsage null when the API gives no split. provider is
now always set (currentProvider/agentName fallback chain).

worker_started moved to post-init so it can read the install's dominant
IDE from session history — person-property ide makes IDE-level
DAU/retention breakdowns non-null without reinstalling.

signal_rate_pct from the spec was NOT added: nothing in the codebase
measures injected-token relevance, and we don't fabricate metrics.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-09 21:33:26 -07:00
Alex Newman 3759d99010 batch: land five reviewed fixes from @rodboev (#2848 #2850 #2851 #2852 #2860) (#2862) 
* fix(context): suppress native Codex transcript AGENTS writes (closes #2837)

* fix(sqlite): preserve legacy NULL observation hashes during migration 29 (closes #2812)

* Keep Codex hook output within the supported schema (closes #2844)

* Honor persisted worker request timeouts in MCP calls (closes #2822)

* Limit Codex AGENTS suppression to native transcript watches

* Preserve migrated hashes through legacy FK rebuilds

* Keep settings-file defaults from inheriting stale env state

* Keep welcome-hint settings overrides request-local

* Keep Bun test state from masking settings fallbacks

* Keep sticky test mocks from bleeding into later routes

* Keep generated worker bundle aligned after baseline carry-over

* Keep settings-file defaults from inheriting stale env state

* Keep welcome-hint settings overrides request-local

* Keep Bun test state from masking settings fallbacks

* Keep sticky test mocks from bleeding into later routes

* Keep generated hooks aligned after baseline carry-over

* Keep settings-file defaults from inheriting stale env state

* Keep welcome-hint settings overrides request-local

* Keep Bun test state from masking settings fallbacks

* Keep sticky test mocks from bleeding into later routes

* Keep generated bundles aligned after baseline carry-over

* Keep settings-file defaults from inheriting stale env state

* Keep welcome-hint settings overrides request-local

* Keep Bun test state from masking settings fallbacks

* Keep sticky test mocks from bleeding into later routes

* Keep generated scripts aligned after baseline carry-over

* Keep worker recycle tests isolated from module cache

* Preserve native Codex watch filtering while narrowing AGENTS suppression

* Keep migration backfills out of user hash namespaces

* fix(sqlite): bound stored prompt payloads to stop prompt-table bloat (closes #2793)

* fix(sqlite): normalize prompt dedupe against the stored payload

* Keep settings-file defaults from inheriting stale env state

* Keep welcome-hint settings overrides request-local

* Keep Bun test state from masking settings fallbacks

* Keep sticky test mocks from bleeding into later routes

* Keep worker recycle tests isolated from module cache

* Keep baseline fallback fixes consistent with regenerated assets

* Keep parser mode tests isolated and prompt storage observable

* Keep parser mode tests isolated and prompt storage observable

* Keep parser mode tests isolated and prompt storage observable

* build: regenerate plugin bundles for combined batch

* test: restore module mocks after codex-context processor tests

The module-level mock of utils/project-name leaked into
tests/utils/project-name.test.ts when the full suite ran in one
process.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-09 14:59:04 -07:00
Rod Boev 35fbf917e0 fix(settings): keep file defaults from inheriting stale env overrides (#2853) 
* Keep settings-file defaults from inheriting stale env state

* Keep welcome-hint settings overrides request-local

* Keep Bun test state from masking settings fallbacks

* Keep sticky test mocks from bleeding into later routes

* Keep worker recycle tests isolated from module cache
 2026-06-09 14:52:01 -07:00
Alex Newman 6259b20c90 chore: bump version to 13.4.1 2026-06-07 18:45:06 -07:00
Alex Newman 166b956e88 Merge remote-tracking branch 'origin/main' into mutual-aardvark 
# Conflicts:
#	package.json
#	plugin/scripts/context-generator.cjs
#	plugin/scripts/mcp-server.cjs
#	plugin/scripts/server-service.cjs
#	plugin/scripts/worker-service.cjs
#	plugin/ui/viewer-bundle.js
#	src/npx-cli/commands/install.ts
#	src/server/runtime/ServerService.ts
#	src/server/runtime/create-server-service.ts
#	src/services/worker-service.ts
#	tests/hooks/server-client.test.ts
 2026-05-29 14:22:31 -07:00
Alex Newman 34c734a1a4 refactor(server): rename server-beta → server runtime (plan phase 1) 
Foundational rename: the server runtime is now `server`, not `server-beta`.
Removes the literal-string regression in `runtime-selector.ts` where only
`CLAUDE_MEM_RUNTIME='server-beta'` was accepted (anything else silently
fell back to the worker runtime).

What changed (1a–1d per plans/2026-05-25-cmem-sdk-and-server-rename.md):

- 1a Regression fix: `selectRuntime()` accepts both `'server'` and
  `'server-beta'` (canonicalizing to `'server'`); new settings keys
  `CLAUDE_MEM_SERVER_{URL,API_KEY,PROJECT_ID}` read first, legacy
  `*_BETA_*` keys fall back via a `pickFirstNonEmpty` helper.
- 1b Code identifiers: ~80 `ServerBeta*`/`serverBeta*`/`SERVER_BETA_*`
  symbols (classes, types, functions, vars, non-persisted constants) →
  `Server*`/`server*`/`SERVER_*`.
- 1c File renames: 14 files moved via `git mv` (tracked as renames);
  build target `server-beta-service` → `server-service` (emits
  `plugin/scripts/server-service.cjs`); dispatch sites in
  `worker-service.ts`, `runtime.ts`, `ServerService.ts` keep a documented
  `existsSync` fallback to the legacy `.cjs` for installs running from a
  pre-rename plugin cache (plan §1c line 149).
- 1d Persisted-value back-compat (zero-risk path):
  - DB table name `server_beta_schema_migrations` preserved
  - Job/source enum strings (`server_beta_generate_event`, …) preserved
  - `lockedBy: 'server-beta-worker'` literal preserved
  - Installer writes new canonical settings keys + `'server'` runtime
    value going forward; reads dual-accept old + new.

Verification:
- Typecheck: 24/24 baseline errors unchanged (no new errors introduced).
- Tests: 1810 pass / 54 fail / 19 skip — failure count matches the
  pre-Phase-1 baseline; no new failures introduced.
- Build: `npm run build` succeeds; `plugin/scripts/server-service.cjs`
  emitted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-29 14:01:34 -07:00
Alex Newman d13fc437fa feat(provider): configurable OpenAI-compatible base URL (DeepSeek/LM Studio/custom) 
Make the OpenRouter provider's base URL configurable via CLAUDE_MEM_OPENROUTER_BASE_URL
(+ OPENROUTER_BASE_URL env), turning it into a generic OpenAI-compatible client. Shared
resolver appends /chat/completions to a base or uses a full URL verbatim; default
behavior unchanged when unset. Model id passes through verbatim. Applied to both worker
and server providers.

Closes #2382 #2590 #2622 #2393.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-05-28 18:48:39 -07:00
Alex Newman daf6d9dc0f fix: concrete standalone bugs — project name, dot-path, path-match, CLAUDE.md denylist 
- #2663: derive project name from git repo root (rev-parse --show-toplevel), stable
  across subdirs/worktrees; fall back to basename(cwd) outside a repo.
- #2401: 'include last message' encodes cwd with both '/' and '.' -> '-' to match Claude
  Code's transcript dir naming, so a '.' in a path component no longer no-ops.
- #2691: PreToolUse:Read queries observations by BOTH absolute + cwd-relative path
  (value IN candidates) so context injection matches PostToolUse storage.
- #2400: CLAUDE_MEM_FOLDER_MD_SKELETON_DENYLIST suppresses empty/skeleton CLAUDE.md
  injection in deny-listed folders.
- #2473: confirmed host-side (resolved upstream); added invariant test that our MCP
  server/tool names stay colon/dot-free.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-05-28 18:41:06 -07:00
Alex Newman 8151cd5a8f fix(spawn): plan-02 canonical ${CLAUDE_PLUGIN_ROOT} resolution + Windows spawn fixes 
- Centralize Rule B absolute-path bake (src/services/integrations/install-paths.ts) used
  by Cursor/Gemini/Windsurf/Mcp installers; centralize Rule A shell template
  (src/build/hook-shell-template.ts).
- Build-time guardrail asserts hooks.json/codex-hooks.json/.mcp.json match the canonical
  generator (prevents drift); validation matrix test executes the resolution pipeline.
- #2696: quote cmd.exe uvx args so 'protobuf<7'/'onnxruntime>=1.20' aren't parsed as
  redirection on Windows (ChromaMcpManager).
- #2695: Codex CLI spawn uses shell:true + quoting on Windows so PATHEXT resolves
  codex.cmd (no more spawnSync ENOENT).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-05-28 18:22:56 -07:00
Alex Newman 7a33685c46 fix(install): plan-04 installer failure transparency + diagnostic CLI 
- Explicit failure taxonomy + single installerError() decision point replacing ad-hoc
  swallow sites; 'Installation Complete' only when all ABORT-level deps satisfied,
  else 'Installation Partial' with remediation.
- npm install strict-first; --legacy-peer-deps only on confirmed ERESOLVE, announced.
- Missing uv/bun = ABORT with platform-specific remediation as primary message.
- Postinstall allowlist guard (scripts/check-postinstall-allowlist.js) prevents a
  re-run of the tree-sitter-swift install hang.
- 48-cell IDE x failure-mode test matrix.
- #2548: 'npx claude-mem doctor' read-only diagnostic probe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-05-28 18:05:41 -07:00
Alex Newman 0b4611e890 feat(observer): plan-11 output-fidelity contract — classify, recover, verify 
- Classify observer output (#2485): src/sdk/output-classifier.ts distinguishes
  xml/idle/prose/poisoned; ResponseProcessor logs a preview on invalid parse so
  dropped batches are visible, not silent.
- Recover from poison (#2485): SessionManager.respawnPoisonedSession kills+respawns
  the SDK session after N consecutive invalid outputs while preserving the
  SessionMessageBuffer's pending messages.
- Verify before persist (#2574): src/sdk/commit-verification.ts validates emitted
  commit hashes via git cat-file -e; fabricated hashes are flagged ([unverified
  commit]) not persisted, with provenance logging.
- Test isolation: new worker tests snapshot+restore mock.module'd modules in afterAll
  so the full suite stays order-independent (1847 pass, 0 fail/error).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-05-28 17:16:08 -07:00
Alex Newman a8cf671648 fix(tests+pipeline): repair test suite (46->7 fails) + plan-09 schema self-healing 
Test-suite repair:
- Fix cross-suite mock.module contamination: polluting files (worker-api-endpoints,
  hook-execution-e2e, file-context, chroma-mcp-manager-*, summarize-*) now snapshot
  and restore real modules in afterAll, making the suite order-independent (17 fails).
- Align stale tests to current contracts (parser {valid,observations[],summary} API,
  worker-spawner WorkerStartResult, welcome-card v3 key, haiku default model, .mcp.json
  removal, claude-md mock).

Product fixes:
- plan-09: SchemaRepair.ts + Database.openWithSchemaRepair() — self-heal malformed
  on-disk schema (orphaned index / dropped column) via sqlite3 .recover before PRAGMA,
  preserving data. Migrations re-establish canonical schema. (#2433 groundwork)
- Welcome-hint flag now reads env before cached settings (env always wins).
- Replace console.* with logger.* in background services (plugin-state, project-filter,
  worktree, smart-file-read/parser, CorpusRoutes).

Remaining 7 failures are openclaw Observation I/O tests, addressed in plan-08.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-05-28 16:44:37 -07:00
Alex Newman 7a6eaca565 fix(build): plan-10 artifact hygiene — green typecheck, drop better-auth from worker, CI gate 
- Fix 24 TS errors (logger Component union, Express 5 query params, clack
  spinner.stop arity, React 19 ref typing, dead SdkSessionRecord import). #2538
- Externalize better-auth from worker bundle (3.0MB -> 2.29MB). #2584
- Add worker-bundle size guard in build-hooks.js + .github/workflows/ci.yml
  running typecheck/build/test as required checks. #2570 #2538
- Tighten npm files allowlist / .npmignore so maintainer CLAUDE.md never ships. #2537

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-05-28 16:15:20 -07:00
Alex Newman e4ec654355 chore: bump version to 13.3.0 2026-05-21 03:25:35 -07:00
Alex Newman e7bbb2a9aa server-beta: Phases 4–13 — event pipeline, generation, MCP, compat, Docker, team audit, observability (#2383) 
* feat(server-beta): Phase 4 — Postgres event-to-generation-job pipeline

Adds POST /v1/events, /v1/events/batch, GET /v1/jobs/:id, GET /v1/events/:id,
and POST /v1/memories on the server-beta runtime, backed by Postgres.

- Event row + outbox generation-job row insert in one withPostgresTransaction.
- BullMQ enqueue happens after commit; enqueue failure leaves the row queued
  for Phase 3 startup reconciliation.
- ?generate=false skips the outbox; ?wait=true returns queue status only,
  never observation IDs (provider generation is Phase 5).
- Batch pre-validates all event projectIds against api-key scope before any
  write; mixed-project batches reject 403 with zero side effects.
- /v1/memories is a direct insert alias — no generator, no outbox.
- Cross-tenant /v1/jobs/:id returns 404 to avoid leaking row existence.
- New PostgresAuthMiddleware reads api_keys by SHA-256 hash; populates
  req.authContext.teamId/projectId; legacy ServerV1Routes (SQLite, used by
  worker runtime) is left untouched.
- Tests: unit suite hardened with stubbed pool.query so route registration
  is safe; integration tests skip cleanly without CLAUDE_MEM_TEST_POSTGRES_URL.

Verification: 87 pass / 1 skip / 0 fail. No new typecheck errors. Required
greps for WorkerService and MemoryItemsRepository in src/server/routes/v1
and src/server/runtime return no hits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): Phase 5 — provider observation generator

Adds independent provider generation under src/server/generation/ with no
worker coupling. Server beta can now generate observations end-to-end:
event -> outbox -> BullMQ -> provider -> parser -> persisted observation.

- ProviderObservationGenerator orchestrates: lock outbox (queued -> processing),
  reload agent_event from Postgres (BullMQ payload is advisory only), call
  provider, hand raw text to processGeneratedResponse, route errors via
  markGenerationFailed with retryable flag from ServerClassifiedProviderError.
- processGeneratedResponse parses with parseAgentXml, persists via
  PostgresObservationRepository with deterministic
  generation_key = generation:v1:{job_id}:{index}:{fingerprint},
  links via PostgresObservationSourcesRepository, advances outbox status,
  appends observation_generation_job_events, audits — all in one
  withPostgresTransaction. Idempotent on retry via UNIQUE constraints.
- Three provider adapters under src/server/generation/providers/:
  Claude, Gemini, OpenRouter. Self-contained — no imports from
  src/services/worker/*. Worker providers unchanged.
- Shared error classification + prompt builder under providers/shared/.
  Prompt builder strips <private> at the edge; fully-private batches
  emit <skip_summary /> without billing the provider.
- ActiveServerBetaGenerationWorkerManager wires BullMQ Worker via
  ServerJobQueue.start(...) with concurrency 1 + autorun:false +
  worker.on('error') per BullMQ docs.
- New GET /v1/events/:id/observations on ServerV1PostgresRoutes returns
  observations linked via observation_sources, team/project scoped.

Verification: 104 pass / 4 skip / 0 fail. No typecheck regressions.
Anti-pattern greps clean for services/worker imports under src/server,
WorkerRef/ActiveSession/SessionStore in src/server/generation.

Deferred: ModeManager loading uses a stable fallback observation type
list; summary and reindex queue lanes are not yet wired.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): Phase 6 — independent server session semantics

server_sessions is now the canonical Server beta session model. Sessions
are independent of legacy worker ActiveSession state.

- PostgresServerSessionRepository extended: findByExternalIdForScope,
  endSession (idempotent via COALESCE(ended_at, now())),
  markGenerationStarted/Completed/Failed, listUnprocessedEvents (filters
  agent_events with completed agent_event jobs).
- ServerSessionRuntimeRepository wraps the repo; every method requires
  explicit team_id + project_id and validates scope via assertProjectOwnership.
- SessionGenerationPolicy supports per-event (default), debounce
  (BullMQ delayed-job replace via getJob+remove+add), and end-of-session.
  Configured via CLAUDE_MEM_SERVER_SESSION_POLICY and
  CLAUDE_MEM_SERVER_SESSION_DEBOUNCE_MS env vars; per-team override hooks
  are exposed on ServerV1PostgresRoutesOptions for future settings layer.
- POST /v1/sessions/start (find-or-create on (project_id, external_session_id),
  GET /v1/sessions/:id (scoped 404), POST /v1/sessions/:id/end
  (transactional: end + create summary outbox via UNIQUE collapse +
  enqueue post-commit). Re-ending is fully idempotent.
- processSessionSummaryResponse persists summary as kind='summary'
  observation with the same idempotency model
  (generation_key + observation_sources UNIQUE).
- ProviderObservationGenerator dispatches on source_type:
  agent_event -> processGeneratedResponse, session_summary ->
  processSessionSummaryResponse; loadEvents handles session-summary
  by loading unprocessed events.
- ActiveServerBetaGenerationWorkerManager wires summary BullMQ lane
  alongside event lane (concurrency=1, autorun=false, error listener
  attached per BullMQ docs).

Verification: 110 pass / 6 skip / 0 fail. Net typecheck error count
unchanged at 24 (pre-existing, none in Phase 6 files). Anti-pattern
greps clean for ActiveSession/SessionStore in src/server/runtime,
no worker imports anywhere in src/server.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): Phase 7 — hook routing without worker dependency

Hooks can now talk directly to server-beta when CLAUDE_MEM_RUNTIME=server-beta
is selected, with a clean worker fallback when server-beta is unhealthy.

- src/services/hooks/server-beta-client.ts — typed HTTP client for
  /v1/sessions/start, /v1/events, /v1/sessions/:id/end. Throws
  ServerBetaClientError with kind classification (missing_api_key,
  transport, timeout, http_error, invalid_response) and isFallbackEligible
  helper. Zero imports from services/worker/.
- src/services/hooks/runtime-selector.ts — reads CLAUDE_MEM_RUNTIME from
  settings, returns worker or server-beta context, logs
  [server-beta-fallback] reason=<code> on every config-time fallback.
- src/services/hooks/server-beta-bootstrap.ts — Postgres-backed API key
  bootstrap. Find-or-creates local-hook-team + local-hook-project,
  generates cmem_<random> key (SHA-256 hashed), inserts into api_keys
  with scopes events:write/sessions:write/observations:read/jobs:read.
  Settings file written with chmod 0600. rotateServerBetaApiKey() wired
  to a new `claude-mem server keys rotate` command.
- src/cli/handlers/{observation,session-init,summarize}.ts — every hook
  handler tries server-beta first when configured, falls through to the
  existing worker path on transport/5xx/429/missing-key. One WARN line
  per fallback. Hook JSON output shape unchanged.
- src/shared/SettingsDefaultsManager.ts — three new keys with defaults:
  CLAUDE_MEM_SERVER_BETA_URL, CLAUDE_MEM_SERVER_BETA_API_KEY,
  CLAUDE_MEM_SERVER_BETA_PROJECT_ID.
- src/npx-cli/commands/install.ts — when installer selects server-beta
  runtime and CLAUDE_MEM_SERVER_DATABASE_URL is set, bootstraps a local
  API key automatically. Warns and continues if the DB URL is missing.

plugin/scripts/*.cjs bundles rebuilt via npm run build to pick up the
new hook handler code path. No plaintext keys in the bundle (verified).

Verification: 16 hook unit tests pass; 275 server/storage/services tests
pass with 7 pre-existing failures (verified independent of this change
via git stash --include-untracked). Build clean. No new typecheck
errors in Phase 7 files.

Anti-pattern guards verified:
- /api/sessions/observations only reached via explicit fallback path
- server-beta runtime never starts the worker process
- API keys live only in ~/.claude-mem/settings.json (chmod 0600), never
  in the bundle (grep confirmed)
- Worker fallback preserved, observable via single WARN line per call

Deferred: semantic context injection (UserPromptSubmit hook) stays
worker-only; server-beta does not yet expose /v1/context/semantic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): Phase 8 — MCP backed by server-beta core

MCP tools now route through server-beta in server-beta mode while keeping
worker-mode search/timeline/get_observations tools fully working.

- src/servers/mcp-server.ts — five new observation_* tools registered:
  observation_add, observation_record_event, observation_search,
  observation_context, observation_generation_status. Three memory_*
  compatibility aliases delegate to the canonical handlers. Worker
  auto-start is gated when selectRuntime() === 'server-beta' so MCP
  in server-beta mode never spawns the worker.
- src/services/hooks/server-beta-client.ts — addObservation,
  searchObservations, contextObservations, getJobStatus added so MCP
  shares one transport with hooks (Phase 7).
- src/server/routes/v1/ServerV1PostgresRoutes.ts — POST /v1/search and
  POST /v1/context REST cores backed by PostgresObservationRepository
  full-text search (GIN tsvector from Phase 1).
- Existing memory_search/timeline/get_observations tools call
  callWorkerAPI unchanged in worker mode; worker tests unaffected.

Verification: 39 pass / 4 skip / 0 fail on targeted suite. Pre-existing
7 baseline failures verified independent (git stash). No new typecheck
errors. WorkerService grep clean across src/servers/mcp-server.ts and
src/server/.

Anti-pattern guards verified:
- No duplicate generation logic in MCP — observation_record_event hits
  /v1/events which owns event+outbox+enqueue inside one tx
- WorkerService not imported anywhere under MCP server-beta path
- No hardcoded worker URLs — all transport via Phase 7 ServerBetaClient
- memory_* aliases retained, single handler per pair

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): Phase 9 — compatibility adapters without coupling

Legacy /api/sessions/observations and /api/sessions/summarize endpoints
keep working on server-beta runtime by translating to AgentEvent and
session-end calls — no worker code, no route duplication.

- src/server/services/IngestEventsService.ts — shared event-ingest path
  used by both /v1/events and the compat adapter. Owns transactional
  event row + outbox row + lifecycle log + post-commit BullMQ enqueue,
  honors Phase 6 SessionGenerationPolicy.
- src/server/services/EndSessionService.ts — shared session-end path
  used by both /v1/sessions/:id/end and the compat adapter. Idempotent
  ended_at + summary outbox + deterministic summary job id.
- src/server/compat/SessionsObservationsAdapter.ts — translates legacy
  POST /api/sessions/observations payload (Claude Code transcript shape)
  -> AgentEvent (source_adapter='claude-code-compat',
  event_type='tool_use') -> IngestEventsService.ingestOne. Resolves
  contentSessionId to server_sessions via find-or-create.
- src/server/compat/SessionsSummarizeAdapter.ts — translates legacy
  POST /api/sessions/summarize -> EndSessionService.end. Preserves the
  legacy agentId -> {status:'skipped', reason:'subagent_context'}
  behavior so existing clients see the same response shape.
- src/server/routes/v1/ServerV1PostgresRoutes.ts — refactored to
  delegate to the new shared services (-203 LoC net) so /v1 and
  /api compat both call the SAME canonical code path.
- src/server/runtime/ServerBetaService.ts — registers both compat
  adapters alongside ServerV1PostgresRoutes, sharing service instances.
- docs/server-beta-parity-map.md — full enumeration of legacy /api/*
  routes labeled native, adapter, or unsupported (with reasons).
  Viewer read-path adapters explicitly listed as unsupported pending
  a future viewer-rewrite phase.

Verification: 7 compat tests pass, 6 v1-routes tests still pass
(refactor preserved behavior), 4 session-routes tests pass. Pre-
existing 16 baseline failures verified independent via git stash.
Zero new typecheck errors.

Anti-pattern guards verified:
- No services/worker/http/routes or WorkerService imports under
  src/server/compat or src/server/runtime
- Compat adapters are thin translators with names ending in *Adapter
  and a top-of-file comment noting they are legacy compatibility
- /v1/* remains the canonical Server beta API; compat adapters
  call shared services rather than acting as a parallel API

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): Phase 10 — Docker stack and deployable runtime

Server beta now ships as a Docker stack with no worker process anywhere
and a separate horizontal generation worker for scaling.

- src/server/runtime/create-server-beta-service.ts — validateServerBetaEnv()
  fails fast on missing CLAUDE_MEM_SERVER_DATABASE_URL, requires
  CLAUDE_MEM_QUEUE_ENGINE=bullmq in Docker, rejects
  CLAUDE_MEM_AUTH_MODE=local-dev and CLAUDE_MEM_ALLOW_LOCAL_DEV_BYPASS
  inside containers (detected via /.dockerenv or CLAUDE_MEM_DOCKER=1).
  Adds CLAUDE_MEM_GENERATION_DISABLED so the HTTP service can run
  generator-free.
- src/server/runtime/ServerBetaService.ts — runServerBetaGenerationWorker
  for the dedicated consumer process; runServerBetaApiKeyCli is a new
  Postgres-backed `server api-key` command (the legacy worker CLI wrote
  to SQLite and was invisible to the Postgres runtime); getQueueHealth
  shim feeds /api/health a consistent ObservationQueueHealth shape.
- src/npx-cli/commands/{runtime,server}.ts — `claude-mem server worker
  start` subcommand that boots only the BullMQ consumer.
- docker/claude-mem/{Dockerfile,entrypoint.sh} — entrypoint forces
  CLAUDE_MEM_DOCKER=1 + CLAUDE_MEM_RUNTIME=server-beta and exposes
  three modes: server (HTTP only, generation disabled), worker (BullMQ
  consumer), shell. Worker bundle is no longer the default CMD.
- docker-compose.yml — full stack: postgres + valkey + claude-mem-server
  (HTTP-only) + claude-mem-worker (generation consumer). Wires
  service-to-service env vars.
- scripts/e2e-server-beta-docker.sh + docker/e2e/server-beta-e2e.mjs —
  E2E now hits /v1/sessions/start, /v1/events?wait=true, /v1/jobs/:id;
  asserts no worker-service.cjs process anywhere in the stack;
  one-shot docker compose run --rm verifies local-dev auth is
  rejected with the expected stderr; restart-and-verify confirms
  Postgres durability and BullMQ retry idempotency.
- docs/server.md — full Phase 10 doc: stack diagram, env table,
  worker mode, auth-in-Docker policy.
- docs/api.md — event generation semantics (wait=true, generationJob).

Verification: full Docker E2E PASSED on live daemon
(phase1 + phase2 + restart-and-verify + revoked-key + no-worker-
process + local-dev-rejected). Unit tests 292 pass / 9 skip / 7 fail
(7 fails pre-existing baseline). Zero new typecheck errors.

Anti-pattern guards verified:
- entrypoint never execs worker-service.cjs; E2E greps prove no
  worker process anywhere in the stack
- validateServerBetaEnv refuses local-dev auth in Docker with explicit
  remediation message; ALLOW_LOCAL_DEV_BYPASS rejected the same way
- Docker requires CLAUDE_MEM_QUEUE_ENGINE=bullmq; in-process queue
  rejected at startup
- claude-mem worker / worker-service / WorkerService greps clean
  in docker/

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): Phase 11 — team-aware generation with audit chain

Generation jobs now carry team_id/project_id/api_key_id/actor_id/
source_adapter from enqueue through execution; the outbox is reloaded
from Postgres before any side effect so BullMQ payload can never act
as auth authority.

- src/server/jobs/types.ts — ServerGenerationJobPayloadSchema (Zod
  discriminated union) requires team_id, project_id, generation_job_id,
  source_adapter, api_key_id, actor_id (nullable), source_type, source_id,
  plus event_id / server_session_id per kind. assertServerGenerationJobPayload
  is called at enqueue (outbox.ts) and again at execution boundary.
- src/server/services/{IngestEventsService,EndSessionService}.ts +
  SessionGenerationPolicy.ts — thread identity context (apiKeyId, actorId,
  sourceAdapter) into both event and summary BullMQ payloads.
- src/server/generation/ProviderObservationGenerator.ts —
  loadCanonicalOutbox loads the outbox row WITHOUT scope filter, then
  compares candidate.team_id/project_id to payload.team_id/project_id;
  mismatch -> ServerGenerationScopeViolationError (non-retryable),
  failed status, generation_job.scope_violation audit. isApiKeyRevoked
  checks api_keys (revoked_at, expires_at, row missing) before any
  provider call; revoked -> generation_job.revoked_key audit + non-
  retryable failure. generation_job.processing audit emitted on lock.
- src/server/generation/processGeneratedResponse.ts — generated
  observations carry team_id/project_id/server_session_id from the
  reloaded source row (not job payload). observation_sources.metadata
  records source_adapter, actor_id, api_key_id for traceability.
  observation.created audit per observation; generation_job.completed
  audit per terminal transition. All audit rows reference the same
  generation_job_id in details.
- src/server/routes/v1/ServerV1PostgresRoutes.ts — GET /v1/teams/:id/jobs
  and GET /v1/projects/:id/jobs with SQL-layer scoping (WHERE team_id=$1
  [AND project_id=$2] [AND status=$3]); cross-tenant returns 404 to
  avoid leaking row existence. Pagination via status/limit/offset.
  audit_log rows for event.received, event.batch_received, observation.read.
- src/server/compat/{SessionsObservationsAdapter,SessionsSummarizeAdapter}.ts —
  propagate apiKeyId and sourceAdapter='claude-code-compat'.

Verification: 162 pass / 10 skip / 0 fail. Pre-existing failures in
tests/services/queue and tests/services/worker confirmed independent
via git stash. Zero new typecheck errors in server-beta files.
Required greps:
  rg "team_id.*req\.body|project_id.*req\.body" src/server -> 0 matches
Audit chain integration test passes — generation_job.processing,
observation.created, and generation_job.completed audit rows all
share the same generation_job_id reference.

Anti-pattern guards verified:
- BullMQ payload never acts as auth authority — Postgres outbox
  reload with mismatch check happens before every side effect
- team_id / project_id never derived from request body for scope
  decisions; always req.authContext.teamId / projectId
- Application-layer team/project filtering forbidden — listJobsForScope
  pushes scope into the SQL WHERE clause
- Project-scoped key on cross-project /v1/teams/:id/jobs returns 404
- Revoked api keys cause non-retryable failure with audit before
  any provider call

Deferred: a redundant generation_job.queued audit_log row (already
covered by observation_generation_job_events lifecycle log per Phase 1
schema split). Compat adapters set actor_id=null but propagate
api_key_id which is the canonical reference downstream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): Phase 12 — observability and operations

Operators can now inspect, retry, and cancel generation jobs from the
CLI; queue lane metrics flow into /api/health and /v1/info; every
request gets a stable request_id that flows through HTTP -> audit ->
outbox -> generator -> completion log.

- src/server/middleware/request-id.ts — honors safe inbound X-Request-Id,
  mints uuid v4 otherwise. Set on req.requestId and echoed via response
  header so external traces can correlate.
- src/server/jobs/ServerJobQueue.ts — QueueEvents wired with completed,
  failed, progress, stalled, error listeners; lifecycle counters
  exposed via observe() API. Logs emitted as
  [generation] job=<id> source_type=<...> duration=<ms> attempts=<N>
  reason=<message>. Stalled and error counters survive worker restart.
- src/server/jobs/types.ts — ServerGenerationJob payload schema
  extended with optional request_id; flows through from HTTP into
  every BullMQ job.
- src/server/queue/ObservationQueueEngine.ts — health snapshot now
  carries per-lane (event, summary) counts via
  ObservationQueueHealthLaneSnapshot.
- src/server/runtime/{ActiveServerBetaQueueManager,
  ActiveServerBetaGenerationWorkerManager,ServerBetaService}.ts —
  per-lane getJobCounts feed /api/health and /v1/info; stalled events
  audit through audit_log with action generation_job.stalled.
- src/server/routes/v1/ServerV1PostgresRoutes.ts —
  GET /v1/jobs (status/source_type/since/limit/offset, scope from
  api-key, payload stripped unless ?include=payload AND admin scope),
  POST /v1/jobs/:id/retry (idempotent; queued -> no-op; audit
  generation_job.retried_by_operator), POST /v1/jobs/:id/cancel
  (terminal -> no-op; audit generation_job.cancelled_by_operator;
  generator reload-before-side-effects already prevents double work).
- src/server/services/IngestEventsService.ts +
  SessionGenerationPolicy.ts + ProviderObservationGenerator.ts —
  request_id propagated end to end. Generator extracts request_id
  from BullMQ payload and includes it in lock/processing/completion
  logs and audit details.
- src/npx-cli/commands/server-jobs.ts +
  src/npx-cli/commands/server.ts — `claude-mem server jobs
  status|failed|retry|cancel`. status compares Postgres outbox counts
  to BullMQ queue counts and surfaces divergence. failed prints
  attempts + last_error message. --team and --project filters.

Verification: 350 pass / 12 skip / 7 fail (pre-existing baseline,
verified independent via git stash). 18 new tests added (request-id
middleware, server-jobs CLI seams, jobs list/retry/cancel routes
Postgres-gated). Zero new typecheck errors.

Anti-pattern guards verified:
- agent_events.payload only emitted in /v1/jobs response inside the
  admin-gated branch (?include=payload + admin scope) — returns 403
  otherwise
- jobs retry on a queued row is a no-op (no double BullMQ enqueue,
  no double UPDATE)
- Every operator action writes to audit_log with the
  *_by_operator action and request_id correlation in details
- Stalled events audit through generation_job.stalled

Sample correlated trace (one request_id end to end):
  HTTP middleware: req.requestId = 'req-abc'
  audit event.received: details.requestId = 'req-abc'
  BullMQ payload: { request_id: 'req-abc', generation_job_id: 'gj_x' }
  generator lock log: [generation] job locked { jobId, requestId }
  audit generation_job.processing: details.requestId = 'req-abc'
  completion log: [generation] job=evt_... duration=1230ms

Deferred: live /api/health round-trip integration test (needs
Redis); stalled event live integration test (needs Redis); storing
request_id on the observations row itself (spec did not require).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(server-beta): add Phase 13 release readiness report

Captures the final verification gate: tests (1749 pass, 45 fail all
pre-existing baseline, zero regressions), required greps clean,
Docker E2E green end-to-end, all 7 exit criteria met, build clean,
typecheck unchanged from main. Documents deferred items.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* build(server-beta): rebuild server-beta-service bundle

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(server-beta): address Greptile review on PR #2383

- ProviderObservationGenerator.lockOutbox: skip duplicate worker run when
  another lock is active instead of returning the row, which previously let
  two BullMQ workers issue the (paid, rate-limited) external provider call
  before the persistence-layer terminal-status guard collapsed the duplicate.
  Reconciliation still recovers from a stale lock on startup or next retry.
- docker-compose.yml: require POSTGRES_USER/PASSWORD/DB env vars (no
  defaults). Stack refuses to start without explicit secrets. Added a header
  warning that the file must not be deployed unmodified.
- e2e-server-beta-docker.sh: export ephemeral test creds for the new
  required env vars so the Docker E2E driver still runs unattended.
- ServerBetaService api-key list: bound query with LIMIT/OFFSET (default 100,
  max 500) and add optional --team filter to prevent unintentional
  cross-tenant key metadata disclosure on shared admin hosts.
- SessionGenerationPolicy: fix dead `??` fallback for NaN parseInt result;
  use `||` so DEFAULT_DEBOUNCE_MS actually applies.
- ServerV1PostgresRoutes: `?wait=true` now actually waits — polls the outbox
  row until terminal status (timeout 30s, 100ms interval) on both
  /v1/events and /v1/events/batch. Returns `waitTimedOut: true` if the cap
  is hit so callers can re-poll the status endpoints.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(server-beta): address CodeRabbit + Greptile second review on PR #2383

P1 fixes
- Operator retry endpoint was re-publishing the Postgres outbox metadata
  column as the BullMQ payload; the worker's
  assertServerGenerationJobPayload always rejected it, leaving the row
  stuck in queued until startup reconciliation. Persist the BullMQ payload
  on the outbox row at create-time inside IngestEventsService and
  EndSessionService, then re-enqueue that canonical payload on retry.

Major fixes
- prompt-builder: escape server_session_id when interpolating into the
  XML prompt; previously a session id containing `<`, `&`, or quotes
  could inject XML into the provider input.
- ServerJobQueue: route both worker.on('stalled') and the QueueEvents
  'stalled' subscriber through a single notifyStalled helper that
  dedupes by jobId for 30s, so counters.stalled increments once per
  stall. QueueEvents 'error' now routes through notifyQueueError so
  it increments counters.errored and runs onError listeners — keeping
  observability symmetric across both sources.
- ServerV1PostgresRoutes: convert PostgresObservationRepository from
  three dynamic imports to a single static import for consistency.
- mcp-server / ServerBetaClient: actually forward the
  observation_record_event tool's `generate` flag through to the
  /v1/events endpoint as `?generate=false` instead of voiding it.
- server-sessions.markGenerationFailed: guard jsonb_set against a null
  error payload so the failure path can't null out metadata before the
  generation_status='failed' write commits.

Minor fixes
- server-sessions.endSession: keep updated_at stable on repeated calls
  so the documented idempotency contract holds.
- SettingsDefaultsManager + ServerBetaService.getServerBetaPort: derive
  the server-beta default port from UID (37877 + uid%100), matching the
  worker port pattern, so two users on the same host don't collide.
  Docker stacks always pass CLAUDE_MEM_SERVER_PORT explicitly so the
  containerized deployment is unaffected.
- server-session-runtime test: close the pg.Pool in afterAll.
- server-beta-release-readiness.md: escape pipes inside table inline
  code, add `text` language tag to the fenced log block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(server-beta): address Greptile + CodeRabbit third review on PR #2383

P1 fixes
- SessionsObservationsAdapter.resolveServerSession: catch unique-violation
  (23505) on concurrent compat inserts and re-fetch instead of returning
  500. Two compat callers carrying the same contentSessionId can both
  observe `existing===null` and race on the (project_id,
  external_session_id) unique constraint; the second now resolves to the
  raced row instead of dropping the event.
- /v1/events/batch: pass `sourceAdapter: null` to ingestBatch so each
  event's BullMQ payload (and persisted outbox payload column) reflects
  its own event.sourceAdapter via buildEventBullmqPayload's fallback,
  rather than stamping the whole batch with the first event's adapter.

Minor
- server-session-runtime test afterEach: wrap DROP SCHEMA in try/finally
  so client.release() always runs even if the drop throws.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): drop `pool as never` cast — pg.Pool already matches PostgresPool

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(server-beta): retry of completed job now 409s instead of duplicating

retryGenerationJob previously fell through to the reset+re-enqueue path
when called on a job in `completed` status. The observations index
dedupes on (generation_job_id, parsed_observation_index, content) but
LLM output is non-deterministic, so a second provider run almost always
produced a different content string and bypassed the index, persisting a
parallel set of observation rows attributed to the same generation job.

Match cancelGenerationJob's 409 guard for completed jobs. failed and
cancelled remain valid retry targets.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* build(server-beta): rebuild bundles after rebase onto main

Regenerates the three plugin bundles so they reflect the rebased source
state. Mechanical rebuild output only — no source changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(server-beta): wrap resolveServerSession in try/catch for structured error response

Greptile P1 on PR #2383: resolveServerSession was called before the try/catch
in both compat adapters, so Postgres errors during session lookup (timeout,
pool exhaustion, etc.) escaped to Express's default error handler and returned
HTML/text 500s. Legacy clients calling response.json() would get a parse
failure instead of the documented { stored: false, reason: 'internal_error' }
(or { status: 'error', reason: 'internal_error' } for the summarize adapter)
shape.

Move the resolveServerSession call inside the existing try block in both
adapters so any failure flows through the structured catch handler.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(server-beta): catch 23505 unique violation in POST /v1/sessions/start

Greptile P1 on PR #2383: concurrent requests with the same externalSessionId
can both pass the findByExternalIdForScope check, both call repo.create,
and the loser hits the (project_id, external_session_id) unique constraint.
The handler treated that as an unknown error and returned a 500.

Apply the same pattern resolveServerSession already uses: catch error.code
'23505' when externalSessionId is set, refetch the row inserted by the
winning request, and return 200 with that session.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-11 00:26:11 -07:00
Alex Newman 36b0929fae Server-beta: Postgres storage + independent runtime + BullMQ queue (Phases 1–3) (#2351) 
* Add server beta runtime foundation

* Address server beta review findings

* Resolve server beta review comments

* Tighten server beta review follow-ups

* Harden server beta auth and search

* Avoid unnecessary FTS rebuilds

* Block scoped keys from creating projects

* Release BullMQ claims best effort on close

* Address server beta review blockers

* Reset BullMQ claims best effort

* Add Postgres observation storage foundation

* feat(server-beta): add independent runtime service

Introduce src/server/runtime/ as a self-contained server-beta runtime
that owns its lifecycle, Postgres bootstrap, and HTTP boundary without
depending on WorkerService.

ServerBetaService wraps the existing Server class, exposes
/healthz and /v1/info with runtime="server-beta", and persists state
to dedicated paths (.server-beta.pid|.port|.runtime.json). The four
boundary managers (queue, generation worker, provider registry, event
broadcaster) are intentionally disabled in this phase and report their
status through /v1/info; later phases activate them.

Adds plans/2026-05-07-finish-bullmq-branch-ship-plan.md to track the
remaining work for this branch.

Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): route CLI lifecycle and bundle separate runtime

scripts/build-hooks.js now produces plugin/scripts/server-beta-service.cjs
as a separate Node CJS bundle, alongside the existing worker-service
bundle. The server-beta runtime is now installable independently.

src/npx-cli/commands/server.ts routes start|stop|restart|status to the
server-beta lifecycle instead of the legacy worker. The worker keeps its
own start|stop|restart|status under the worker namespace; the two
runtimes can be operated independently.

src/services/worker-service.ts adds a server-* command parser branch
that delegates to the sibling server-beta-service.cjs bundle so
direct worker-service invocations still route to the right runtime.

tests/npx-cli-server-namespace.test.ts updated to expect server-beta
lifecycle routing.

Includes rebuilt plugin/scripts/*.cjs bundles produced by
build-and-sync.

Phase 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): add BullMQ job queue primitives

Introduce src/server/jobs/ as the queue-side primitives that Phase 3 of
the server-beta runtime needs to operate.

types.ts defines a discriminated union over the four job kinds (event,
event-batch, summary, reindex) and maps each to a per-kind BullMQ queue
name and deterministic-ID prefix.

job-id.ts builds deterministic, colon-free BullMQ jobIds from
(kind, team, project, source). The colon ban exists because BullMQ uses
':' as a Redis key separator internally; embedding ':' in jobIds
breaks scan and state lookups.

ServerJobQueue.ts is a thin wrapper over BullMQ Queue + Worker that
enforces autorun:false, default concurrency 1, and an attached error
listener — all per BullMQ docs requirements. Test seams accept queue
and worker factories so unit tests do not need Redis.

outbox.ts publishes through the Postgres ObservationGenerationJob
repository as canonical history. enqueueOutbox writes the row first,
then publishes to BullMQ; if BullMQ throws, the row is transitioned to
failed and a failed event is appended. reconcileOnStartup re-enqueues
queued + processing rows after a restart, replacing terminal BullMQ
jobs that may still be holding the deterministic ID slot. markCompleted
and markFailed wrap transitionStatus and append the matching event row.

Includes 20 unit tests covering deterministic ID stability, colon-free
output, queue lifecycle, error-listener attachment, double-start
refusal, idempotent enqueue, BullMQ failure rollback, startup
reconciliation, max-attempts skipping, and completion / failure /
retry transitions.

Phase 3 commit 1 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(server-beta): activate queue boundary in runtime service

Wire ActiveServerBetaQueueManager into the server-beta runtime graph.
The active manager owns one ServerJobQueue per generation kind (event,
event-batch, summary, reindex) and surfaces lane metadata through
boundary health.

Selection is opt-in and fail-fast: if CLAUDE_MEM_QUEUE_ENGINE is set to
bullmq the active manager is constructed (and any Redis/config error
throws — no silent fallback to SQLite, per Phase 3 anti-pattern guard).
For any other engine the disabled boundary remains so worker-era and
test setups stay compatible.

Widens ServerBetaBoundaryHealth.status to a discriminated union
('disabled' | 'active' | 'errored') with optional details. The disabled
adapter still emits status='disabled', which keeps the existing
server-beta-service test green.

ServerBetaService receives the manager through a new optional
queueManager field on CreateServerBetaServiceOptions so test graphs
and Phase 4 wiring can inject custom managers.

Adds tests/server/runtime/active-queue-manager.test.ts covering bullmq
guard, active health shape, per-kind queue access, close behavior, and
post-close errored health.

Phase 3 commit 2 of plans/2026-05-07-server-beta-independent-bullmq-observation-runtime.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(server-beta): cap /v1/events/batch at 500 events

Prevents unbounded array DoS surface flagged in PR review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-08 01:20:07 -07:00
Alex Newman 65f2fd8cdd fix: harden startup and schema repair contracts 
Reliability patch covering startup path resolution, install marker compatibility, export CLI request contracts, schema repair safety, hard-stop retry-loop handling, and the PR babysit status helper.
 2026-05-06 18:29:26 -07:00
Alex Newman c4097b4ebb add Claude SDK gateway installer setup 2026-05-04 20:21:36 -07:00