8 Commits

Author SHA1 Message Date
Rohit Ghumare 249d8ff032 feat(commits): link agent sessions to git commits (#498)
* feat(commits): link agent sessions to git commits

Adds a commit-link layer so memories captured during an agent session
can be traced back to the git commit they produced, and vice versa.

What ships:

- KV.commits namespace storing CommitLink records keyed by full SHA.
- Session.commitShas[] back-reference for fast forward lookup.
- POST /agentmemory/session/commit upserts a link (sha, branch, repo,
  message, author, authoredAt, files, sessionId); merges sessionIds on
  re-link and preserves linkedAt.
- GET /agentmemory/session/by-commit?sha= hydrates the link plus its
  sessions.
- GET /agentmemory/commits?branch=&repo=&limit= lists recent links,
  filtered and sorted desc by linkedAt, limit clamp 1..500 (default 100).
- src/hooks/post-commit.ts shells git rev-parse/log/diff-tree and POSTs
  to /agentmemory/session/commit. Honors AGENTMEMORY_URL,
  AGENTMEMORY_SECRET, AGENTMEMORY_CWD, AGENTMEMORY_SESSION_ID,
  AGENTMEMORY_COMMIT_SHA. Best-effort, 1500 ms timeout. Wired into
  tsdown hookEntries so it bundles to dist/hooks/ and plugin/scripts/.
- Two MCP tools: memory_commit_lookup, memory_commits.
- Four user-invocable plugin skills:
  - commit-context: traces a file/function/line via git blame plus
    memory_commit_lookup plus memory_recall.
  - commit-history: lists agent-linked commits; parses branch=/repo=/
    limit= from $ARGUMENTS.
  - handoff: resumes the most recent session for cwd; surfaces an
    unanswered user-facing question before the brief.
  - recap: summarizes the last N sessions for cwd, grouped by date;
    parses last <n> / today / this week.

Wire-up to capture from a real repo:

    ln -sf "$(realpath node_modules/agentmemory/dist/hooks/post-commit.mjs)" .git/hooks/post-commit
    chmod +x .git/hooks/post-commit

No new dependencies. Additive only.

* fix(commits): address CI + review findings on commit-link feature

- Update mcp-standalone.test.ts to expect 14 CORE_TOOLS (was 12) now that
  memory_commit_lookup and memory_commits are registered. Restores CI on
  Node 20 and 22.
- Wrap CommitLink upsert and session.commitShas mutation in withKeyedLock
  to prevent lost updates when concurrent post-commit hooks race against
  each other or against api::session::commit callers. Locks keyed by
  `commit:<sha>` and `session:<sessionId>` so unrelated writers do not
  serialize.
- Parallelize hydration of linked sessions via Promise.all in both
  api::session::by-commit and the memory_commit_lookup MCP handler;
  serial kv.get loops dragged response time for commits with many
  sessions.
- Sharpen handoff SKILL.md project-path matching: resolve relative
  $ARGUMENTS to an absolute normalized path and require directory-
  boundary equality, not raw string prefix, so sibling repos sharing a
  prefix do not collide.
- Sharpen commit-history SKILL.md HTTP fallback: build the
  /agentmemory/commits URL with URL-encoded query values so branch and
  repo values containing `?`, `&`, or `#` cannot corrupt the request.

* chore(consistency): bump documented tool and endpoint counts

Bring README.md, AGENTS.md, and the boot log in src/index.ts in line
with the live tool registry (53 MCP tools, +2 from this branch) and
the live REST registration (124 endpoints, +3 from this branch).

The Consistency test in test/consistency.test.ts derives both counts
from source and asserts the markdown matches; this commit only updates
the literal numbers, no behavior change.
2026-05-18 11:19:47 +01:00
이민재 705321be34 docs: refresh current project stats (#330)
* docs: refresh current project stats

* docs: update REST endpoint badge count

* docs: refresh stats for v0.9.16

---------

Co-authored-by: 이민재 <19909783+honor2030@users.noreply.github.com>
2026-05-16 10:00:00 +01:00
Rohit Ghumare 14da6c816c feat: add token-efficient recall and file compression tool (#150)
* feat: add token-efficient recall and file compression tool

Add memory_recall format and token budget support, plus a new memory_compress_file tool with API/MCP wiring, tests, and updated tool/endpoint docs.

* chore: remove temporary architecture assets

Delete temporary architecture and memory-cycle asset files added during implementation.
2026-04-16 10:36:17 +01:00
Rohit Ghumare 5d70ecfb3c feat: migrate agentmemory to iii v0.11 and add upgrade command (#116)
* feat: migrate agentmemory to iii v0.11 and add upgrade command

Migrate the codebase from legacy iii-sdk v0.3 APIs to v0.11 trigger/register patterns, update runtime configs, and align stream/state behavior with newer engine semantics. Add a new `agentmemory upgrade` CLI command so users can refresh dependencies and iii runtime components with one entrypoint.

* chore: remove pnpm lockfile from migration PR

Drop pnpm-lock.yaml from this branch to keep the migration PR focused on source and config changes only.

* fix(ci): sync npm lockfile with iii-sdk v0.11 dependency

Update package-lock.json so npm ci in CI matches package.json after the iii-sdk migration.

* fix(ci): resolve build parse errors after v0.11 migration

Fix malformed braces introduced during API migration in triggers/health/branch-aware files so tsdown build passes in CI.

* fix: harden migration follow-up fixes and audits

Apply the reviewed v0.11 follow-up fixes across runtime, docs, and tests by tightening validation, correcting upgrade/error handling, and adding missing audit coverage on state mutations. This also addresses stream fallback behavior, lock consistency, retention source-bucket cleanup, and test/mock alignment so build and test remain green.

* fix: address 9 unresolved CodeRabbit findings on iii v0.11 migration

CodeRabbit flagged 9 remaining issues on #116 — all real. Each was
verified against the current branch code before applying. Two
findings were deliberately skipped as policy/scope issues and are
documented at the bottom.

### Applied (9 real findings)

- src/triggers/api.ts — api.ts numeric query param validation:
  multiple sites forwarded `parseInt(params.limit)` result to the
  downstream function without a Number.isFinite check. A non-numeric
  query value produced NaN at the iii-sdk trigger boundary. Added
  parseOptionalInt and parseOptionalFloat helpers at the top of the
  file, wired into api::crystal-list, api::lesson-list, and
  api::insight-list (5 sites total).

- src/triggers/api.ts — api::observe, api::context, api::session::start,
  api::session::end were forwarding req.body verbatim to sdk.trigger
  with no validation. Added explicit type checks mirroring the existing
  api::search pattern, construct a sanitized payload object before
  triggering.

- src/cli.ts — p.confirm() can return a cancel Symbol on Ctrl+C, which
  is truthy, so the upgrade path ran even when the user cancelled.
  Added p.isCancel() check and explicit boolean comparison.

- src/cli.ts — removed dead `failed` flag in runUpgrade: every caller
  already calls process.exit(1) immediately, so the final `if (failed)`
  branch was unreachable. Simplified requireSuccess accordingly.

- src/functions/leases.ts — mem::lease-renew recorded audit events as
  "lease_acquire", which conflated renewals with first claims. Renamed
  the audit event to "lease_renew" so downstream audit filters can tell
  the two operations apart.

- src/functions/relations.ts — the pair lock
  `mem:${firstId}:${secondId}` serialized concurrent relate(A,B) calls
  with identical pairs, but did NOT protect concurrent relate(A,B) +
  relate(A,C). Both modify memory A's `relatedIds` array, which is a
  classic last-writer-wins race producing lost relation edges. Fixed
  by replacing the pair lock with nested per-entity locks in canonical
  sort order: withKeyedLock(mem:firstId) wrapping withKeyedLock(mem:secondId).
  Since the ids are sorted deterministically, no deadlock is possible.

- src/functions/sketches.ts + src/functions/summarize.ts + new
  src/functions/audit.ts safeAudit helper — recordAudit was awaited
  after kv.set calls. An audit write failure would reject and the
  caller would see an error even though the target state was already
  persisted. New safeAudit wrapper swallows audit errors and logs them
  via ctx.logger.warn, preserving the mutation's success. Applied to
  all 11 audit sites in sketches.ts and the single site in summarize.ts.

- src/mcp/server.ts — three issues in the MCP handler:
  1. memory_profile's refresh flag used `args.refresh === "true"`,
     ignoring boolean `true` from MCP clients that send proper types.
     Now accepts both.
  2. memory_sketch_create built sketchPayload with
     `asNonEmptyString(args.title)` which could return undefined,
     then forwarded that to the downstream function. Added explicit
     validation + 400 response at the MCP boundary.
  3. memory_recall, memory_team_feed, memory_audit_query used
     `(args.limit as number) || 10` which replaced an explicit 0 with
     10. Changed to typeof check so explicit 0 (rare but legal) is
     preserved.

- src/functions/governance.ts — mem::governance-bulk used Promise.all
  for the delete batch, which fails fast on the first error. The audit
  record still said `deleted: candidates.length` even if only half
  actually succeeded. Switched to Promise.allSettled, split results
  into successfulIds and failures arrays, record audit with both counts
  plus the per-failure details for traceability.

- src/functions/mesh.ts — mem::mesh-register, mem::mesh-sync,
  mem::mesh-receive, mem::mesh-remove all dereferenced `data.*` without
  checking that data was passed. A TypeError would bubble up on null
  payload. Added early null/type guards returning structured errors.

- src/functions/obsidian-export.ts — resolveVaultDir(data.vaultDir)
  was called without validating that vaultDir was a string, and
  `new Set(data.types)` without validating types was an array of
  strings. Added explicit validation returning 400-style error
  responses.

- src/functions/retention.ts — the eviction loop silently swallowed
  kv.delete errors via a bare `continue`. Added ctx.logger.warn on
  catch, included memoryId and sourceBucket in the log, and now
  returns `failed` count alongside `evicted` in the response.

- src/viewer/index.html — two WebSocket bugs:
  1. connectWs assigned to the mutable global state.ws before binding
     handlers, so old sockets could have their callbacks fire and
     mutate retry/direct state after state.ws was overwritten by a
     new socket. Fixed by creating a local `ws` variable, binding
     all handlers to it, only then assigning state.ws = ws. Each
     handler guards `if (state.ws !== ws) return` so stale callbacks
     are dropped.
  2. handleStreamEvent routed EVERY incoming event to routeWsMessage,
     so non-observation events (session.activity, etc.) were being
     treated as timeline observations and causing UI confusion. Added
     a looksLikeObservation helper + event_type gate that only routes
     real observation payloads.

- test/retention.test.ts — added an explicit sourceBucket eviction
  test that seeds a semantic memory at high age, runs retention-score,
  then runs retention-evict at high threshold and asserts BOTH
  KV.semantic and KV.memories are empty. Proves the candidate.sourceBucket
  branch added in this PR actually routes to the right bucket.

- src/types.ts — side fix: the ExportData.version union on line 254
  used comma separators instead of pipes in 3 positions (0.7.9,
  0.8.0, 0.8.1 → needed | between them). tsdown strips types so the
  build passed, but tsc --noEmit would have thrown and any IDE showed
  squiggles. Pre-existing latent bug, fixed while in the file.

### Deliberately skipped

- retention.ts eviction audit (CodeRabbit asked to add recordAudit
  to the eviction loop): policy change, not a bug fix. Verified:
  auto-forget.ts, evict.ts, retention-evict, remember-forget all
  skip audit by convention — only governance audits. Adding audit
  everywhere is a policy decision tracked in issue #125.

- file-index.ts sequential → parallel session lookup: micro-opt,
  the loop is already cache-backed, negligible savings, not worth
  the readability cost.

Tests: 655/655. Build clean.

* fix(api): harden request validation at HTTP boundaries

Validate and sanitize session, observe, and context inputs plus numeric query params before forwarding to memory functions, returning 400 for invalid values instead of propagating malformed payloads. Also remove duplicate CLI command registration introduced during merge resolution.

* fix: address 6 new CodeRabbit findings on iii v0.11 migration (#116 round 2)

CodeRabbit's second review pass on #116 flagged 6 real issues introduced
by the previous round of fixes (commit 3e3ac7e). Each was verified
against the current code before applying.

### Fixed

- src/functions/governance.ts — mem::governance-bulk switched to
  Promise.allSettled in the previous round but fanned out every
  kv.delete concurrently. The governance endpoint can target tens of
  thousands of memories in a single call; firing them all at once
  overwhelms the state worker. Batched with BATCH_SIZE=50 chunks,
  awaited serially while preserving the per-batch allSettled so a
  single failure doesn't abort the rest.
- src/functions/governance.ts — the recordAudit call AFTER the bulk
  delete is already committed was still using `await recordAudit`, so
  an audit write failure would bubble up as a "failed" response even
  though the rows were actually deleted. Switched to safeAudit so the
  deletes are reflected faithfully in the response regardless of
  audit health.
- src/functions/relations.ts — the mem::relate handler was emitting
  three recordAudit calls INTERLEAVED with three kv.set writes:
  relation → audit → source update → audit → target update → audit.
  A thrown audit in the middle would leave KV.relations and
  source/target.relatedIds in a partial state. Restructured to
  complete all three durable writes first, then emit a single
  safeAudit at the end. Also changed the operation from the generic
  "evolve" (which made audit queries indistinguishable from actual
  version evolutions) to a new "relation_create" event, added to the
  AuditEntry.operation union in src/types.ts.
- src/functions/retention.ts — mem::retention-score used
  `{ ...DEFAULT_DECAY, ...data.config }` which is only a shallow
  merge. A caller passing `{ config: { tierThresholds: { cold: 0.2 } } }`
  would drop the hot and warm thresholds, causing every downstream
  `s.score >= config.tierThresholds.hot` comparison to produce NaN
  and misclassify every tier bucket. Deep-merged tierThresholds
  explicitly.
- src/functions/retention.ts — mem::retention-evict previously
  used `candidate.sourceBucket || KV.memories` as the delete bucket.
  That's correct for new retention scores (which now store
  sourceBucket) but wrong for pre-migration scores whose sourceBucket
  is undefined: the fallback would try to delete semantic ids from
  KV.memories and silently no-op, leaving the real semantic memory
  alive. Fixed by splitting the delete path: if sourceBucket is set,
  use it; otherwise attempt both KV.memories and KV.semantic with
  individual .catch(() => {}) wrappers so whichever bucket actually
  contains the id succeeds and the other is a no-op. Legacy rows
  retire naturally on the next scoring run, which writes sourceBucket.
- src/triggers/api.ts — api::obsidian-export was validated at the
  downstream mem::obsidian-export layer in the previous round, but
  the API boundary handler was still forwarding body.vaultDir without
  a type check. Added the same 400 response pattern used by
  api::search / api::observe / api::context for consistency.

### Skipped from CodeRabbit's suggestions

- CodeRabbit's proposed fix for retention.ts:249 was
  `candidate.sourceBucket || KV.retentionScores` which would try to
  delete the memory from the retention scores namespace — that's
  where the scoring entry lives, not where the memory lives. That
  fix is wrong. The two-bucket fallback approach above is the
  correct one.

Tests: 655/655. Build clean.

* fix(agentmemory): harden deletion, relation, and API validation paths

Bound retention/governance side effects and make audit logging best-effort so successful state writes are not masked by telemetry failures. Also tighten boundary validation for MCP/API inputs, resolve relation lock edge cases, and align retention test mocks with v0.11 trigger/registerFunction call shapes.

* fix: address remaining CodeRabbit validation and eviction issues

Remove duplicate CLI help examples, harden eviction/delete bookkeeping, allow file-context without sessionId, sanitize governance failure output, validate retention-evict inputs with bounded limits, and tighten summarize/observations/obsidian-export input validation.

* fix(v0.11): align stream send payloads and migration docs

Use the engine-compatible stream payload key `type` for stream::send events and update docs/plugin examples to reflect v0.11 function registration and trigger request shapes.
2026-04-15 14:02:22 +01:00
Rohit Ghumare 9c27eff610 feat: implement reflect — auto-synthesize insights from memory clusters (#89)
Adds mem::reflect: traverses knowledge graph, groups related memories
by concept clusters, synthesizes higher-order insights via LLM.

New files:
- src/prompts/reflect.ts — LLM prompt for insight synthesis
- src/functions/reflect.ts — 4 functions: reflect, insight-list,
  insight-search, insight-decay-sweep
- test/reflect.test.ts — 13 tests

New type: Insight (confidence-scored, fingerprint-deduped, decayable)
New KV scope: mem:insights
New MCP tools: memory_reflect, memory_insight_list (in ESSENTIAL set)
New REST: POST /reflect, GET /insights, POST /insights/search
Hooks into consolidation pipeline as "reflect" tier after semantic

Review fixes applied:
- Fixed project filter no-op (was discarding filter result)
- Fixed BFS depth tracking (standard level-count pattern)
- Added attribute-order constraint to LLM prompt
- Whitelisted fields in api::insight-search (no raw body passthrough)
- Fixed Jaccard fallback to compare document-id sets not concept strings

43 MCP tools, 103 REST endpoints, 627 tests passing
2026-04-06 16:08:34 +01:00
Rohit Ghumare 5510e94e39 fix: simplify review — temporal-graph name, test precision, stale counts
- Fix README: temporal-graph → temporal-graph-extract + temporal-query
- Fix test precision: toBeCloseTo(0.55, 1) → precision 2 (±0.005)
- Fix stale test counts: 581 → 614 across README and AGENTS.md
- Add obsidian export error-path test (errors undefined on success)

614 tests passing
2026-04-06 13:17:48 +01:00
Rohit Ghumare e26320e3aa feat: auto-install iii-engine, clack CLI, viewer lessons tab, README update
CLI (src/cli.ts):
- Auto-install iii-engine via curl | sh with interactive clack/prompts
- Edge cases: Windows (manual install note), curl missing, PATH fallback
- Error shows all install methods: curl | sh, cargo install, Docker
- Spinner feedback for install, engine start, readiness check

Viewer (src/viewer/index.html):
- Version v0.5.0 → v0.7.0
- Added Lessons tab with search, confidence gauges, source badges
- Added Lessons + Crystals stat cards to dashboard
- Dashboard fetches lessons and crystals data

README:
- Quick start is now single command: npx @agentmemory/agentmemory
- Prerequisites: iii-engine via curl | sh (not npm)
- Updated all stale counts and version references

AGENTS.md: test count 573 → 581

581 tests passing, all builds clean
2026-04-05 11:28:49 +01:00
Rohit Ghumare e8f410b537 feat: v0.7.0 — lessons, tool visibility, auto-consolidation, obsidian export, npx bootstrap (#82)
* feat: v0.7.0 — lessons, tool visibility, auto-consolidation, obsidian export, npx bootstrap

Five DX improvements based on competitive research against Mem0, Engram, CodeMem:

1. `npx agentmemory` zero-config startup
   - New CLI bootstrap (src/cli.ts) auto-detects and starts iii-engine
   - Tries `iii` binary first, falls back to `docker compose up -d`
   - Bundles iii-config.yaml and docker-compose.yml in dist/

2. Simplified MCP tool surface (7 core tools by default)
   - AGENTMEMORY_TOOLS=all unlocks all 49 tools
   - Default: save, recall, consolidate, forget, sessions, diagnose, lesson_save
   - Call handler remains unfiltered — any tool callable by name

3. Auto-consolidation on session end
   - CONSOLIDATION_ENABLED defaults to true (was false)
   - Session-end hook: session/end → crystallize → consolidate → bridge sync
   - Consolidation pipeline always registered (timer gated by config)

4. First-class lesson memory type with confidence decay
   - Lesson interface: confidence (0-1), reinforcements, decayRate, source
   - 5 functions: lesson-save, lesson-recall, lesson-list, lesson-strengthen, lesson-decay-sweep
   - Dedup via SHA-256 fingerprint — duplicate saves strengthen existing
   - Crystal lessons auto-flow into lesson system at confidence 0.6
   - Daily decay sweep with parallel KV writes

5. Obsidian-compatible Markdown export
   - Export to ~/.agentmemory/vault/ with YAML frontmatter + wikilinks
   - MOC.md (Map of Content) index file
   - Parallel KV reads, auto-export via OBSIDIAN_AUTO_EXPORT=true

Stats: 49 MCP tools, 99 REST endpoints, 573 tests passing

* docs: add AGENTS.md, fix tool/endpoint counts across README, plugin, cli

- Create AGENTS.md with strict consistency rules for MCP tools, REST
  endpoints, versions, KV scopes, and audit operations
- Fix MCP tool count: 38 → 41 across README.md (4 occurrences)
- Fix REST endpoint count: 93/95 → 99 across README.md and index.ts
- Fix plugin.json: version 0.6.1 → 0.7.0, tool count 5 → 41
- Fix cli.ts help text: 48+ → 41 MCP tools
- Fix README api.ts path reference → triggers/api.ts

* fix: address code review findings — consolidation guard, decay bug, async fs, export-import lessons

Inline fixes:
- Revert CONSOLIDATION_ENABLED to opt-in (=== "true"), matching original behavior
- Add early-exit guard in consolidation-pipeline handler when disabled
- Gate session-end crystallize+consolidation calls on CONSOLIDATION_ENABLED
- Fix lesson decay over-decay bug: add lastDecayedAt to Lesson, compute
  incremental delta instead of reapplying full age every sweep run
- Add LESSON_DECAY_ENABLED flag (default true) to gate the sweep timer

Outside diff fixes:
- Add lessons to export-import (export + import + replace cleanup)
- Log warning instead of swallowing obsidian auto-export errors

Nitpick fixes:
- Switch obsidian-export to async fs/promises (mkdir, writeFile)
- Add per-item try/catch in obsidian-export, return errors array
- Replace governance_delete with smart_search in ESSENTIAL_TOOLS (non-destructive default)
- Fix CLI whichBinary for Windows (uses "where" on win32)
- Use dynamic port in CLI error message instead of hardcoded 3111
- Return 201 for newly created lessons, 200 for strengthened
- Wrap lesson audit calls in try/catch so audit failure doesn't surface
- Fix build script to not swallow tsdown failure
- Remove exact tool count from test, use >=41 + uniqueness + required names
- Add test/consistency.test.ts: validates version, tool count, README consistency
- Add isConsolidationEnabled mock to consolidation-pipeline test

579 tests passing (573 original + 6 new consistency checks)

* fix: guard remaining audit calls, correct endpoint count, add CI + npm publish

Review fixes:
- Wrap lesson_recall and lesson_strengthen audit calls in try/catch
- Fix REST endpoint count: 99 → 100 (verified via grep) across index.ts,
  README.md, and AGENTS.md
- Use regex in consistency test for README assertions (flexible phrasing)
- Add consolidation gate tests: disabled returns early, force=true bypasses

CI/CD:
- Add .github/workflows/ci.yml — test on Node 18/20/22
- Add .github/workflows/publish.yml — auto-publish to npm on GitHub release
  (uses NPM_TOKEN secret + provenance)

Nitpick:
- Single-char term filter (t.length > 1) kept intentionally — prevents noise
  from single-letter matches; documented in AGENTS.md if needed

581 tests passing

* fix(ci): add --legacy-peer-deps for zod v3/v4 peer conflict

@anthropic-ai/claude-agent-sdk@0.2.56 requires zod@^4.0.0 as a peer
dependency but the project uses zod@^3.23.0. The lock file resolves
this locally but npm ci is strict about peer deps in CI.

* fix(ci): drop Node 18 from matrix — tsdown requires Node 20+

tsdown/rolldown uses node:util.styleText which is only available in
Node 20.12+. Updated engines field to >=20.0.0 to match.

* fix(ci): add inlineOnly: false to tsdown config, target node20

tsdown errors on CI with "Consider adding inlineOnly option" when
dependencies are bundled. Setting inlineOnly: false suppresses this.
Also updated target from node18 to node20 to match engines field.
2026-04-04 17:57:01 +01:00