857f71e3c6
feat: v0.6.0 advanced retrieval with real-world benchmarks ( #76 )
...
* feat: add 9 orchestration modules for v0.5.0
Add actions, frontier, leases, routines, signals, checkpoints,
flow-compress, mesh, and branch-aware modules with full MCP tools,
REST endpoints, and 170 new tests. Includes SSRF protection,
race-condition-safe keyed mutex locking, SHA-256 fingerprinting,
and fixes for 29 CodeRabbit review findings.
- 9 new source files (src/functions/*)
- 8 new test files (170 tests, total 386)
- 10 new MCP tools, 23 new REST endpoints
- 8 new KV scopes, new types for orchestration
- Version bump to 0.5.0, README and viewer updated
* feat: add sentinels, sketches, crystallize, diagnostics, facets modules
5 new modules inspired by beads patterns but with original naming and
iii-engine real-time streaming (no polling):
- sentinels: event-driven condition watchers (webhook, timer, threshold,
pattern, approval) that auto-unblock gated actions via SSE
- sketches: ephemeral action graphs with auto-expiry, promote or discard
- crystallize: LLM-powered compaction of completed action chains into
compact crystal digests with key outcomes and lessons
- diagnostics: self-diagnosis across 8 categories (actions, leases,
sentinels, sketches, signals, sessions, memories, mesh) with auto-heal
- facets: multi-dimensional tagging (dimension:value) with AND/OR queries
- 5 new source files, 5 new test files (132 tests, total 518)
- 9 new MCP tools (total 37), 21 new REST endpoints (total 93)
- 4 new KV scopes (total 33), new types for all modules
- README stats and function table updated
* fix: address code review findings across v0.5.0 modules
- actions: validate edges before persisting, set blocked status for requires deps
- leases: use mem:action lock key, reject blocked actions, check expiry on release
- checkpoints: validate linkedActionIds exist, check requires edges in unblock
- mesh: add SSRF validation on peer registration
- routines: remove invalid "failed" action status check
- export-import: add v0.5.0 scope export/import (actions, sentinels, sketches, etc)
- mcp/server: validate CSV inputs are strings before splitting
- schema: replace runtime require with static import
- README: fix stale tool/endpoint counts (28→37, 72→93)
* fix: second round code review — mesh locks, routines DAG, MCP input validation, README counts
- mesh.ts: add withKeyedLock on action writes in receive path, add IPv6 private ranges to SSRF check
- routines.ts: validate DAG (duplicate orders, unknown deps), set dep actions to blocked, refresh stepStatus in routine-status
- checkpoints.ts: runtime type enum validation, set linked pending actions to blocked
- leases.ts: validate ttlMs is finite positive number
- export-import.ts: add skip strategy checks for all v0.5.0 import blocks
- mcp/server.ts: typeof guards on tags, config JSON.parse, actionIds, linkedActionIds, categories
- README.md: Tools 18→37, Functions 33→50, stats line updated
- test: update checkpoint test for new blocked-on-create behavior
* fix: third round review — lease renew/release safety, mesh locking, export replace cleanup, MCP input guards
- leases.ts: renew extends from max(now, existing expiry) instead of now; release verifies action ownership before mutation
- mesh.ts: withKeyedLock on memory writes in mesh-receive; applySyncData validates id/updatedAt and locks both memory and action writes
- routines.ts: stepStatus maps "blocked" to "pending" explicitly; progress includes blocked/cancelled counts; routine-freeze wrapped in withKeyedLock
- export-import.ts: remove unused RoutineRun import; replace strategy clears all orchestration namespaces; skip strategy for graphNodes/graphEdges/semantic/procedural
- mcp/server.ts: sentinel_trigger JSON.parse with typeof+try/catch; facet_query typeof guards on matchAll/matchAny; remove redundant requires cast; .filter(Boolean) on concepts/files/tags/requires CSV splits
- README.md: clarify API table is a representative subset
* fix: fourth round review — redirect SSRF, blocked-on-create, missing action handling, boolean normalization
- mesh.ts: add redirect:"error" to both outbound fetch calls to prevent SSRF via redirect
- routines.ts: create actions with status "blocked" directly when hasDeps (eliminates two-pass race); handle missing actions in routine-status as cancelled; progress.total uses run.actionIds.length
- mcp/server.ts: sentinel config accepts object values directly; normalize unreadOnly/dryRun for both JSON booleans and string values
- README.md: consistent bundle size (365KB) across both occurrences
* feat: v0.6.0 advanced retrieval — triple-stream search, stemming, real benchmarks
Search improvements:
- Porter stemmer for word normalization (authentication ↔ authenticating)
- 40+ coding-domain synonym groups (db ↔ database, k8s ↔ kubernetes)
- Binary-search prefix matching replaces O(n) full scan
- Session diversification (max 3 results per session)
- Co-occurrence graph edges between all concept pairs
New retrieval modules:
- Sliding window inference pipeline (context enrichment at ingestion)
- Adaptive query expansion (LLM-generated reformulations)
- Triple-stream search (BM25 + Vector + Graph with RRF fusion)
- Append-only temporal knowledge graph (versioned edges, point-in-time queries)
- Graph-augmented retrieval (entity search + neighborhood expansion)
- Ebbinghaus retention scoring (decay + tiered hot/warm/cold/evictable)
Real-world benchmarks (240 observations, 20 labeled queries):
- Quality eval: 64.1% recall@10 with Xenova embeddings (vs 55.8% grep)
- Scale eval: 92-100% token savings vs built-in memory at 240-50K observations
- Cross-session: 12/12 queries found vs 10/12 for 200-line MEMORY.md cap
- Token measurement uses actual search results (fixed fake constant bug)
- Removed old microbenchmarks (bench.ts, run-bench.ts, COMPARISON.md)
2026-03-18 08:47:35 +00:00
Rohit Ghumare