Completes the Phase 1 ingest + retrieval loop per ADR-0243 build order step 7.
apps/web
- new app/api/runs/search/route.ts — hybrid Typesense search with the
full agent-first API surface (Rule 12 + 13):
* Auto-applied privacy filters (user_id + readable_by) from the
bearer token — NEVER from the request body (Rule 4)
* Tag filters default to AND semantics
* Modes: hybrid (default), semantic, keyword
* Embeds the query at priority="query" so it preempts any pending
ingest work in the Ollama queue (Rule 9a)
* HATEOAS envelope with _links, next_actions, and timing breakdown
(query_embed_ms, query_queued_ms, typesense_ms, total_ms)
scripts/memory-spike
- new smoke-e2e.ts — full pipeline validation:
1. POST /api/runs with a real claude-code jsonl fixture
2. Poll Typesense for chunks matching the new run_id
3. POST /api/runs/search filtered to the smoke-e2e tag
4. Assert the ingested Run is findable
Run against a live dev stack (Next.js on :3000, worker deployed,
Typesense + Ollama + Inngest healthy).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces the raw-Run capture architecture (ADR-0243 in the Vault):
- CONTEXT.md at repo root — 13 terms + 21 architectural rules grilled out via the
domain-model skill (central ingestion, private-by-default Share Grants,
PDS+App Password identity, NAS-authoritative storage with Typesense as
rebuildable index, qwen3-embedding:8b @ 768-dim Matryoshka, agent-first API).
- packages/memory/ — types, per-turn chunker (claude-code + pi format detection,
tool-result role fix), Ollama embedding client with concurrency pool, Typesense
run_chunks schema, barrel exports. Mirrors @joelclaw/telemetry hexagonal pattern.
- scripts/memory-spike/ — end-to-end validation against real data:
* ingest.ts ingests a single jsonl into run_chunks_spike
* bulk-ingest.ts walks ~/.claude/projects/ and ~/.pi/agent/sessions/ with
sha256 dedup via ~/.joelclaw/memory-spike-ingested.jsonl
* search.ts supports hybrid / semantic / keyword modes
* README.md documents observations from the validation run (~708 chunks from
a 1247-line claude-code session at 1.2 ch/s sequential / 0.9 ch/s observed
under bulk concurrency; ~420ms end-to-end semantic queries; retrieval
quality validated on real Runs).
Typecheck clean, biome clean. No production infrastructure affected; spike
writes to isolated run_chunks_spike Typesense collection on the joelclaw
cluster (cleanly deletable).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
shitratgit[bot]