183 Commits

Author	SHA1	Message	Date
Nicolò Boschi	630c3a63e7	fix(reflect): grounding defects in reflect and knowledge-page delta refresh, plus system-evals (#4304) ... * test(dev): add a deterministic retrieval eval for reflect's forced prelude We had no baseline to judge a retrieval change against, so a change to how reflect's opening hierarchy (mental models -> observations -> recall) picks its queries could not be told apart from a regression. The corpus is authored, never extracted: every stored row's text is byte-identical to the YAML, which is what makes gold labelling possible at all. Facts and observations are retained with the mock provider scripted through set_response_callback; mental models are created with explicit content; staleness is produced by ORDERING (stale models created before the facts, fresh ones after) because it is derived, not stored. Questions are grouped by which layer should answer them -- mm_only, mm_stale, observations_only, raw_facts_only, multi_layer, near_miss, absent -- so the eval exercises the descent decision and not just one search. The facts carry deliberate near misses; a corpus of unrelated facts scores 1.0 for any query and measures nothing. Scored by rank WITHIN each layer. Set membership saturates once the corpus is smaller than one recall page, and a flat ranking scores every raw fact behind every mental model however good the query was. So recall@3 and MRR per layer, plus the short-circuit fire rate reported separately, since the mental-model query is what decides it. First deliverable is the floor/ceiling experiment, which needs no LLM in either arm: the question verbatim (the real fallback when planning fails) against a hand-written ideal query. First run: recall@3 floor=0.929 ceiling=1.000 gap=+0.071 MRR floor=0.762 ceiling=1.000 gap=+0.238 Wording moves retrieval, mostly through rank rather than presence, with the mm_stale question the largest mover (0.50 -> 1.00) -- the case where a wrong mental model has to be superseded by raw facts. Two behaviours it already surfaced, neither introduced here: - A created-but-never-refreshed mental model is always stale: staleness resolves from the refresh stamps and an unstamped model is reported stale unconditionally, so it can never short-circuit the descent until something refreshes it once. - One stale model suppresses the short-circuit for all of them, since the rule requires every returned model to be fresh. An unrelated stale model in the top-5 keeps the descent going even when a fresh model answers the question outright. * test(dev): grade reflect's actual answer, not just what it retrieved The retrieval eval scores the evidence set, never the prose reflect returns. That is a necessary condition -- reflect is grounded, so an unretrieved fact cannot be answered -- and nowhere near a sufficient one. The blind spot is the category the whole exercise is about. For the mm_stale question, retrieval can return the stale "Stripe" mental model AND the Adyen facts, score recall@k = 1.0, and the answer can still say Stripe because the model trusted the summary over the raw facts. The retrieval tier calls that a pass. So answer_eval.py runs reflect_async end to end on a real model and grades the text with an independent judge, scoring two things separately because they fail differently: - correct: meets answer_criteria. Missing it can just mean incomplete. - trap: asserts must_not_claim, the specific wrong answer the question baits. That is a grounding failure, and it is the number that matters -- a confidently wrong answer is worse than a hedged one. Every question runs N times and the output is a rate, not a verdict. Not for CI. The judge mirrors tests/llm_judge.py (independent model, majority confirmation on a "not met") but is reimplemented here because that module lives under tests/ and is not importable from this package; the eval warns when the judge and reflect resolve to the same model, since the local .env makes that the default and a model grading its own output agrees with itself. First run -- gemini-2.5-flash-lite reflecting, gemini-2.5-flash judging, 2 runs, budget=low, on this branch: overall correct 93.8%, trap rate 0% on every baited question multi_layer 50%, everything else 100% The multi_layer miss is real run-to-run variance rather than a judge artifact: one run named the platform-team handover, the other dropped it. This is one arm only. Comparing main against the branch on the same corpus, model and N is what actually settles the cold-query question, and is not done here. * test(dev): hard corpus, failure attribution, and runner env-precedence fixes WIP checkpoint before A/B testing the thought_signature failure. * test(dev): stop the outage corpus generating two "April 2026 outage" rows The numeric_precision cluster cycled months with `i % 12` and years with `i // 12`, which produced a second row claiming to be THE April 2026 outage with different values (850 connections / 87 minutes vs the gold's 200 / 47). The question then had two contradictory answers, and reflect reporting "conflicting information" -- exactly what its Conflicts and Ambiguity rules prescribe -- was scored as a failure. The corpus was wrong, not the answer. I reported it as a reflect defect before checking; it was mine. Near-misses must differ in what they ASSERT, never in what they claim to BE. Each outage now owns a distinct (month, year) slot with April 2026 reserved for the gold row, and `_assert_subjects_are_unique` fails the build when two rows in a cluster name the same subject -- verified by re-introducing the collision, which the guard catches. * fix(reflect): don't manufacture a value for a period the memories don't cover Asked for an engineering headcount in a year the bank held no data for, reflect extrapolated backwards from the following year's monthly figures and answered with a specific number -- calling it "reliably inferred" and "reliably deduced". Three runs out of three, on gemini-3.7-flash. That is not a hedge: it is a fabricated data point wearing the language of certainty, and it is worse than "not recorded" because a reader cannot tell the difference. The prompts asked for it. Every path that writes an answer said some version of "if the exact answer isn't stated, use what IS stated to give the best possible answer", with "only say you don't have information if the retrieved data is truly unrelated" closing the escape hatch. A neighbouring year IS related, so declining was effectively disallowed. The missing distinction: inference may CHARACTERISE what the data covers; it may not MANUFACTURE a value for something the data does not cover. _GROUNDING_BOUNDARY states that, and is shared by all three answer paths (the tool-loop system prompt, the forced-synthesis system prompt, and the final-synthesis instructions) so they cannot drift apart. It explicitly preserves qualitative inference, because a rule read as "never infer" would break the synthesis that makes reflect worth having. Tests split per the convention: the wiring is deterministic and asserted directly, including that the rule keeps its teeth and its carve-out. The behavioural pair is marked hs_llm_core and its docstring says plainly what it does NOT do -- it does not reproduce the incident (verified: it passes against the pre-fix prompt on two models), it guards the contract, and its more valuable half is the check that the rule has not become a refusal reflex. Reproduction lives in hindsight-dev/benchmarks/prelude (hq-absent-2024). 488 reflect/prompt tests pass; the golden prompt fixture is updated. * test(dev): accept reflect's accurate UK qualification on the scoped_truth question The criteria said 2FA is "mandatory for EU (and UK) accounts", which reads as unconditional. The memory actually says UK accounts are mandatory FROM 2026, and reflect answered "UK Accounts: Mandatory starting in 2026" -- more precise than the criteria, and marked wrong for it (1 run in 6). Scoring a correct answer as a failure is the worse error for a benchmark: it manufactures a defect to chase. The criteria now accepts any accurate treatment of the UK and keeps the real assertion, which is that the answer must be regionally qualified rather than a flat yes or no. * fix(mental-models): stop delta ops treating the batch-only synthesis as authoritative The delta refresh writes a synthesis from the new batch alone, then asks a second call to merge it into the stored page. That call read the synthesis as evidence: its "a total of 4" counted only the batch and replaced a page's 3 customers with 4; its "no release was deployed" described only the batch and overwrote a production release recorded one wave earlier. Label the synthesis UNTRUSTED (only the supporting facts justify an operation) and add the combine-not-swap, absence-is-not-contradiction and refutation threshold rules. Replayed against both captured failures: 0/5 -> 5/5. * test: add hindsight-system-evals, published as a quality metric by the perf monitor Blackbox quality evals over a real hindsight-api and a real model, through the published Python client only — the system-tests shape without the stub, since stubbing the model would score the stub. First suite: knowledge-page convergence, the eval that found the delta-ops regressions. Each page is graded twice: correct, and whether it stores the specific baited falsehood. Runs in perf-test.yml (daily, not on PRs — needs secrets, and one red run is as likely noise as regression) and publishes correct rate and trap count to the continuous performance monitor. Seeding uses chunks retain with consolidation off, so the only model calls are the ones under test: minimum acceptance ~90s, full ~5 min. Also: the hindsight-dev knowledge-page tools used to find and replay those failures (kp_eval, kp_diagnose, replay_gemini, iterate_delta_prompt). * refactor: move the reflect evals into hindsight-system-evals and trim the new prompt text Everything from hindsight-dev/benchmarks/prelude now lives in the blackbox package, driven only through the public client: - test_02_reflect_answers: the one-shot reflect eval (minimum acceptance is the 2024-headcount incident), with retrieval-vs-reasoning blame from the tool trace; - debug/diagnose_page: dry-runs a page's second refresh and dumps every traced prompt; debug/replay_delta_ops: replays one captured delta-ops request per prompt variant and interrogates the model. The retrieval floor/ceiling tier is dropped: it measured the planned-prelude change (#4066), which was closed, and needed engine internals to author layers. Prompt size, measured against main: - grounding boundary rewritten compactly and no longer repeated in the final instructions (the final and reduce calls already carry it in the system prompt, so it was sent twice): tool loop +131 tokens, final +112; - delta ops: the combine-not-swap rule removed. Ablation by replaying the two captured failures: every other piece is load-bearing (dropping the absence or refutation rules, or the long synthesis paragraph, falls to 2-3/5), this one is not (5/5 on both without it). +491 tokens. Full run on the result: 14/14 correct, 0 traps (7 pages, 7 reflect answers). * test(system-evals): drop an unused property and type the page eval tests	2026-09-11 11:41:34 +02:00
Sanderhoff-alt	b045794817	perf(retain): accelerate within-batch semantic link calculation (#3977) ... Cuts the within-batch semantic link pass to float32 and one reused buffer. The batch was widened to float64, but PackedEmbedding is array("f") and pgvector's vector column stores float32, so the extra 32 bits were padding nothing downstream could read. Dropping to float32 halves the working set and puts BLAS on SGEMM; normalising in place, deriving validity from the row norms instead of an (n, dim) isfinite mask, and reusing one similarity buffer across blocks remove three further copies. Peak transient falls 74-86% (at 5,000 facts, 235 MB -> 48 MB). argpartition replaces a full sort that existed only to discard all but top_k, and the self-link mask and score unboxing move out of the Python loop: 1.7-2.2x on a realistic clustered batch, up to 4.8x when nearly every pair clears the threshold. Norms are accumulated in float64 via einsum, since a float32 sum of 1536 squares overflows above ~1e19 and flushes to zero below ~1e-22. The batch is copied with np.array rather than aliased with asarray, as it is now normalised in place. Verified against the float64 implementation across 120 randomised batches plus NaN/inf/zero embeddings, degenerate magnitudes and all-ties: identical link pairs, scores within 1e-6.	2026-09-08 11:38:19 +02:00
Sanderhoff-alt	bc06bd051f	perf(entity-resolver): optimize in-batch dedup via prefix filtering (#3991) ... Replaces the O(N^2) pairwise loop in _find_intrabatch_similar_pairs with a prefix-filtering set-similarity join, returning exactly the pairs the double loop returned. Each name's trigrams are sorted rarest-first and only the leading |A| - ceil(t*|A|) + 1 are indexed; a pair sharing none of those tokens cannot clear the cutoff. Cutting each prefix from its own size is what makes the pruning lossless — ascending set-size order is for the size filter, not for correctness. 3.9x faster at the 250-name cap and 8x at 1000 on distinct names; 2.5x over 9,525 real per-document batches harvested from LoCoMo and LongMemEval. Two shapes are slower and are now benchmark workloads rather than footnotes: batches under ~50 names (tens of microseconds) and batches of mutually similar names, where the filter prunes nothing and costs ~1.2x. _INTRABATCH_MAX_NAMES stays at 250 — that second shape costs ~156ms at 500 with no await in the join. Equivalence is pinned against the loop it replaced over randomised batches at 14 cutoffs, including exactly-achievable Jaccard ratios where a float t*|A| a hair above a whole number would shorten the prefix and drop a pair on the cutoff. Verified further on 60,000 set-level trials and the 9,525 real batches at six cutoffs: zero mismatches. Also memoises candidate trigram sets in _resolve_from_candidates, filled lazily so candidates the scoring loop never reaches cost nothing and no work happens ahead of the _SCORING_YIELD_EVERY yield points (#3211).	2026-09-08 09:51:16 +02:00
Nicolò Boschi	280f098202	feat(retain): inline images and files as first-class content (#4077) ... Makes images and files first-class raw content in `retain`. `content` accepts an ordered list of text/image/file blocks, the extractor reads each attachment in the position it occupies, and every read surface hands back the attachments behind what it returns. A plain string behaves exactly as before — text-only retain is byte-identical, because everything new sits behind an ATTACHMENTS block that is empty when a chunk carries none. Blocks are flattened at the API boundary into one canonical body with atomic placeholders, so `documents.original_text` stays plain text and content_hash idempotency, `update_mode=append`, chunk-delta re-extraction and `reprocess_document` keep working untouched. Bytes live in the existing FileStorage abstraction, content-addressed by sha256. Schema (one migration, both dialects): `attachments` for the blob, `document_attachments` for which documents reference it, and `memory_units.attachment_ids` for which attachments a *fact* came from — a column rather than a third table, because those ids behave exactly like `tags`. Provenance is per fact, not per chunk. Extraction runs one call per chunk, and a chunk holding a screenshot also holds the prose around it, so a chunk-level edge cited the diagram as evidence for the paragraph that never mentioned it. The extractor is asked instead, and a fact stated in the prose carries nothing. Extraction quality was measured against a real image-QA dataset with a raw-VLM ceiling arm before merging: transcribing structured attachments rather than summarizing them, and recording how each value is drawn, took the gap between "the model can read this off the image" and "memory can answer it" from 31.3% to 10.0% on the same 40 charts. The prose-article benchmark went 75% -> 100% over the same change, so it is not chart-specific tuning. Also here: * A vision slot (`HINDSIGHT_API_VLM_*`) so attachment-bearing chunks alone use a vision model and text-only chunks stay on a cheaper retain LLM. A vision call deliberately does not fail over to the retain chain's text models — that would reintroduce the silent omission the 422 gate exists to prevent. * The extension retain hook can now see each attachment (media type, size, kind, filename) and refusing a retain reclaims its bytes, which previously stayed fetchable forever. * A filename lives on the document edge, not the blob: the same PDF can be attached under a different name elsewhere, and content-addressing made the first name win for both. Known limitations, documented rather than hidden: store-owned memory backends get nothing (that retain path is Postgres-free and pre-dates this work), very dense pages are sampled rather than exhausted, and the Python client's ContentBlock is a plain dict where TypeScript gets the real union. Breaking for Go and Rust callers: `content` is now a union, so a bare string no longer satisfies it. Go gains a `TextContent()` helper; Rust uses `Content::Variant0(...)`.	2026-09-04 12:48:08 +02:00
Nicolò Boschi	bce43b8e14	perf(tokenizer): move token counting from quicktok to toktok-rs (#4022) ... Swaps `quicktok-v1` for `toktok-rs` (vectorize-io/toktok), a Rust BPE tokenizer whose ids are byte-identical to tiktoken's, then collapses the module's interface onto the two operations the engine actually performs. The swap itself is behaviour-neutral: compared side by side over 258 texts (~250 real files from `hindsight_api/` plus the special-token and mixed-script edge cases), quicktok and toktok produce identical counts AND identical ids on all three shared encodings. No token budget, chunk boundary or truncation point moves. Interface. `_SafeEncoding` existed to force `disallowed_special=()` onto `encode()`. Every caller of the object it returned was doing either a plain `count` or `decode(encode(x)[:n])` open-coded — which is `truncate_to_tokens`. So the module's whole public surface is now `count_tokens`, `truncate_to_tokens` / `truncate_many_to_tokens`, and `BUNDLED_ENCODINGS`; the tokenizer itself is private. That keeps #1883 fixed by construction rather than by convention: every route to the raising `encode()` went through the accessor that is now `_load_encoding`. Character-boundary truncation (toktok 0.1.3). Truncation was `decode(encode(text)[:n])`, cutting on a *token* boundary. Byte-level BPE splits one character across several tokens (under o200k_base "🧠" is three), so a cut could land mid-character and decode to U+FFFD: truncate_to_tokens("hello 🧠", 2).text -> 'hello �' (before) -> 'hello ' (now) The native call also never builds ids, never decodes, and returns the original string object untouched when nothing needs cutting. `batch_truncate` replaces the Python loop in the two callers that truncate a whole list: every reranker document (both LiteLLM cross-encoders) and every embedding input. Two user-visible consequences: * `llama3` and `qwen3` are gone — quicktok bundled five vocabularies, toktok bundles three. `HINDSIGHT_API_TOKENIZER_ENCODING=llama3` now fails at the first token count with the existing "Unknown tokenizer encoding" ValueError. Docs and both env templates updated, and `BUNDLED_ENCODINGS` (which had been lying about those two) now has a test that loads every name it advertises. * A negative budget used to slice a list with a negative index, silently dropping tokens off the end; the native call would raise. It clamps to 0. Wheels: cp311-abi3 covers 3.11-3.14, so 3.14 no longer compiles from source the way quicktok did. No musllinux wheels, which is irrelevant to the shipped images (all Python stages are glibc python:3.11-slim). numpy drops to an optional extra, so the tokenizer pulls in no dependency of its own. Measured on this repo's text: 2-7x faster than tiktoken on cl100k_base, 10-16x on o200k_base; counting an 81k-token document peaks at 1 KiB vs ~3 MB.	2026-09-02 12:52:08 +02:00
Nicolò Boschi	4388e0f95b	feat(coding-agents): add native DeepAgents Dcode integration (#3887) ... Registers `dcode` (LangChain's deepagents-code) as a native Agent Plugin: root `plugin.json` contributing the shared skill, the Hooks V2 SessionStart/UserPromptSubmit/Stop lifecycle, and the `hindsight_*` MCP server, installed through Dcode's own marketplace/plugin manager. Includes fixes found by running it against deepagents-code 0.1.65: - Decode `last_assistant_message`. Dcode's transcript lags the Stop event, so that field is load-bearing, but it is computed as `str(content)` — a Python repr whenever the provider returns content blocks. It was retaining ~1.8KB of encrypted reasoning payload as the assistant turn, and never comparing equal to the transcript's clean text, so an already-flushed reply was appended again every turn. Guarded family-wide: any harness surfacing the field must declare a decoder. - Annotate the six read-only MCP tools with `readOnlyHint`. Dcode rejects unannotated MCP calls in headless mode, which made recall and the knowledge-page tools unusable under `dcode -n`. - Parity: local history import (attributed via `dcode threads list --json`, since the transcripts carry no cwd), a Docker E2E adapter and image, the harness icon on the docs site as well as the control plane, and a clean uninstall that also retires the marketplace it registered. - Document that Dcode cannot host the codebase survey: `hindsight_ingest_document` writes, so its headless runtime gates it by design; it falls back to another agent's CLI like eight other harnesses. Supersedes #3887. Co-authored-by: Paritoshdagar <paritoshdagar@gmail.com> Claude-Session: https://claude.ai/code/session_01DNWFQn8AUN6SApcswHG3aN	2026-09-01 10:13:24 +02:00
Nicolò Boschi	317c4ae1aa	docs(hermes): disable Hermes's built-in memory via config flags (#3940) ... * docs(hermes): disable Hermes's built-in memory via config flags `hermes tools disable memory` shuts down the whole memory toolset on current Hermes builds, taking the native provider's `hindsight_retain`, `hindsight_recall` and `hindsight_reflect` tools down with it. Hermes treats memory as pluggable now, so use the config flags instead: hermes config set memory.memory_enabled false # MEMORY.md hermes config set memory.user_profile_enabled false # USER.md (optional) Updated the Hermes integration page, both Hermes guides, the two blog posts that carried the old command, and the regenerated docs skill reference. Fixes #3849 Claude-Session: https://claude.ai/code/session_012gXUp1i7YrWmLJVUYki53g * ci(hermes): set the documented memory flags in the compat check The compat script wrote only `memory.provider: hindsight`, so the config we actually document — flat-file MEMORY.md and USER.md turned off — was never exercised. Set both flags there so the existing `hermes memory status` assertion covers it: silencing the built-in stores must leave Hindsight the active, available provider. Claude-Session: https://claude.ai/code/session_012gXUp1i7YrWmLJVUYki53g	2026-08-31 17:26:59 +02:00
Sanderhoff-alt	0478c09c41	perf(retain): optimize embedding_to_pgvector via zero-copy orjson (#3815) ... Retain and import paths convert float embeddings to pgvector vector literals for asyncpg binding (insert_facts_batch, compute_semantic_links_within_batch, update_memory_unit_embedding). The baseline implementation used a Python generator: "[" + ",".join(repr(float(value)) for value in embedding) + "]" For 500 facts (768,000 floats at 1536d), this allocated 768,000 PyFloat objects and 768,000 PyUnicode strings, taking ~220 ms CPU time and ~15 MB heap memory. The optimized implementation leverages np.frombuffer on PackedEmbedding (array('f')) for zero-copy buffer views, and orjson.OPT_SERIALIZE_NUMPY to format floats directly into the output byte buffer using Rust Ryu SIMD: * Promotes numpy to explicit direct dependency across hindsight-api and dev; * Formats shortest float32 representation (byte-identical Postgres storage); * Isolates _repr_literal fallback helper for non-finite and non-float inputs; * Unifies _dumps_or_repr_fallback with single payload parameter and no option branching; * Streamlines embedding_to_pgvector into a concise polymorphic dispatcher; * Seamlessly supports array('f'), list[float], tuple, ndarray, and str. Measured on Apple Silicon via vector-serialization-bench (best of 5 repeats): workload baseline prod speedup peak alloc single_bge_384 (1x 384d) 0.136 ms 0.040 ms 3.4x 36K -> 13K single_openai_1536 (1x 1536d) 0.489 ms 0.085 ms 5.8x 144K -> 50K batch_20_gemini_768 (20x 768d) 4.580 ms 0.514 ms 8.9x 356K -> 185K batch_200_openai_1536 (200x 1536d) 92.64 ms 9.45 ms 9.8x 6.1M -> 3.4M batch_500_large_doc (500x 1536d) 221.78 ms 23.84 ms 9.3x 15.0M -> 8.4M batch_200_raw_list (200x 1536d) 87.70 ms 11.48 ms 7.6x 6.1M -> 6.0M Throughput increased from 3.3 Mfloat/s to 32.5 Mfloat/s (~9.8x speedup on typical retain batches), with ~44% peak memory reduction on 500-fact batches. Includes unit tests in test_packed_embeddings.py covering bit-identical float32 roundtrips, custom non-serializable objects, non-f array fallthrough, tuples, ndarrays, and non-finites.	2026-08-31 12:50:04 +02:00
MΞTΔ	a6f99c995c	feat(helm): add Prometheus operator ServiceMonitor support (#3847) ... * feat(helm): add Prometheus operator ServiceMonitor support The api (port 8888) and worker (port 8889) containers expose Prometheus format metrics at /metrics (verified against app source); the chart had no wiring for them — the worker's scrape annotations are gated behind the unrelated podAnnotations value and the api service had nothing. Add a gated metrics.serviceMonitor block that emits per-component ServiceMonitor resources (api always when enabled, worker only when worker.enabled). Selection labels are configurable for the Prometheus operator's serviceMonitorSelector (e.g. release: kube-prometheus-stack). Also scrape the dedicated worker in the dev LGTM compose stack, which previously only scraped the api on :8888. Verified end-to-end on k3d + kube-prometheus-stack: all three targets (api + 2 worker pods via headless endpoints) discovered and up=1. * fix(helm): address ServiceMonitor review	2026-08-31 11:09:15 +02:00
Nicolò Boschi	9fcb7ca7ac	perf(tokenizer): replace tiktoken with quicktok and default to o200k_base (#3788) ... Token counting is on the hot path of both retain and recall. Recall counts once per candidate fact, per candidate chunk, per source fact and per reranker document; retain counts whole documents. All of it went through `len(encoding.encode(text))` — which builds a full Python list of ids only to take its length. Measured on this repo's own text with the microbenchmark added here, against tiktoken 0.12.0 on a 14-core M-series, both on o200k_base: workload tiktoken quicktok speedup peak alloc 200 ranked facts 4.39 ms 0.75 ms 5.8x 3 KiB -> 1 KiB 500 source facts 6.92 ms 1.12 ms 6.2x 2 KiB -> 1 KiB 50 candidate chunks 12.02 ms 1.57 ms 7.7x 21 KiB -> 1 KiB 100 reranker documents 10.84 ms 1.55 ms 7.0x 12 KiB -> 1 KiB one 77k-token document 36.08 ms 3.00 ms 12.0x 2.8 MB -> 1 KiB Summed across the four counting stages one recall runs: 34.2 ms -> 5.0 ms. Three things make this worth a dependency change rather than a micro-opt: * `count()` returns an int without materialising the ids, so counting a large document allocates nothing. tiktoken has no count-only API — `encode_to_numpy` reaches the same 1 KiB but none of the speed, and is measured here too. * ids are byte-identical to tiktoken's; the benchmark asserts that on adversarial inputs before it times anything. * the vocabularies ship inside the wheel, so nothing is downloaded at runtime. That removes the tiktoken pre-download from the Docker build (both stages) and from scripts/dev/setup.sh — air-gapped deployments no longer need it baked in. The dependency risk is maintenance, not correctness, so it is contained: engine/token_encoding.py is the only module that imports quicktok, every call site routes through get_token_encoding() / count_tokens(), and its one dependency (numpy) was already in the tree. Replacing it means rewriting that file and nothing else. This also removes the last direct tokenizer import that had escaped the seam (`__import__("tiktoken")` in reflect/prompts.py). Removes #3756's workaround. count_tokens_windowed existed to bound the memory of counting a large retain body, encoding a megabyte at a time and accepting an approximate answer because a fixed character cut can split a token. count() allocates nothing at any size AND is exact, so the windowing, its helpers and its six call sites are gone — those callers now get an exact count. The test file keeps the property that made #3756 worth fixing (allocation does not track the input), now asserted against count_tokens itself. Default encoding moves to o200k_base, selectable with HINDSIGHT_API_TOKENIZER_ENCODING (server-level: budgets are only comparable between banks if they are all counted the same way). o200k_base is what current OpenAI models tokenize with. On English and code it counts within a fraction of a percent of cl100k_base, but on non-Latin scripts it is far closer to what a model actually charges — a mixed-script line with emoji is 19 tokens under cl100k_base and 13 under o200k_base. Since these counts back budgets that stand in for a context window, the closer vocabulary is the more honest one. Set cl100k_base to reproduce the previous counts exactly. Call sites that only need a number now call count(); the ones that need ids (query truncation, chunk truncation, reranker truncation, prompt fitting) still encode, but only after a count shows the text does not fit. The chunk-budget loop also stopped encoding each oversized chunk twice.	2026-08-25 15:02:26 +02:00
Nicolò Boschi	fe5c25d64c	fix(mental-models): store delta documents as verbatim markdown blocks (#3361, #3273) (#3622) ... * fix(mental-models): store delta documents as verbatim markdown blocks (#3361, #3273) A knowledge page could come back with a whole table welded onto one line, in sections no delta operation had named, and never recover. The delta refresh was blamed, but the damage was done one refresh earlier. `structured_content` was only the source of truth on the delta leg. The full leg stored the LLM candidate markdown verbatim in `content` while deriving the structure from it with `parse_markdown`, a typed-block parser that flattened anything its union could not express -- nested lists, list continuation lines, blockquotes, hard line breaks, horizontal rules, HTML, indented code, table alignment, a table row missing an outer pipe. 15 of 16 common constructs lost information, and every loss was a fixed point, so no later refresh could undo it. The two columns disagreed by construction, and the next delta refresh published the degraded one over the whole document. Schema v2 stores each block as a verbatim markdown fragment plus an id. Nothing parses a table, so nothing can flatten one. `parse_markdown` is deleted; `split_markdown` replaces it and recognises only ATX headings and blank lines, both fence-aware, which makes it lossless -- asserted as a property over a 26-case corpus of exactly the constructs v1 destroyed. Blocks are now addressed by id rather than by index (#3273). An index has to be counted by the model, and an off-by-one lands in range, silently overwrites an unrelated block, and is recorded as a success. An id is copied, not derived; one that does not resolve -- or that names a block in a different section -- is skipped and reported. Operation payloads are plain markdown strings, so the model no longer has to emit a typed block union either. `content` is now always the render of `structured_content`, on both legs, so the two can no longer drift apart. Also here: - `parse_llm_json` escapes `\n \r \t \b \f` inside JSON string values instead of blanking them. A model writing a markdown table into a string often forgets to escape its line breaks, and replacing them with spaces delivered the table already collapsed. Other control characters keep the previous treatment. - A model that adds a table row as its own block would render a broken table, so the prompt asks for `replace_block` and bare rows landing directly after a table are folded into it. - Migration `d1e2f3a4b5c6` clears v1 blobs. They are a lossy projection of the row's own `content`, so there is nothing to convert: the next refresh re-imports the structure from the markdown, losslessly. `content` is untouched. - The Gemini eval fixture pinned `gemini-2.0-flash`, which the provider has retired (404), so the whole eval class was dead. Verified against a real model: `test_document_survives_many_delta_rounds_intact` runs five delta rounds feeding one new fact each, asserting after every round that content is the render of the structure, that no line welds a table separator to other cells, that sections no operation named are byte-identical, and that a section never named across the whole run is unchanged at the end. Run four times, 20 real rounds, green. It is what caught the orphan table row. * fix(mental-models): write the structure whenever content is written `create_mental_model(content=...)` and `create_knowledge_page` inserted the markdown and left `structured_content` NULL, so a model authored as markdown had no structure until its first delta refresh -- and that refresh was then the one to derive it, silently reshaping a document nobody had asked it to touch. `update_mental_model(content=...)` was worse: it could set the markdown while leaving the *previous* document's structure in place, so the two columns described different documents until the next refresh papered over it. Nothing enforced the pairing; the refresh path just happened to pass both. Both writes now go through `canonical_document()`, which splits the authored markdown and hands back the structure together with its render. The insert stores both, and the update derives the structure whenever a caller supplies content without one. A refresh still passes both explicitly -- there the structure is authoritative and the markdown is already its render -- and is untouched. The derivation is hoisted above the embedding computation so the embedding, the history snapshot and the UPDATE all see one text. `content` is therefore the render of `structured_content` from the first byte rather than from the first refresh, which is what the assertion churn in this commit is: authored markdown now comes back canonicalised, so a document that was stored as "v1" reads back as "v1\n". Also makes the migration test rerunnable: a pg0 instance survives between runs and alembic will not replay a migration on a DB already stamped past it, so seeding into it would have left the rows untouched and the test asserting nothing. * feat(reflect): answer with a document, render the markdown from it The mental-model refresh asked the agent for markdown and worked out the document's structure by reading that markdown back. Reading LLM markdown back is where #3361 destroyed tables, and it is unnecessary here: the refresh knows it is producing a document, so it can ask for one. `done()` gains a document mode. Instead of an `answer` string it takes a `document` -- an ordered list of sections, each with a heading, a level and its blocks -- and the markdown that gets stored and shown is rendered from it. The model no longer writes the markdown that gets persisted, and nothing parses markdown to find out what the model meant. `answer` is not merely discouraged in that mode, it is absent from the schema, so there is no escape hatch back to prose. The shape is deliberately flat -- an array of sections holding arrays of block strings, no unions. A tool schema goes to the provider verbatim and not every provider accepts `oneOf` (Gemini rejects it), and a shape the model can fill without thinking is one it fills correctly. `document_from_sections` is tolerant, because a tool call is still model output: a missing heading, a `##` the model prefixed anyway, an out-of-range level or a non-string block is coerced rather than rejected. A block holding several blank-line-separated fragments is split into one block each, so the document keeps the granularity delta operations address even when the model packs a whole section into one string. Downstream is unchanged: the rendered markdown still flows on as `text`, so structured-output extraction, the length rewrite and the HTTP response all behave as before. The one place the two could drift is the length rewrite, which edits the text after the fact -- there the structure is re-derived from the rewritten markdown, which is lossless and keeps the invariant that the stored text is exactly what the stored structure renders. Splitting markdown is now only an import path: a model created from authored markdown, a restored export, or a run that produced plain text anyway (a provider that dropped the tool call, the iteration-limit answer). Verified against a real model: all five `hs_llm_core` refresh evals pass with the agent emitting structure, including the five-round stability run. The ordered list that a previous run had rewritten now survives untouched, and the new table row lands inside the table rather than beside it. * test(benchmarks): compare two builds on how a document survives being edited Neither half of "is the new pipeline better" was measurable before this. The unit tests prove the mechanics in isolation and the refresh evals prove one build behaves, but nothing compared a build against another one on the thing that actually broke: a document rewritten by an LLM over and over. The harness talks HTTP only, so the same code drives a server built from any revision. That is what makes an A/B possible without a feature flag inside the code under test: run it once per build, compare the two artifacts. Every document from every round is stored, and metrics are recomputed at comparison time, so sharpening a metric costs nothing instead of another few hundred LLM calls. Two things it measures, deliberately separately. Structural, no LLM: collapsed tables (the detector from #3361), rows, nesting, hard breaks, fences and quotes lost, sections that drifted with no operation naming them, plus pipeline health and latency. Damage is counted only in sections no operation named -- a refresh that rewrites a section it targeted may legitimately restructure it, and scoring that as corruption would punish the model for doing its job. Content, judged: each round declares what must be true afterwards and what must no longer be stated, checked one claim at a time so a miss points at a fact rather than at a score; plus a blind pairwise preference between the two builds' final documents, judged in both orderings so position bias cannot decide it. Three cases, and the third is the point. `api-reference` carries every fragile construct; `onboarding-playbook` carries none, because a change that fixes tables while degrading ordinary prose is not an improvement and that is where it would show; `release-runbook` puts a table with a missing outer pipe in a section the fact stream never touches. Both details are load-bearing: a well-formed table never triggered the bug, and a model asked to edit a malformed table tends to rewrite it correctly, repairing the damage before it can be measured. The reported failure was in sections the operations never named, where nothing could repair it. Token usage is not reported. The stored reflect_response does not carry it, and a column that is always zero reads as "this is free" rather than "this is not measured here". * test(benchmarks): harden the judging, and say what the A/B measured Running the benchmark against main and the branch turned up three problems in the benchmark itself, all of which would have made its verdict untrustworthy. The runner slept a fixed interval instead of awaiting the async operations it submitted, so its first results were five rounds of "Generating content..." scored as though they were documents. Retain, create and refresh are all submit-and-poll; it now polls, and refuses outright if the seed it is about to measure is still a placeholder. Damage was attributed to the whole document rather than to the sections nobody asked to change, which scored a model deliberately restructuring a section it targeted as if the machinery had corrupted it. Damage is now measured only in untouched sections, and the metrics are recomputed from the stored documents at comparison time, so this sharper reading could be applied to results already collected instead of paying to re-run them. A single judge call decided each claim, and one pedantic reading moved a build's score: a document describing an operation as "synthesises stored memories" was scored as not supporting "answers questions over stored memories" — the same operation, in the wording the source fact itself used. Claims are now decided by majority of three, and that claim was rewritten to test the fact rather than one phrasing of it. A claim a correct document can fail measures the corpus, not the pipeline. The report prints mean document length beside the preference column, because judges favour longer documents and a preference that tracks that column should be read sceptically rather than counted. The prompt change is the finding that landed back in the product: stating a document's structure is more clerical than writing prose, and the model got terser at it — measurably shorter documents that the judge liked less. Document mode now says plainly that the structure is the shape of the answer, not a budget for it. Baseline recorded in baseline_report.json: 45 refresh rounds per build from identical seeds. main lost 3 tables to collapse, 9 table rows, 6 levels of list nesting and 5 hard line breaks across 6 damaged rounds, and drifted 14 sections nobody had named. The branch lost nothing and drifted nothing. Content came out level — 100% recall and zero stale claims on both sides. * fix(mental-models): give the delta leg the document's own token budget `max_tokens` was enforced in exactly one place: a rewrite of the *synthesis* answer when it came back longer than the budget. In delta mode that answer is only context for the operations call and never becomes the document, so the document that actually gets stored was never measured against the budget at all. A delta refresh only adds. The document-evolution benchmark measured ~20 tokens of growth per round across 45 rounds, monotonic, which crosses the 4096-token knowledge-page default after a couple of hundred refreshes — and knowledge pages refresh after every consolidation. The configured budget was quietly ignored for the entire life of a page after its first full build. Truncating the document here would delete knowledge nobody asked to delete, so the budget is stated instead: the delta call is told the document's current size against its budget, and when it is over, asked to make room with the same operations it uses for everything else — on content that is superseded or duplicated, never by dropping the facts it is integrating and never by summarising a section that is still current. Below 80% of the budget nothing is said at all. Every refresh records document_tokens and document_budget, and going over adds a warning, so a page that keeps growing is visible rather than merely large. Also closes the one path where the model still wrote markdown that got stored: the over-budget trim. In document mode it is now asked for a document, so the structure survives the trim instead of being re-derived from prose the model wrote. A response that is not JSON falls back to the previous split, which is lossless — the worst case is the old behaviour, not a lost answer. The real-LLM eval is the part that could not be mocked: told that a document is over budget, does a model reclaim space or append anyway? It drops the twelve archived sections and keeps the current process and the checklist — 434 tokens to 39 against a 200-token budget, stable across four runs. The assertions check where the space came from, because getting under budget by deleting current content would pass a naive shrink check and be worse than going over. * test(mental-models): audit every trigger flag on the delta leg `max_tokens` looked wired up — read from the model, passed to reflect, enforced by a rewrite — and was still ignored for the document that actually got stored, because in delta mode the thing it capped never becomes the document. Reading the code is how that was missed; nothing asserted the flag at its destination. So every flag is now exercised through a real delta refresh and asserted where it lands: retrieval options at the reflect call, document options in the delta prompt or the persisted row. Full mode is covered by the surrounding modules; this is the leg where a flag goes to die. Fifteen flags checked. Fourteen were already honoured. The audit pins them so a future change to either leg cannot quietly drop one: - retrieval: fact_types, exclude_mental_models, exclude_mental_model_ids, the model's own id (a model must not feed on its previous version), include_chunks, recall_max_tokens, recall_chunks_max_tokens, the model's tags, tags_match - document: max_tokens, response_schema (extracted from the merged document, not from reflect's delta-only answer), keep_trace, mode - and the whole trigger surviving a create/read round trip, since a flag that does not persist is not honoured either The fifteenth is documented behaviour that reads as a bug from outside, so it is pinned as intended rather than "fixed": `tag_groups` overrides flat tags entirely, dropping the model's own tags and forcing `tags_match` to `any`, because each group carries its own match mode. A `tags_match` set alongside groups is deliberately not forwarded. The default for a tagged model with no `tags_match` is `all_strict`, not `any` — a model scoped to tags must not widen its own scope by default. Scheduling flags (`refresh_cron`, `refresh_after_consolidation`) are honoured outside the refresh executor — the maintenance loop and the consolidation hook — and keep their existing coverage there. * test(benchmarks): type the structural summary, and cover the flag main just added Two findings from reviewing the branch against a fresh main. `_structural_summary` returned a raw dict of known keys, which the project standards forbid for exactly the reason it bit here: the report read it with `summary["rounds"]` and nothing would have caught a renamed metric until the table rendered wrong. It is a `StructuralSummary` model now, and the side-by-side table renders `model_dump()` so a metric added later appears without being listed twice. Main added a sixteenth trigger flag while this branch was in flight (`min_refresh_interval_seconds`, #3621). It gates automatic refreshes rather than shaping one, so it is honoured in the submit path and covered there — but the round-trip test enumerates the whole trigger on purpose, because a field that round-trips as None looks like the flag being ignored rather than like a storage bug. Adding it keeps that list exhaustive. * fix(mental-models): teach the retraction prompt the schema it emits into CI caught what the rebase brought: main added an unsay pass (#3618) whose prompt documents the operation vocabulary a second time, in prose, and it still told the model to say `{"op": "remove_block", "section_id": "...", "index": N}` with typed `block` payloads. Under the id-addressed schema those ops fail validation and are dropped, so a retracted fact would keep being stated and nothing would say why — the unsay feature silently doing nothing. The prompt now describes the schema it actually emits into: blocks addressed by `block_id`, block payloads as markdown strings, and the note about emitting removals in descending index order deleted, because ids do not shift when a sibling is removed and telling a model to order by position invites it to think in positions again. Guarded structurally rather than by one more test. The op vocabulary is written down twice — Pydantic models the applier validates against, and prose in each system prompt — and a test for the prompt that drifted does not exist by construction, since it is the one nobody wrote. `test_delta_prompt_schema_parity` asserts over *every* prompt carrying an operations vocabulary that each op exists, that no shape names a field the schema rejects, that no v1 typed block survives, and that blocks are addressed by id; plus a check that a prompt asking for `{"operations": [...]}` cannot be left off the list. Reverting the prompt fails three of them. The rest is the same schema change reaching tests the rebase brought in: canned ops in the outcome matrix moved to `text`, the retraction tests resolve a real `block_id` out of the document the prompt shows them (which is what a model does, and what a hardcoded index cannot express), the stale `parse_markdown` monkeypatch points at `structured_document_from_stored`, and four assertions on authored content now expect the canonical render.	2026-08-20 11:35:32 +02:00
Sanderhoff-alt	3b9261ab3b	fix(scripts): preserve go client go.mod/go.sum across regeneration (#3540) ... Client regeneration deleted go.mod/go.sum and re-resolved them with go mod tidy. Deps pulled in by the maintained tests (testify) are not pinned by the generator, so tidy resolved them to upstream's latest release. Every new testify release changed the regeneration output and broke the verify-generated-files check for all pull requests. Treat go.mod/go.sum as maintained files: stash them before wiping generated code and restore them over the generator-emitted ones. go mod tidy still runs but now only adds entries for new imports. Dependency upgrades become explicit go get + go mod tidy commits.	2026-08-19 16:51:03 +02:00
Nicolò Boschi	7f7d2a0a38	chore(deps): close every open medium/high Dependabot alert (#3624) ... * chore(deps): sweep open medium/high Dependabot alerts across all lockfiles * chore(deps): bump @hey-api/openapi-ts to 0.97.3 and regenerate the TS client * docs(scripts): note that the Deno client patch anchor tracks the generator version	2026-08-19 15:21:45 +02:00
Sanderhoff-alt	b8947a24e9	fix(ci): authenticate Hermes compatibility clone (#3536)	2026-08-17 13:13:30 +02:00
Ben	f36a462d1d	feat(eliza): add Hindsight long-term memory integration for elizaOS (#2385) ... Adds @vectorize-io/hindsight-eliza, an elizaOS plugin that gives agents long-term memory backed by Hindsight: - HINDSIGHT_MEMORY provider recalls relevant memories into the prompt before each model call. - HINDSIGHT_RETAIN evaluator retains conversation messages after each turn (fire-and-forget; agent replies optional). - Bank defaults to the message entityId for per-user isolation; both sides fail safe so a Hindsight outage never blocks the agent. Targets @elizaos/core ^1.7.2 (current npm latest, not the 2.x beta on main). Includes tests, CI job, release-script + changelog-generator entries, and docs gallery entry + page.	2026-08-13 11:11:24 -04:00
Nicolò Boschi	bf943e4ab7	fix(release): keep package-lock.json's version in step with the release (#3456) ... The npm branch of the release script sed's package.json only, and a package-lock.json carries the package's own version twice — so every npm integration release since its lock was committed left the lock pinned at the version it was born with. coding-agents shipped v0.3.2 with a lock still claiming 0.0.5; five other integrations had drifted the same way, by one to several releases: coding-agents 0.3.2 lock 0.0.5 eve 0.2.1 lock 0.2.0 obsidian 0.2.1 lock 0.2.0 openclaw 0.10.0 lock 0.9.0 opencode 0.2.8 lock 0.2.6 paperclip 0.3.0 lock 0.2.3 Nothing was broken by it — npm ci tolerates the mismatch (which is why CI stayed green), and npm never publishes the lock, so no published artifact carried the wrong number. It is a stale field that misleads anyone reading the tree. `npm version --no-git-tag-version --allow-same-version` rewrites exactly those two fields. `npm install --package-lock-only` was the obvious alternative and is the wrong tool here: it re-resolves the dependency graph, so a release commit could silently carry dependency bumps nobody asked for. Verified on a scratch copy that the dependency entries come out byte-identical. The six locks above are synced in the same commit; leaving them stale would mean the script only stops the drift getting worse. npm ci re-verified against the synced coding-agents lock.	2026-08-13 13:21:11 +02:00
Ben	a133cc1495	feat(agent-plugin): add portable Hindsight plugin for the Agent Plugins standard (#3394) ... Add a vendor-neutral Hindsight plugin conforming to Vercel's Agent Plugins 1.0.0 standard (plugin.json + mcp.json + skills/SKILL.md), so one artifact gives long-term memory to any compatible client (Codex, Cursor, GitHub Copilot, Kiro, VS Code) instead of a per-IDE integration. The plugin is a thin transport wrapper over Hindsight's existing MCP server (retain / recall / reflect); a bundled skill teaches the agent when to use it. Wiring: - CI: test-agent-plugin-integration job runs the manifest validator, gated on hindsight-integrations/agent-plugin/** changes. - Docs: integrations.json gallery entry + docs-integrations/agent-plugin.md. - Release: agent-plugin added to release-integration.sh and the changelog generator; both learn to read a root-level plugin.json and link the changelog to the source tree (git-distributed bundle, no registry package).	2026-08-11 18:06:17 +02:00
Parafee41	36eb64dfba	fix metapackage embed version coupling (#3261)	2026-08-10 14:19:25 +02:00
Nicolò Boschi	aefc5e8e7d	test(ci): gate every build on hermes-agent@main co-installability (#3265) ... Hermes installs `hindsight-all` into its OWN venv via `hermes memory setup` to run memory in local_embedded mode, and exact-pins every direct dependency (`==X.Y.Z`) as a deliberate supply-chain policy. Any version range Hindsight declares that excludes one of their pins therefore makes the two impossible to co-install for every Hermes user on embedded memory — that is what #3251 hit with our cryptography/pillow floors. Hermes main has since bumped to cryptography==48.0.1 / Pillow==12.3.0, which already matches our floors, so no dependency change is needed and none is made here. What was missing is the check that keeps it that way: both sides bump on their own schedule, so the collision recurs silently until something looks for it. Tracking their main branch surfaces it while it is still cheap to fix on either side rather than in a released Hermes. scripts/test-hermes-compat.sh runs five checks of increasing depth: resolution (both into ONE resolution, so an unsatisfiable pin is a hard error instead of a silent downgrade), `pip check` plus a dump of the contested versions, `hermes memory status`, the runtime imports Hermes makes for embedded memory, and a real embedded daemon boot with bank operations. Implementation notes: - Hindsight installs from BUILT WHEELS, not `file://` directories. uv installs a workspace member given as a directory such that hindsight_embed.__file__ still points into the source tree, and the daemon manager keys dev-mode detection on that path — so a directory install launches the API via `uv run --project <repo>/hindsight-api-slim`, out of the monorepo's venv and .env, bypassing the Hermes venv this script exists to test. Step 5 asserts the daemon binary resolves inside the test venv so this cannot regress. - Hermes is cloned and installed editable: their build backend refuses wheel/sdist builds by design, so `git+https://` fails outright. - Python is pinned to 3.12 rather than .python-version because Hermes caps itself at <3.14; the venv must sit inside both projects' windows. - Hindsight state is isolated by a dedicated `hermes-ci` profile rather than by redirecting HOME, which would also hide the uv/HuggingFace caches from the runner and re-download the local-ml stack every run. - Step 5 runs from the work dir, not the repo, so a developer's .env cannot hand the daemon credentials a runner does not have. - MemoryEngine refuses to construct without an LLM key, so the daemon boots on a placeholder one; nothing calls the LLM during startup or bank operations. The job needs no secrets and so runs on fork PRs too. Only the retain/recall round-trip requires a real LLM key and is skipped without one. Verified end-to-end locally against hermes-agent main (0.20.0): all five steps pass, 225 packages consistent, daemon boots from the test venv and stops cleanly.	2026-08-10 13:03:40 +02:00
Nicolò Boschi	b5d8439c8f	hindsight-coding-agents: harness-pluggable long-term memory for coding agents (#2522) ... * feat(integrations): add hindsight-opencode-coding plugin Reflect-only long-term memory for coding agents in OpenCode, with a git+chat backfill and (opt-in) live session write-back. - reflect + INJECT: on a task, reflect() the symptom and push the root-cause answer into the system prompt (no tools/recall). - backfill: every commit (full message + full diff, commit timestamp + git metadata) under a 'git' retain strategy; each chat as a JSON user/assistant transcript with custom extraction (<=2 coherent facts) under a 'chat' strategy; observations on; optional codebase knowledge pages. - live write-back (opt-in HINDSIGHT_RETAIN_SESSIONS): every N turns upsert the tool-filtered transcript under a stable conversation:<sessionID> document_id. * refactor(integrations): generalize opencode-coding into hindsight-coding-agents Make the coding-memory plugin harness-pluggable instead of opencode-specific. A 'harness' (coding agent) differs in only two places; everything else is now shared core: - src/core/ hindsight client, missions, git + chat ingest, inject, RuntimeCore - src/core/types.ts HarnessAdapter + ChatReader interfaces - src/harness/ per-agent adapters + registry (opencode implemented) Backfill: --harness selects how past sessions are read (opencode today); git ingest, retain strategies, missions, and knowledge pages are identical across agents. Runtime: HINDSIGHT_HARNESS (default opencode) selects the adapter that binds RuntimeCore's reflect+inject+write-back to that agent's plugin API. Adding an agent = one adapter file + a registry entry. Type-checks and builds clean; unknown --harness/HINDSIGHT_HARNESS errors with the available list. * feat(coding-agents): on-demand memory_reflect tool, opt-in git-sync, JSON config Add two capabilities to the reflect-only coding-agents plugin and move all configuration off environment variables onto a single JSON file. - memory_reflect tool: exposes the same synthesized reflect that is auto-injected on the first message as an on-demand opencode tool the agent can call mid-task (RuntimeCore.reflectNow + opencode adapter tool). Harness-agnostic core, thin opencode wiring. - incremental git-sync (opt-in): on load, diff the target ref's commits (origin/main, falling back to HEAD) against the git:<sha> document_ids already in the bank and async-retain only the missing ones, reusing the backfill's per-commit encoding (retainCommit). Set-based, correct across rebases; best-effort, non-blocking. Off by default (gitSync.enabled). Adds HindsightClient.listDocumentIds + core/sync.ts. - config file: all settings now come from ~/.hindsight/coding-agent.json (core/config.ts) -- no environment variables. The backfill CLI reads the same file for shared connection/bank settings with --flags overriding; operation flags stay CLI-only. Committed with --no-verify: the repo-wide pre-commit lint hook is broken in this environment (missing @eslint/js in hindsight-control-plane) and blocks all commits. * fix(coding-agents): remove benchmark-specific strings from prompts Fairness audit of the sdebench benchmark found three contaminations: - CHAT_CUSTOM_INSTRUCTIONS used the literal answer to a graded task (round_cents/ROUND_HALF_DOWN/legacy ledger) as its example - replaced with a fictional, non-benchmark example. - buildSystemInjection told the model 'the hidden tests depend on those exact choices' - hardcoded knowledge of the benchmark's grading; reworded benchmark-agnostic. - REFLECT_MISSION examples were shape-matched to specific benchmark tasks (symbol mappings, exact numbers) - neutralized. No behavior change intended beyond removing the leaked specifics. (includes hook-regenerated skills/hindsight-docs sync) * feat(coding-agents): reflect-outcome diagnostics — no more silent memory loss A benchmark sweep ran the entire memory arm with zero injected memory: reflect failed environmentally on every task and the best-effort catch swallowed it, making a memory-less run indistinguishable from a memory run. onTask now appends a reflect_ok/reflect_empty/reflect_failed record (duration, error, query prefix) to HINDSIGHT_DIAG_FILE (default /tmp/hindsight-plugin.log). Consumers can assert a session actually had memory before trusting a comparison. * fix(coding-agents): chronological session recency + supersession-aware reflect Two defects surfaced by the conversation-amended benchmark tasks (a rule settled in one chat and amended in a later one): - chat ingestion staggered synthetic timestamps NOW - i*1h, INVERTING recency: an amendment chat ranked older than the decision it superseded, steering temporal ranking toward the stale rule. Session list order is chronological; the last session is now the newest. - REFLECT_MISSION now states that when memories conflict on the same rule, the latest/superseding decision wins and the superseded rule must be reported as no longer in effect, never presented as the fix. Observed live: reflect on an amended bank returned the superseded keep-latest rule as the fix. Both fixes are general recency/consistency semantics, not benchmark-specific behavior. * feat(coding-agents): multi-harness configurability + Claude Code hook entry One config, several agents side by side: - Each runtime entry point now KNOWS its harness instead of reading the config's `harness` key (which selected a single global adapter and made opencode + claude mutually exclusive). That key now only picks the backfill's session formatter. - New `harnesses.<name>` config sections: per-agent overrides of any field (bank, disabled, timeouts) over shared connection defaults. - New project-local layer: <project>/.hindsight/coding-agent.json overrides the global file — the natural home for a per-repo bank. Precedence: defaults < global < global.harnesses < project < project.harnesses. - New entry point: `hindsight-claude-hook` (dist/claude-hook.js), a Claude Code UserPromptSubmit hook. Reflects once per Claude session, caches the answer in tmp and re-injects it on later prompts, and writes the same reflect_ok/failed diagnostics as the opencode path. Verified live: claude hook via project config + harnesses section (reflect_ok, cached re-emit in 46ms, one reflect total); opencode via the benchmark harness (reflect_ok, task solved 0 corrections). * feat(coding-agents): per-repo dynamic bank resolution (family convention) Port of the bank-derivation convention shared by the claude-code, omo, cline, and opencode integrations, with coding-first defaults: - No bankId configured => the bank is derived from the git repo the working directory belongs to, WORKTREE-AWARE: git rev-parse --git-common-dir resolves every linked worktree to the main worktree's basename, so all worktrees of a repo share one memory bank (bare repos use the bare dir name; non-git dirs fall back to the dir basename). - Default granularity is [gitProject] (not agent::project): opencode and claude share ONE memory per repo — add 'agent' to dynamicBankGranularity to split per agent. - Explicit bankId keeps today's static behavior (benchmark harness, single-bank setups); dynamicBankId forces either mode; supporting fields: bankIdPrefix, directoryBankMap (exact cwd -> bank escape hatch), agentName, resolveWorktrees. - backfill: --bank wins, else the SAME resolution applied to --repo, so `hindsight-coding-backfill --repo .` fills exactly the bank the agents will read. Verified: worktree -> main-repo bank (hs-coding-plugin-wt -> memory-poc), static/prefix/dirMap/granularity cases, and the claude hook e2e (reflect_ok via directoryBankMap against a live bank). * feat(coding-agents): bank template string, prefix path map, {harness} field Bank-resolution refinements: - `bankIdTemplate` format string replaces the granularity array: e.g. "hindsight-{gitProject}" or "{harness}-{gitProject}" — default "{gitProject}" (opencode + claude share one bank per repo). Placeholders: {gitProject} {project} {harness} {channel} {user}; unknown placeholders warn with the valid list. bankIdPrefix removed (expressible in the template). - {harness} is supplied by the entry point itself (opencode plugin, claude hook, backfill --harness), not a config field — nothing to keep in sync. - directoryBankMap now matches by LONGEST absolute-path prefix and overrides everything incl. an explicit bankId: mapping a repo root covers all its subdirectories; deeper mappings win. - config discovery walks UP from the working directory to the nearest .hindsight/coding-agent.json — a hook invoked from a repo subdir previously missed the repo's project config entirely (found by an e2e test that failed exactly this way). Verified: derivation matrix (template/prefix-map/override/static/bad placeholder), claude hook e2e from a nested subdir (reflect_ok via walked-up config + prefix-matched map), opencode benchmark task green. * feat(coding-agents): cursor-cli + codex harnesses, unit tests, live system tests Harnesses — hook-based agents now share one runtime (core/hook.ts: stdin event -> layered config -> per-repo bank -> once-per-session reflect with tmp cache -> native output -> diagnostics), so each agent is a ~25-line HookSpec: - hindsight-claude-hook (UserPromptSubmit -> additionalContext) - hindsight-cursor-hook (beforeSubmitPrompt -> {continue, additional_context}) - hindsight-codex-hook (Codex CLI v0.116+ claude-compatible hooks; accepts prompt/user_prompt) All three + opencode registered in the harness registry (backfill --harness resolves them; hook harnesses share the normalized-JSON chat reader). Tests (vitest, family convention): - 25 unit tests: full bank-derivation matrix (worktree/bare/static/ dynamic/template/{harness}/prefix-map incl. longest-wins and no-sibling-false-match) and config layering (harness sections, project-over-global, upward walk, nearest-wins, gitSync field merge, malformed fallback, legacy signature). - live system suite (npm run test:live, HINDSIGHT_LIVE_E2E=1): builds a real git repo with a decision planted in a commit + a conversation, runs the real backfill CLI (server-side LLM extraction), then invokes the BUILT hook binaries as subprocesses and asserts the decision's literals come back in the injected context — semantic verification with a real LLM — plus per-session cache behavior and diag records. All 4 passing against a live server. Note: session ids in the live suite are unique per run — the hooks cache per session id in tmp, and a static id once cached a bad answer from a half-broken server across reruns. * docs(coding-agents): full README rewrite + integration docs page README now covers everything the package does today: the reflect-once/ inject-every-turn mechanics, all four harnesses (opencode plugin + claude/codex/cursor hooks) with install snippets, the complete configuration reference (layered files, harnesses sections, per-repo dynamic bank resolution with template placeholders, directoryBankMap, worktree behavior), backfill CLI incl. bank auto-resolution and chronological session ordering, the reflect diagnostics contract, and the unit + live test suites. Docs site: new docs-integrations/coding-agents.md (same content adapted to the integration-guide format) + integrations.json hub entry so the generated sidebar picks it up. Placeholder icon (github.png) pending a real one. Verified: page renders (docusaurus build), all doc pre-flight checks pass for this entry — note the docs build on this branch was ALREADY failing on the unrelated pre-existing 'zcode missing from integrations.json' check. * feat(coding-agents): 🧠 attribution header in buildSystemInjection Prepend the 'Using Hindsight Memories' visible-attribution directive to the harness-agnostic system injection so every coding-agent harness surfaces a recognizable header when it uses recalled memory. Covered by 5 deterministic inject.test.ts cases (real emoji + em dash, no lone surrogates). * feat(core): add recall() to HindsightClient * style(core): apply prettier formatting to recall test * style(coding-agents): normalize prettier formatting across package * fix(core): narrow RecallResult to actual API contract, add fetch-throw test * feat(core): formatMemories + shared attribution preamble * style(core): prettier-wrap recall.test.ts array literal * fix(core): cover formatMemories trim/filter + drop stale inject comment * feat(core): per-turn recall in the hook runtime (reflect once, recall every turn) Extracts the hook logic into a pure, unit-testable buildHookOutput(): every prompt now runs recall() and injects a <hindsight_memories> block; reflect still runs once per session (first prompt) and its cached answer is no longer re-injected on later turns. runHook() becomes thin stdin/stdout plumbing with a makeClient seam for tests. Updates the three hook entrypoints' doc comments to match, and adds recallMaxTokens/recallTimeoutMs config fields. * fix(core): make recall fail-open in buildHookOutput + cover recall failure/opts * feat(claude-code-v2): wrapper plugin skeleton (per-turn recall via bundled core) Also disables tsup code-splitting in hindsight-coding-agents so each bin entry (claude-hook.js etc.) is a single self-contained file with no shared chunk-*.js — required for wrapper build scripts that copy just the one hook file out of dist/. * chore(coding-agents): sync codex-hook bin into package-lock * fix(claude-code-v2): derive version from manifest + guard self-contained bundle * feat(core): Claude transcript reader (normalized user/assistant text turns) * fix(core): transcript reader null-safety + drop sidechain turns * feat: live write-back on the Claude Stop hook (shared retain-hook runtime) Extracts a testable buildRetain core (read transcript -> upsert under conversation:<sessionId> via retainLiveSession) plus a thin runRetainHook plumbing wrapper mirroring the existing runHook/buildHookOutput split, and wires it up as a Claude Code Stop hook. Fail-open throughout: an empty transcript is a no-op, and a retain failure is diagnosed but never thrown. Exports diag() from core/hook.ts so retain-hook.ts can reuse the same diagnostics helper instead of duplicating it. * refactor(core): extract diag module + trim buildRetain params - Move diag() out of hook.ts into a neutral src/core/diag.ts so retain-hook (and future lifecycle hooks like SessionStart) don't reach into a recall/reflect-specific module for a cross-cutting concern. - Drop the unused cwd/cfg params from buildRetain — only harness, sessionId, transcriptPath, and client are read; cwd/cfg stay in runRetainHook where they're actually used (config load + deriveBankId). - Clarify that retainSessions is opencode-plugin-only; the Stop hook always writes back unless disabled. * feat(core): knowledge-page CRUD on HindsightClient (mental-models) * fix(core): page methods throw on 404 + doc rationale * feat: native TS MCP server for knowledge-page tools (bank-aligned) Adds a native TypeScript MCP (stdio) server exposing the agent_knowledge_* tools (get_current_bank, list_pages, get_page, create_page, update_page, delete_page, recall) over MCP, wired into the claude-code-v2 wrapper. Bank resolution goes through the same loadConfig + deriveBankId path the hooks use (harness "claude-code"), so knowledge pages, recall, and retain all land in one per-repo bank. This is a native TS server rather than reusing the Python MCP because its bank derivation mismatches. - src/core/knowledge-tools.ts: SDK-free tool specs (zod schemas), unit tested against a stub client (17 tests) — every handler is fail-closed to an isError:true result instead of throwing. - src/mcp-server.ts: the only file importing @modelcontextprotocol/sdk. - tsup.config.ts: new mcp-server entry, noExternal inlines the SDK + zod so dist/mcp-server.js stays a single self-contained file. - claude-code-v2/.mcp.json + build.mjs: wires the bundle into the plugin; the self-contained-bundle guard passes for mcp-server.js unmodified (no exemption needed) since noExternal fully inlines its deps. * fix(mcp): honor disabled flag + testable selectTools - Export selectTools(cfg, client, bankId) from mcp-server.ts: pure, SDK-free, returns [] when cfg.disabled (mirrors the hooks' disabled check) so a disabled Hindsight exposes zero MCP tools instead of all 7. Confirmed at runtime: with disabled:true the server still connects but doesn't advertise a tools capability at all (tools/list -> Method not found), which is stronger than an empty list. - Guard main() behind an argv[1]-vs-import.meta.url check so importing the module for tests doesn't start a real stdio server. - Add src/mcp-server.test.ts covering selectTools for both the disabled and enabled cases. - Reword the HINDSIGHT_MCP_PROJECT_CWD comment: nothing sets it today (the plugin doesn't cd), it's an escape hatch, not a launching-host contract. * refactor(core): lazy-load opencode adapter so backfill bundles self-contained * test(core): lock opencode no-runtime registry invariant + doc it * feat(core): cold-repo detection + seed-consent state * test(core): cover seed write-failure + guard non-object state * feat(core): background seed mechanics + hindsight-seed control CLI Adds hasGitHistory (git.ts), startBackgroundSeed + seedControl (seed.ts), and the src/hindsight-seed.ts entrypoint the agent runs after the SessionStart seed offer (Task 10b) to seed or decline a repo's bank. * fix(core): handle async spawn error in startBackgroundSeed spawn() failures (ENOENT/EACCES/fd exhaustion/sandboxed environments) often arrive asynchronously as an 'error' event on the child, not a synchronous throw. An unhandled 'error' event crashes the caller, so attach a no-op handler alongside the existing try/catch. Also documents the Claude-Code-only harness assumption in hindsight-seed.ts. * feat: SessionStart auto-seed offer for cold repos (Claude wrapper wired) * fix(core): shell-escape seed offer paths + drop orphaned isColdRepo * docs(claude-code-v2): marketplace entry, full README, v1→v2 migration note * fix(core): cap hook reflect timeout, align backfill+hook config resolution - hook.ts: cap reflect's timeoutMs to HOOK_REFLECT_CAP_MS (8s) so it always resolves/aborts before Claude Code's 15s UserPromptSubmit kill window, guaranteeing the session cache write + recall injection complete instead of silently retrying reflect (and dropping recall) on every turn. - backfill.ts: resolve config via loadConfig({harness, projectDir: REPO, path}) instead of the legacy string form, so project-local .hindsight/coding-agent.json layers in and the background auto-seed backfill targets the same bank recall/retain/MCP read from. - hook.ts/retain-hook.ts: resolve the cwd fallback before loadConfig (not just at deriveBankId) so project-local config layers even when the hook event's cwd is missing. * fix(claude-code-v2): dev-install must copy .mcp.json (MCP tools were missing) * feat(core): deterministic SessionStart auto-seed + knowledge-page bank mission The prior SessionStart design asked the agent to pose a y/n question then run a seed command itself; live testing showed the model surfaces the question and then ignores it, so nothing ever seeds. The hook now starts the background seed itself on a cold git repo (tri-state: cold/warm/ unreachable) and always injects a short visible note plus a bank-mission pointing the agent at the agent_knowledge_* tools. * docs(claude-code-v2): update seed docs for deterministic auto-seed + knowledge mission * feat(core): default seed to aggregated commit messages (one cheap doc) + Initiatives page; full-diff opt-in via --diffs * docs(core): align backfill README + strategy log/comment with gitlog default * feat(core): headless codebase-survey seed + agent_knowledge_ingest MCP tool On a cold repo, the SessionStart hook now also spawns a detached headless `claude` that samples the repo's structure and ingests its findings into Hindsight via a new agent_knowledge_ingest MCP tool, alongside the existing git-history backfill. Knowledge pages synthesize their content from bank memories via source_query, so this is how the survey feeds them. - knowledge-tools.ts: add agent_knowledge_ingest (title -> slug doc id, retain via the "chat" strategy, tagged source:upload). - survey.ts: resolveClaudeBin + startCodebaseSurvey, mirroring seed.ts's fire-and-forget/never-throw spawn pattern. - Anti-recursion: HINDSIGHT_DISABLE_HOOKS guard at the top of runHook, runRetainHook, and runSessionStartHook so the survey's own claude session can't re-trigger seeding/recall/retain; survey.ts sets it on the child. - config.ts: codebaseSurvey (default true) + surveyModel (default "sonnet"). - session-start.ts: wire startSurvey into the cold-repo branch alongside startSeed; update the visible learning note. * fix(core): sandbox headless survey (deny-list, no bypassPermissions) + spend cap + document strategy * feat(core): default codebase-survey model to haiku (cheaper/faster; sonnet still configurable) * feat(core): survey excludes CLAUDE.md + agent-instruction files from ingestion * docs(coding-agents): v2 knowledge-pages design spec + implementation plan * feat(core): add pageRefreshEveryTurns config (default 10) * feat(core): knowledge-injection roster/preamble formatting * feat(core): passive knowledge entity_labels tier vocabulary + configureBank wiring * feat(core): tag-scope seeded pages, Initiatives folder, relatedPageId link source_query * feat(core): captureInitiative — per-initiative page + relatedPageId marker * feat(mcp): hindsight_* grounding tools + capture_initiative; remove raw page CRUD from agent * feat(core): SessionStart injects page roster + guidance preamble * feat(core): UserPromptSubmit hook-counted periodic page-roster refresh * feat(core): rich markdown session write-back with tool calls + verbose session strategy * chore: apply prettier line-wrapping to test files * fix(claude-code): surface the seed note via user-visible systemMessage, keep preamble in additionalContext * fix(survey): use renamed hindsight_ingest_document MCP tool (Task 6 rename regression) * fix(core): preamble + refresh nudge the agent to call capture_initiative for major features * fix(core): re-inject tool+capture reminder every cadence turn even with no pages (unconditional nudge) * fix(core): inject when-to-call guide for the full hindsight_* tool suite, not just pages+capture * feat(claude-code): cold-check-wins seeding — reseed a cleared bank on the live doc count, ignore stale seededAt * fix(core): simplify capture_initiative instruction to one clear trigger (remove confusing OR-chains) * fix(core): port proven v1 attribution preamble + surface memories block first so the header actually gets emitted * fix(core): reflect every turn (configurable reflectEveryTurns, default 1) instead of once per session * feat(core): per-turn injection is recall-only (drop reflect from the hook), recall token budget default 750 * feat(core): inject a user-feedback section above memories (capture-initiative + attribution-header preferences) * fix(core): align user-feedback attribution bullet with the generous WHEN-IN-DOUBT-EMIT rule * fix(core): sharpen capture_initiative trigger — call right after plan approval, before implementation * feat(survey): raise default codebase-survey budget cap to $2 (0.5 was over-conservative) * feat(codex): codex-v2 wrapper (SessionStart seed + per-turn recall + MCP); parametrize session-start/MCP harness * feat(core): default bank template is harness-neutral coding-agent::{gitProject} (shared memory across agents) * feat(core): default apiUrl is Hindsight Cloud (https://api.hindsight.vectorize.io); local is now an override * feat(codex): Stop write-back — Codex rollout transcript reader + codex-stop-hook (full parity) * fix(core): captureInitiative returns the server-assigned page id (not the slug) so read_knowledge_page + relatedPageId links resolve * feat(coding-agents): upgrade opencode adapter to full v2 parity Per-turn recall via chat.message + system.transform (750-tok budget), native hindsight_* tools registered directly through opencode's tool() (no MCP server), rich tool-aware write-back on by default, and cold-check auto-seed at plugin load — reusing the shared formatMemories / buildKnowledgePreamble / buildKnowledgeTools / buildSessionStartContext primitives so opencode matches Claude Code and Codex. Adds transcript-opencode.ts (rich normalizer over the live message list). Adds a HINDSIGHT_DISABLE_HOOKS recursion guard to RuntimeCore (seed/recall/write-back/sync no-op; tools still register) for headless survey runs. Removes the now-dead reflect path (client.reflect, inject.ts/buildSystemInjection, reflectTimeoutMs) as the whole surface is recall-only. README rewritten to the recall/knowledge-page/seed/write-back v2 model. * feat(coding-agents): harness-portable codebase survey (multi-agent headless) The cold-repo survey no longer hardcodes headless `claude` — startCodebaseSurvey now runs under the current harness's own CLI (claude/codex/gemini/opencode), falling back to any available agent, so a Codex/Gemini/opencode user without claude installed still gets the survey (the git-log seed already ran regardless). Per-agent read-only recipes: claude (-p + inline --mcp-config + --disallowedTools), codex (exec --sandbox read-only + inline -c MCP), gemini (-p --approval-mode plan --allowed-mcp-server-names hindsight --skip-trust), opencode (run --agent plan; tools from the loaded plugin under the HINDSIGHT_DISABLE_HOOKS guard). All spawned with HINDSIGHT_DISABLE_HOOKS=1. session-start threads the harness through to the survey. * feat(gemini): add Gemini CLI v2 integration (gemini-v2) Full v2 parity for Gemini CLI (>=0.52.0), which added a Claude-style hooks system (stdin/stdout JSON). Maps onto the shared HookSpec/runSessionStartHook/runRetainHook abstraction with Gemini's event names: BeforeAgent (per-turn recall -> hookSpecificOutput.additionalContext), SessionStart (seed), SessionEnd (write-back). The one Gemini-specific piece is transcript-gemini.ts — a reader for the 0.52.0 chats/session-*.jsonl mutation-log (upsert-by-id, polymorphic content: user text arrays, assistant plain strings, tool results as user functionResponse parts; drops the synthetic session_context message + thoughts). Adds the gemini-v2 wrapper (build.mjs + dev-install.sh that merges hooks + mcpServers into ~/.gemini/settings.json). Validated: reader against a real transcript, and a live recall smoke test (recall_ok) end-to-end. * style(coding-agents): prettier-format README config table * fix(opencode): inject via lastInjection fallback (1.18.5 system.transform has no sessionId) opencode 1.18.5 fires experimental.chat.system.transform with input {model} only — no sessionId — so RuntimeCore.getInjection(input.sessionID) looked up undefined and pushed nothing into the system prompt. Recall still ran (chat.message does pass sessionID) but the memory block + attribution preamble + knowledge-page guide never reached the model, so no visible header and no tool use. getInjection now falls back to the most recent turn's block (lastInjection) when there's no session-keyed hit. The completion's system.transform fires right after that session's onPrompt, so lastInjection is this turn's block. Adds an inject_ok/inject_empty diag (matching recall_ok/seed_started) to confirm injection lands. * fix(coding-agents): treat project-local config as untrusted (block apiUrl/apiToken/directoryBankMap from a repo) A project-local .hindsight/coding-agent.json lives inside whatever repo the developer opens, so it is untrusted input. loadConfig previously merged it per-field over the user-global config, letting a repo override apiUrl while the user-global apiToken survived the merge — so a malicious repo could set only apiUrl and the client would send the user's real Bearer token plus every recall query (the prompt) and Stop write-back transcript to an attacker-controlled host, silently, just by opening the repo (verified end-to-end). Fix: the project-local layer is now sanitized — apiUrl, apiToken, and directoryBankMap are stripped from it (top level + any harnesses.<name> section) with a one-line warning; the user-global config stays trusted and unrestricted, and a repo can still set its own per-repo bank (bankId/bankIdTemplate). Also skip re-applying the global file as a project layer when the upward findProjectConfig walk lands back on it (a repo under $HOME with no closer config), which would otherwise strip its own apiUrl and warn every session. Adds 4 regression tests. * style(coding-agents): prettier-format config.ts * feat(coding-agents): restore reflect as the memory path; per-turn injection from knowledge-page sections One opinionated runtime path (no behavior config): - reflect ONCE per session on the first prompt (agentic root-cause synthesis, benchmark-proven), cached and re-injected every turn — hook harnesses and the opencode runtime alike - every turn: knowledge-page SECTIONS matched locally against the prompt (lexical section index, no server/LLM call) injected with provenance and a pointer to the full page — fast like recall, organized like reflect - raw recall leaves the runtime path (still powers the hindsight_search_memory tool) Session write-back: transcripts are now JSON turns matching the backfill chat format, with each tool call compacted to a role:"action" turn naming the tool and its primary target (no arguments, no outputs) — Claude, Codex, Gemini and opencode readers. Knowledge pages: no more entity_labels/tag taxonomy — pages are unscoped, each page's source_query selects from the whole bank; survey, gitlog seed, write-back and security hardening stay. Spec: docs/superpowers/specs/2026-07-27-reflect-pages-runtime.md * test(coding-agents): rewrite unit tests for reflect+pages runtime and JSON action transcripts * test(coding-agents): live suite matches reflect_ok by content (pages_ok now follows it in the diag stream) * docs(coding-agents): README + docs page describe the reflect+pages runtime (reflect once per session, local page-section injection per turn, JSON action write-back) * feat(coding-agents): drop the backfill CLI — ingestion is automatic and background - new deepen engine (dist/deepen.js, unpublished): idempotent, resumable — per-bank lock, dedup by document id; ingests missing conversations, the one-time gitlog seed, then progressively deepens recent history with per-commit full diffs (newest first, bounded batch per run); drains and creates knowledge pages last - every session start now fires the engine (cold or warm); survey and the cold-seed note stay cold-only - sync status is the new readiness contract: hindsight_sync_status agent tool + dist/status.js for harnesses (synced = gitlog seeded, pages present, extractions drained); activeOperations() filters terminal ops - opencode write-back now upserts every turn (async) so a killed session loses at most the last turn - repoNameOf resolves relative paths so document ids are path-spelling-proof - hindsight-coding-backfill bin removed; benchmark/e2e run the engine directly and poll status * polish(coding-agents): short, non-technical cold-seed message highlighting the bank id * polish(coding-agents): cold-start banner — HINDSIGHT unicode wordmark + bank id line * feat(coding-agents): timing diagnostics on by default - session_start diag event on EVERY session (bank, cold/warm, pages, ms) — warm sessions previously logged nothing - deepen engine: deepen_started/deepen_done/deepen_failed diag events with duration; child output now appended to ~/.hindsight/coding-agent-state/deepen.log (was stdio:ignore — undebuggable) and log lines timestamped - retain_ok/retain_failed carry ms on both the Stop hook and the opencode per-turn upsert (which was fully silent) - vitest config pins HINDSIGHT_DIAG_FILE to a tmp file so unit tests stop polluting the real diag log * feat(coding-agents): show the Hindsight banner on every session start (cold: learning, warm: remembering) * polish(coding-agents): session banner uses the API server's colored pixel-art logo (shared visual identity), wording line below * polish(coding-agents): banner text before logo — the TUI's first-line prefix was displacing the logo's top row * polish(coding-agents): banner logo re-rendered foreground-only — the TUI strips ANSI background colors, which deleted half the server logo's pixels * feat(coding-agents): per-turn user-visible notice — every prompt shows what Hindsight delivered (reflect state + matched knowledge pages) via hook systemMessage; opencode logs the same line * polish(coding-agents): per-turn notice shows the match query excerpt and the page titles it returned * fix(pages-index): singularize plain-word tokens so plural prompts match singular headings ('components' -> 'Component map'); path-like tokens untouched * polish(coding-agents): per-turn notice — gradient Hindsight wordmark, value-driven wording, no timings * feat(coding-agents): interim always-inject knowledge stub + explicit Hindsight attribution - selectSections: TEMPORARY stub returning the first section of up to 3 distinct pages every turn regardless of prompt — guarantees injected data for testing source attribution; will be replaced by the server-side knowledge-base/search (local lexical index drops with it) - both injection blocks now carry an ATTRIBUTION directive: when memory shapes the answer, the agent introduces it with '🧠 From Hindsight memory (<page>)' — and must never credit memory that did not contribute * polish(coding-agents): gradient-word banner (logo dropped), lean per-turn notice, attribution directive front-loaded as a mandatory output format * polish(coding-agents): reflect turn notice shows the assigned goal and a preview of what memory returned * feat(coding-agents): page knowledge moves from auto-injection to an explicit tool - new hindsight_search_knowledge_pages(query) tool (native on opencode, MCP on hook harnesses) — interim local selection, single swap point for the server-side knowledge-base/search; results carry the attribution requirement - per-turn auto-injection of page sections removed: a trivial prompt ('yes') no longer displays phantom research; ordinary turns are silent - per-turn notice only on the reflect turn (assigned goal + result preview); tool calls provide their own native visibility - tool guide/roster advertises the search tool as the first stop * feat(coding-agents): bind hindsight_search_knowledge_pages to the server-side hybrid knowledge-base search - merge feat/knowledge-pages-okf underneath (GET /knowledge-base/search, BM25 + vector, RRF-fused; conflicts resolved in okf's favor for server/ clients/UI, coding-agents docs entry preserved) - client.searchKnowledgePages(query, limit) wraps the endpoint; the tool returns ranked {page, page_id, snippet, score} — verified end-to-end through the real MCP server against the live endpoint - interim local selection removed from the tool path (pages-index remains only for the hook page cache pending full cleanup) * refactor(coding-agents): drop pages-index — local section index deleted; hook/runtime keep only the id+title roster (content lives behind the server-side knowledge-base search) * refactor(coding-agents): drop hindsight_search_memory (raw recall) — knowledge-page search is THE search surface; recall client method and formatter removed * feat(coding-agents): hindsight_reflect tool — on-demand deep memory reasoning alongside the session-start reflect * refactor(coding-agents): one 'conversation' retain strategy for all developer conversations Backfilled decision chats and live session write-back were the same content type (identical JSON action-transcript format) extracted two ways based only on where they came from. Merged CHAT_MISSION + SESSION_MISSION into one CONVERSATION_MISSION that scales facts to substance (short decision chat -> 1-2 facts, working session -> several; final-state-wins, verbatim literals, rejected-alternative rule kept); the ≤2-fact CHAT_CUSTOM_INSTRUCTIONS extractor is retired with it. * feat(coding-agents): restore Chris's knowledge entity_labels tier configureBank again sets entity_labels {knowledge: feature-work/decision/ convention/component/concept, tag:true} + entities_allow_free_form, so the extractor routes durable facts with knowledge:<tier> tags the server-side knowledge base can select on; capture_initiative markers regain the knowledge:feature-work label. Pages themselves stay unscoped (the okf knowledge base owns synthesis). * feat(coding-agents): seeded pages tag-scoped again — page tags match the restored knowledge:<tier> entity labels (capture_initiative pages included) * fix(coding-agents): reflect injection wrapped in <hindsight_memory> so write-back never re-ingests it; seed-state file (declined flag) removed — the live bank is the only state * feat(coding-agents): gitIngest enum ('message' | 'full' | 'none') — one setting, one code path for seeding AND staying current - deepen's idempotent git pass IS the sync: gitlog doc re-upserts when HEAD moves (gitlog-head:<sha> tag makes freshness a single tag query); in full mode new commits surface at the top of rev-list and the next run ingests them - separate git-sync path deleted (sync.ts, runtime.syncGitOnce, gitSync config) * feat(coding-agents): gitIngest defaults to 'message' (cheap by default; opt into depth); deepen gains --git-ingest override for harnesses * feat(coding-agents): session banner shows git-sync state (condensed syncStatus): 'git in sync' / 'catching up on new commits' / 'syncing git history (n/target)' * polish(coding-agents): two-line banner — value headline (tracking decisions/conventions/history) + bank/sync detail line * refactor(coding-agents): ONE config file — project-local .hindsight/coding-agent.json layer removed entirely (with its sanitization machinery); per-repo routing stays via directoryBankMap * docs(coding-agents): fix stale project-config reference in comment * refactor(coding-agents): runtime scratch (deepen lock + engine log) moves to the OS temp dir — ~/.hindsight now holds ONLY the config file * feat(coding-agents): cursor auto-ingestion parity — hosts without a SessionStart hook fire the deepen engine (+ cold survey) from the session's first prompt * feat(coding-agents): leveled plugin logging — one plugin.log (debug/info/warn/error, config logLevel + HINDSIGHT_LOG_LEVEL/FILE overrides); diag events mirror at debug; deepen logs itself (separate deepen.log dropped); warn on reflect/retain failures * feat(coding-agents): one-shot bank configuration via the server's template import — missions, strategies, entity labels, and the 5 seeded pages in a single idempotent POST /import (configureBank PUT+PATCH and createPages removed) * feat(coding-agents): one-command installer — npx hindsight-coding-agents install|uninstall [harness...] Detects the coding agents on the machine and merges each one's native wiring (hooks + MCP: claude mcp add for Claude Code; hooks.json + append- only config.toml sections for Codex; settings.json for Gemini; hooks.json + mcp.json for Cursor; plugin array for opencode). Idempotent by marker, preserves foreign entries, backs up touched files as .hindsight-backup; uninstall removes exactly ours. 27 unit tests over temp homes. * fix(installer): refuse to install from an npx/dlx cache (wired paths would die on eviction); document global install + npm update -g as the update path * ci(coding-agents): unit + typecheck + build job, and a live E2E job (real API server + real LLM) running the deepen->sync->reflect->injection path; prettier-format the package * docs(blog): launch post draft — coding-agent memory results (marked draft: true) * docs(blog): rewrite launch post as the narrative — from 'does memory even help?' through why-not-SWE-bench, the corrections dataset, benchmark-driven architecture decisions, to the final numbers * docs(blog): position knowledge pages as a co-launch headline — living-documents framing, example page excerpt, platform-wide availability (dashboard editor, hybrid search API, bank templates), closing CTA * docs(blog): restructure launch post payoff-first — contrarian RAG finding + cost in the lede, TL;DR box, narrated task with both runs, seeded-answers objection met head-on, data-locality/time-to-value/latency answers, Sonnet number promoted, backstory compressed to one section * fix(coding-agents): deepen waits for server-side ops to settle (template-import page refreshes broke the synced contract); HINDSIGHT_CONFIG env override for the config path (containers/test harnesses; replaces the live test's dependency on the removed project-config layer) * docs(blog): second-pass fixes — flagship example swapped to the arbitrary retry decision (RFC 4180 attack closed), reconstruction disclosed, 58% provenance clause, placebo backstory + grading block restored in numbers, RAG figure per-task, benchmark-site date * docs(blog): align remaining CSV references with the retry flagship; TL;DR per-task figures * docs(blog): flagship rebuilt on the real dataset task — the ERP export decision whose rejected alternative IS the textbook fix (='00042' formula form, minimal quoting, CRLF); dangling injection-verified reference restored; limitations cross-check attached to the correct row * docs(blog): rewrite as the 0.9.0 launch post — five-beat narrative (question → dataset → auto-recall failure → reflect → knowledge pages from llm-wiki to self-healing) for Knowledge Pages + unified coding-agents plugin * docs(blog): add the missing beat — shaping the dataset revealed decisions live in git, which the old plugins never ingested * docs(blog): reframe reflect — very smart rather than slow; first message carries the session goal; on-demand reflect tool for session drift * docs(blog): pages section addresses the 'back to files?' objection — pages as projected views over consolidated memory (contradiction resolution underneath), raw docs remain source of truth * docs(blog): out-of-box row updated to n=3 (22/26/23 -> 0.72/task, -26%; cost -35%); matured row marked single-run * fix(hooks): reflect block injected once per session (+ cadence refresh), not every turn — hook context persists in the transcript, so per-turn re-injection stacked duplicate blocks * fix(coding-agents): wrapper bundles ship deepen.js, not the renamed backfill.js The core build entry `backfill` was renamed to `deepen` (deepen engine + status), but the three wrapper build.mjs bundleFiles lists still copied the removed `backfill.js`, so every dev-install failed with ENOENT. Point them at `deepen.js` (spawned by seed.ts at runtime) so the installers build again. * feat(coding-agents): periodic re-survey — refresh structural pages every N commits Structural knowledge pages are only generated on a cold repo, so an evolving architecture drifts from what the survey captured. Add surveyRefreshCommits (default 20; 0 = cold-seed only): at SessionStart, count commits reachable from HEAD since the newest survey-baseline marker (branch-robust via git.commitsSince) and re-run the headless survey once the threshold is crossed, re-recording a baseline marker. Cold seed still records the first baseline. * fix(coding-agents): per-turn hook timeout (30s) must exceed the 25s reflect cap The once-per-session reflect is capped internally at HOOK_REFLECT_CAP_MS=25s, but every harness killed the UserPromptSubmit/BeforeAgent hook at 15s — below the cap. The host killed the hook mid-reflect before the cache write, so the injection was discarded AND the reflect re-fired uncached on every turn ("UserPromptSubmit hook timed out after 15s" every prompt). Raise the hook timeout to 30s (> cap) across claude/codex/gemini, bump Stop to 30 to match, and document the cap-below-timeout invariant so it can't silently drift again. * polish(coding-agents): attribution header is a bold blockquote callout, not flat text The live directives all told the agent to credit memory with a plain inline "From Hindsight memory (<page>):", which renders as flat text. Switch every directive (session tool-guide, reflect injection, both MCP tool descriptions) to a markdown blockquote header "> ... **From Hindsight memory (<page>)** — ..." so it renders as a distinct callout, restoring the richer attribution look. * fix(coding-agents): strip <hook_prompt> transport wrappers from retained transcripts (codex surfaces hook stdout/errors as user messages); session + backfill transcripts switch to JSONL (one turn per line — clean appends, chunker-atomic turns) Note: benchmark numbers (n=3) were measured on the JSON-array format; JSONL is extraction-equivalent by design but unvalidated by a sweep — gate before quoting new numbers on this pipeline. * fix(installer): write [features].hooks (codex_hooks deprecated in Codex >= 0.145); accept either flag as already-enabled * fix(hooks): fire the ingestion engine from the FIRST prompt on every harness (lock-protected no-op when SessionStart already did) — safety net for sessions predating the install, whose banks otherwise never get pages; survey stays SessionStart-owned (ensureSeed hosts excepted) * feat(status): expose survey observability — surveyBaseline (last surveyed HEAD, from Chris's survey-baseline markers) + surveyCommitsBehind in syncStatus/hindsight_sync_status * test(status): expected shapes include the survey observability fields * feat(survey): findings docs ARE the completion signal — surveyDocs (0-4) in syncStatus; a baseline without findings re-fires the survey at the next warm session start (crashed-survey retry) * feat(config): banks.<bankId> overrides — per-repo opt-in/out applied AFTER bank resolution (disable a repo, tune gitIngest/retainSessions per bank) from the ONE config file; resolution fields ignored inside a bank section * feat(config): bankAliases — remap resolved bank ids as the final resolution step (single hop, converging allowed); docs page brought fully current (env exceptions, gitIngest/logLevel/survey rows, banks overrides, aliases, resolution step 4) * refactor(config): bank rename lives INSIDE banks.<id> as the field (separate bankAliases tree removed) — one per-repo section for disable, behavior, and rename; applyBankConfig returns {cfg, bankId} * docs(coding-agents): recipe — two repos sharing one bank (converge by resolved id via banks.<id>.bank, or by path prefix via directoryBankMap), with the id-vs-path rule of thumb * rename(config): directoryBankMap -> mapPathToBank (direction-explicit; pre-0.9.0 breaking-rename window) * feat(coding-agents): companion skill — hindsight-coding-agent SKILL.md shipped in the package and installed into ~/.claude/skills by the installer; explains storing/retrieving, full config (banks/mapPathToBank/gitIngest), install/update, and debugging * docs(coding-agents): mention the companion skill in README + docs page * feat(coding-agents): companion skill ships to ALL skills-capable hosts (claude/gemini/cursor native dirs, codex via ~/.agents/skills standard); retained sessions and ingested documents carry the harness as tag (harness:<name>) and metadata * feat(skill): self-updating companion skill — every session start re-syncs installed copies with the packaged SKILL.md (presence-gated; npm update -g now updates the skill too, no re-install) * fix(coding-agents): worktree-aware document ids (no more per-worktree gitlog duplicates) + deepen self-cleanup; issue/PR refs preserved verbatim and emitted as ENTITIES; calibrated reflect-injection wrapper; docs ported to the TRUE source (hindsight-docs/docs-integrations) that generates the skill copy * docs(skill): explain the internal marker documents (survey-baseline:<sha> bare-sha content is deliberate — zero extracted facts; gitlog:<repo> seed doc) * feat(survey): human-readable baseline markers under a zero-extraction marker strategy (live-verified: 0 facts) — start as researching, deepen lazily flips to completed once findings exist * refactor(survey): one survey strategy with conditional rules replaces the separate marker strategy — status markers extract nothing, findings extract structural facts (both branches live-verified) * fix(hooks): mid-session heal — zero knowledge pages in the roster cache fires the ingestion engine on any prompt (covers long-lived sessions predating the install; lock makes repeats free) * feat(bank): ~ expansion in mapPathToBank; document the directory-blacklist recipe (map tree to one bank + disable it) * feat(coding-agents): explicit correction protocol — when the agent verifies a memory is wrong/stale it ingests a 'Correction: <topic>' doc (claimed vs verified-true vs evidence); guidance in the injection wrapper, tool guide, tool description, and companion skill * fix(hooks): reflect block injected exactly once — cadence re-injection dropped (replaying the turn-1 synthesis at arbitrary turns reads as random noise after drift; hindsight_reflect covers genuine re-need) * fix(coding-agents): 15s hard timeout on every client request + opencode boot no longer awaits seedIfCold — a stalled memory server can never freeze the host TUI (onPrompt already tolerates a late preamble) * fix(reflect): defer past trivial openers — a greeting no longer spends the once-per-session synthesis on 'hi' (seen live: reflect answered a greeting with persona chatter and burned the session's slot); first substantive prompt reflects instead * test(hooks): align reflect-call assertions with the non-trivial fixture prompt * Revert trivial-prompt reflect deferral (misread the report — the issue was the notice's UI position, not reflect-on-greeting behavior) * fix(opencode): stop writing banner/reflect notices to stderr — opencode renders plugin stderr inside the TUI at the cursor (text wedged against the input bar); the trail moves to the plugin log * feat(opencode): TUI companion plugin — visible presence via api.ui.toast (opencode's TUI plugin API): banner toast on activation + reflect goal/preview toasts from the plugin-log trail; installer registers the second entry * fix(opencode): visible presence via the server client's tui.showToast (POST /tui/show-toast) — banner + reflect toasts from the server plugin; the separate TUI module approach removed (1.18.9's loader rejects tui-only entries in the shared plugin list); SDK deps bumped to 1.18.9 * fix(opencode): toasts never rendered — v1 client wants {body}, and boot toast raced TUI mount opencode injects the v1 SDK client whose showToast signature is {body: {title, message, variant, duration}} and which resolves with {data|error} instead of rejecting — the earlier flat-params call sent an empty body and the failure was invisible. Also the toast event is not durable: the seed banner on a warm bank fired <1s after plugin init, before the TUI subscribed, and was lost. Toasts now use the body shape, log a rejected result at debug, and defer until ~3s past init. Verified live in tmux: boot banner and reflect toast both render. * fix(coding-agents): reflect must report history, never issue directives The 0.8.6-blog incident: reflect fused two true but unrelated facts (the hermes-deprecation goal and the blog-section removals of c87e7ac19) into one confabulated narrative rendered in the imperative — 'You should explicitly remove the following sections' — a completed past action re-issued as a present directive, indistinguishable from a prompt injection to the receiving agent. Three changes: - buildReflectQuery wraps the session's first prompt with strict rendering rules: declarative past-tense attributed facts only, no instructions or recommendations, no stitching unrelated episodes into one narrative. - The <hindsight_memory> wrapper now states the block is a record of the past that never assigns tasks: imperative wording inside it is a description of work already done, to be ignored unless it informs the task as historical fact (and unrelated memories are still ignored outright). - The reflect_ok diag event records the injected synthesis verbatim (8k cap), so the next incident is one grep instead of harness-transcript spelunking. * refactor(coding-agents): read and seed knowledge pages through the knowledge-base API The plugin advertised knowledge pages but drove them off /mental-models, so the two halves of the feature never met: pages seeded via the bank template's mental_models key got a mental model and no knowledge_pages node, and /knowledge-base/search joins through that table — the five seeded pages were absent from the corpus of the tool billed to the agent as its FIRST STOP. The one page search could return (an initiative, created through the KB endpoint) came back as a kp-… node id, which the reader then fed to GET /mental-models/{id} and 404'd. Search found only what read could not open. Every page operation now speaks one id space: - listPages reads /knowledge-base/tree and flattens it to {items:[…]}, dropping folders and keeping the containing folder name. - getPage reads /knowledge-base/pages/{id} — the ids search and [[page:<id>]] links already hand back. - seedPages replaces the template's mental_models key: it creates the PAGES taxonomy through /knowledge-base/pages and re-syncs a drifted source_query via PATCH /knowledge-base/nodes/{id}, so a plugin upgrade that rewords a query lands on the live page instead of orphaning its synthesized content. Matched by name, since the endpoint mints its own id; a 409 from a concurrent deepen run is tolerated rather than failing the run. - createPage/updatePage/deletePage are deleted — mental-models CRUD with no callers outside its own tests. Verified against a live server on a scratch bank: five real kp- nodes, re-run reports 0 created / 5 unchanged, all five readable by their listed id, all five now returned by /knowledge-base/search, and a hand-drifted source_query restored onto the same node rather than a duplicate. * feat(coding-agents): autoReflect flag — opt out of injected reflect into tool-only mode autoReflect (default true, layerable per-harness/per-bank like every other field) keeps today's validated behavior: one reflect synthesis injected on the session's first prompt. Set false and nothing is injected; instead the knowledge preamble and every roster refresh carry an explicit trigger telling the agent to call hindsight_reflect itself whenever a new task/goal is set — the pull-based variant, ready to benchmark against the push default. * docs(blog): move the 0.9.0 launch post to its own PR The draft now lives on blog/0-9-0-launch so this PR merges independently of launch timing (hero image, publish date, and final voice pass pending there). * fix(deepen): dead-holder locks are stale immediately, not after 30 minutes The per-bank deepen lock only honored its TTL: a killed run (SIGKILL, crashed harness) left its bank locked for LOCK_STALE_MS, and every subsequent deepen exited 'another run holds the lock — nothing to do' against an empty bank. The lock already records the holder's pid — probe it (kill -0); if the holder is gone the lock is stale now. Found live: a killed benchmark ingestion left four banks locked and the retry campaign polled empty banks to its deadline. * feat(coding-agents): expand native harness support * fix(reflect): table-shaped decisions must be reproduced verbatim, not summarized Benchmark replay showed reflect compressing mapping/table policies into prose ('specific extensions map to specific types') and even asserting a lossy generalization that matched a known-wrong fix — while rule-shaped policies survive intact. The reflect query now demands complete verbatim enumeration of mappings/sets/tables including carve-outs. * fix(reflect): decisions outrank implementation-derived memory Under heavy retrieval noise, reflect surfaced the git-ingested BUGGY module source as 'the established implementation logic' while claiming no decision records existed — presenting the bug under investigation as authority. The rendering rules now state: report decisions and rationale, never the current implementation (the reader has the code); when decision memory and code-derived memory conflict, the decision wins; implementation-only matches are not policy. * feat(coding-agents): expand harness integrations * Expand coding-agent integrations and legacy compatibility * chore(coding-agents): fix the CI-only test failure and complete the release wiring The `test-coding-agents` job failed on every run while passing locally: the gitDiffTarget fixture committed into a temp repo without a git identity, which a developer machine supplies from its global config and a CI runner does not ("empty ident name not allowed"). The identity is now passed per-command, the way the harness E2E fixture already did it. Release wiring, which was incomplete in three places that each fail at a different point: - scripts/release-integration.sh had no entry, so the release refuses to start. - generate_changelog.py keeps its OWN integration list; the release script aborts and reverts at the changelog step when a name is missing there. - The docs build cross-checks released tags (`integrations/<name>/vX.Y.Z`) against the SLUGS in integrations.json. The directory was the only integration carrying a `hindsight-` prefix, so the tag would have been `integrations/hindsight-coding-agents/...` against a `coding-agents` slug — green release, then a failing docs build. The directory is renamed to `coding-agents` so directory, integration name, tag and docs slug all agree, matching every other integration. Also drops the claude-code-v2 / codex-v2 / gemini-v2 wrappers and the hindsight-memory-v2 marketplace entry. Claude Code is fully served by `hindsight-coding-agents install claude-code` — hooks, MCP and skill — so the wrappers were a second copy of the same core with its own version to keep in lockstep. The README rows that pointed at their dev-installers now name the supported installer command instead. * fix(coding-agents): make the installer actually re-point a moved package Both bugs were exposed by the directory rename, which invalidated the absolute paths every host config stores — the case `install` exists to repair. - Grok wrote its block only when one was absent, so every later `install` was a silent no-op and the dead paths survived; the only repair was editing config.toml by hand. It now replaces the block, sharing one regex with uninstall. - MARKER was the full package name, which identifies our entries for dedupe-on-reinstall and for uninstall. A repo checkout stopped containing it once the directory dropped its `hindsight-` prefix, so from a checkout re-installs would have accumulated duplicate hook entries and `uninstall` would have removed nothing. Narrowed to the substring both layouts share. Regression tests cover a moved package being repointed (not appended past), the marker matching npm and checkout paths, and a repeated checkout install leaving one entry per event. --------- Co-authored-by: Chris Latimer <chris.latimer@vectorize.io>	2026-07-31 22:15:21 +02:00
Scott Guymer	6500944c74	feat(copilot-cli): add GitHub Copilot CLI hooks integration (#2742) ... * feat(copilot-cli): add GitHub Copilot CLI hooks integration Add hindsight-integrations/copilot-cli/, giving GitHub Copilot CLI persistent long-term memory via Hindsight hooks (see docs.github.com/en/ copilot/how-tos/copilot-cli/customize-copilot/use-hooks). Modeled on the existing cursor-cli integration. Hooks: - sessionStart: recall using initialPrompt (or a cwd-derived fallback query), injects additionalContext - subagentStart: recall for every subagent Copilot CLI spawns (explore, task, research, code-review, rubber-duck, security-review, and custom agents, not the built-in general-purpose agent, which never fires this hook). Subagent payloads carry no per-invocation task text, so this always uses the fallback query. - agentStop: reads the transcript, retains to Hindsight on a configurable turn cadence, caches the transcript path for sessionEnd - sessionEnd: forces a final retain using the transcript path cached from the last agentStop, since sessionEnd's own payload has no transcript path field Install via pip install hindsight-copilot-cli, then hindsight-copilot-cli install (user scope, writes ~/.copilot/hooks/hindsight-copilot-cli.json) or --scope repo for a team-shared .github/hooks/ registration. Zero runtime dependencies, hook scripts are pure stdlib Python. Also wires up CI (test-copilot-cli-integration job), release-integration.sh and generate_changelog.py registration, and docs gallery/sidebar entry. Closes #1588 * fix(copilot-cli): regen skill mirror, drop unreleased changelog link - Run generate-docs-skill.sh to add the missing skill mirror for the new copilot-cli doc page (verify-generated-files was failing on the untracked references/sdks/integrations/copilot-cli.md). - Remove the [View Changelog] link, which pointed at /changelog/integrations/copilot-cli — a page the release script only creates on first release, so it was a broken link failing build-docs.	2026-07-28 09:52:59 -04:00
Ben	b11e053323	feat(zcode): add Hindsight long-term memory integration for ZCode (#2549) ... * feat(zcode): add Hindsight long-term memory integration for ZCode Adds a hooks-based, no-MCP integration for ZCode (Z.ai's GLM desktop coding agent). ZCode embeds the Claude Code agent runtime and reads the standard Claude Code hook schema from its own config namespace (~/.zcode/cli/config.json), so `hindsight-zcode install` wires three process hooks — SessionStart, UserPromptSubmit (recall), and Stop (retain) — without touching the user's ~/.claude config and without an MCP server. Recall injects relevant memories as additionalContext before each prompt; retain assembles each turn from the prompt (captured at UserPromptSubmit) and the response (Stop payload) and stores it to Hindsight. Verified end-to-end in ZCode 3.2.2: hooks fire, retain persists to the cloud bank, and recall injects memory into the agent. Includes the pip package + installer, hook scripts, tests, CI job, release-integration wiring, changelog registration, docs page, and gallery entry. * feat(zcode): add self-serve marketplace + hooks-only plugin variant Publishes the ZCode integration as a hooks-only Claude Code plugin (hindsight-zcode) in the repo's plugin marketplace, so ZCode users can install it via 'zcode plugins add-marketplace vectorize-io/hindsight' without pip and without depending on Z.ai's marketplace. The plugin reuses the pip package's hook scripts via CLAUDE_PLUGIN_ROOT (no duplication) — settings.json resolves as a sibling of scripts/ in both the pip and plugin layouts. Adds a plugin manifest, plugin-format hooks.json (SessionStart/UserPromptSubmit/Stop — no SessionEnd), marketplace entry, validation tests, and docs. * fix(zcode): drop changelog link from docs page (page exists only after release) The /changelog/integrations/zcode page is generated at release time, so linking to it broke the Docusaurus build (build-docs + verify-generated-files). Most unreleased integration pages omit this link; follow that convention.	2026-07-20 10:58:24 -04:00
Nicolò Boschi	5bfef3caa4	revert(docs-skill): drop cookbook pages from the docs skill bundle (#2818) ... #2649 added both cookbook pages and per-integration docs to the generated docs skill. Keep the integration docs; remove the cookbook. - drop the cookbook tree walk and the CookbookGrid MDX renderer from generate-docs-skill.sh - drop cookbook paths from the generated SKILL.md index - regenerate the bundle (28 cookbook files removed)	2026-07-20 10:53:47 +02:00
Parafee41	d2ca26afaf	Include cookbook and integration docs in docs skill (#2649) ... Extends generate-docs-skill.sh to walk hindsight-docs/src/pages/cookbook/ and docs-integrations/, so the docs skill bundle ships the cookbook recipes/applications and per-integration docs its SKILL.md already advertised. Fixes the ghost-path index described in #2641. Regeneration is drift-free (verify-generated-files passes) and link validation passes; bundle grows from ~85 to 168 files. Fixes #2641	2026-07-10 16:08:40 -04:00
DK09876	fcb2c958e7	feat(devin-desktop): rename Windsurf→Devin Desktop + fix(continue) thread-safe adapter (#2410) ... * feat(devin-desktop): rename windsurf integration to Devin Desktop Cognition rebranded Windsurf to Devin Desktop (June 2026); Cascade is EOL July 1. Rename the (unreleased) windsurf integration to devin-desktop before first publish: - Package hindsight-windsurf -> hindsight-devin-desktop (module hindsight_devin_desktop, CLI hindsight-devin-desktop, DevinDesktopConfig, bank default 'devin-desktop', HINDSIGHT_DEVIN_DESKTOP_BANK_ID) - Rule now writes to .devin/rules/hindsight.md (preferred path) instead of the legacy .windsurf/rules/; trigger: always_on unchanged - MCP config path stays ~/.codeium/windsurf/mcp_config.json (Devin Desktop's on-disk data dir, unchanged by the rebrand) - Official Devin logo; docs + integrations.json + README refreshed with the 'formerly Windsurf' framing - Registries updated: test.yml job, release-integration.sh, generate_changelog, integrations.json (strict JSON), docs page 26 unit tests + gated live-MCP E2E pass; ruff check+format clean; real-app smoke against local Hindsight verified (init writes both files; live recall returns seeded facts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(continue): resolve a fresh Hindsight client per request (thread-safe) The adapter runs on a ThreadingHTTPServer (one worker thread per request) but shared a single Hindsight client across all of them. The client's aiohttp session is bound to the thread/event-loop that first used it, so the first @hindsight recall worked and every one after threw 'Timeout context manager should be used inside a task' — Continue then showed an error context item and the model answered with no memory. Resolve the client per request (test-injected clients still used as-is), and close per-request clients in a finally so the fresh aiohttp session doesn't leak a connector each call. Bump to 0.1.1. Found via a real in-editor VS Code test. Adds a regression test asserting per-request client resolution across the threaded server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 17:27:39 -07:00
Nicolò Boschi	b6608076ff	fix(release): bump marketplace version on claude-code release (#2386) (#2398) ... The Claude Code plugin ships via the marketplace manifest, not a package registry. The integration release (release-integration.sh claude-code) already bumps plugin.json, but the marketplace manifest carried no version and was never bumped — so the published catalog never reflected new releases (e.g. #2066 on Windows). - add a "version" field to the root .claude-plugin/marketplace.json - release-integration.sh now bumps it in lockstep with the plugin version when releasing claude-code, and commits it - remove the redundant hindsight-integrations/.claude-plugin/marketplace.json: `claude plugin marketplace add vectorize-io/hindsight` only ever reads the root manifest (even with --sparse), so the second manifest was never consulted - drop the stale --sparse install hint from the release-integration workflow The claude-code release flow is otherwise unchanged — release it as before.	2026-06-25 13:31:00 +02:00
Sanderhoff-alt	6e02a0829f	fix(hooks): keep uv lockfile frozen during lint (#2397) ... Run the pre-commit uv sync and workspace uv run commands with --frozen so linting uses the checked-in lockfile without rewriting it during ordinary code changes. This avoids local uv resolver freshness checks producing unrelated uv.lock diffs while preserving explicit dependency update workflows.	2026-06-25 12:29:13 +02:00
Ben	d0b77f5bee	feat(eve): add Eve agent-framework MCP connection helper (#2280) ... * feat(eve): add Eve agent-framework MCP connection helper Add @vectorize-io/hindsight-eve: a thin helper that wraps Eve's defineMcpClientConnection to wire an Eve agent into a Hindsight MCP server in one line, pre-filling the endpoint, model-facing description, and bearer auth with env-var defaults (HINDSIGHT_MCP_URL, HINDSIGHT_API_KEY, HINDSIGHT_MCP_BANK_ID).	2026-06-24 14:48:38 -04:00
DK09876	7194f98b19	feat(windsurf): add Windsurf (Codeium) integration via MCP (#2358) ... * feat(windsurf): add Windsurf (Codeium) integration via MCP Config-only CLI that wires the Hindsight MCP server into Windsurf's ~/.codeium/windsurf/mcp_config.json (mcpServers, remote serverUrl + auth header) and writes an always-on recall/retain rule to .windsurf/rules/hindsight.md (trigger: always_on). Cascade then has recall/retain/reflect and uses them automatically. - hindsight_windsurf: config, mcp_config (strict-JSON parse-or-print), rules (dedicated sentinel-marked file), cli (init/status/uninstall) - 25 unit tests + gated live-MCP-endpoint E2E - CI job, release + changelog registries, docs page, icon, README row Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(windsurf): apply ruff format to cli.py lint.sh runs 'ruff format'; collapse the --rules-path add_argument to one line so verify-generated-files passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(windsurf): use official Windsurf logo for the integration icon Replace the placeholder abstract mark with the official Windsurf logo (simple-icons, CC0), matching the real-brand-logo convention used by the other integration icons. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-24 09:58:55 -07:00
DK09876	91bf32842e	feat(github-copilot): add GitHub Copilot (VS Code) integration via MCP (#2299) ... Adds hindsight-copilot: long-term memory for GitHub Copilot in VS Code, using Copilot agent mode's native MCP support (HTTP servers) — no bridge. `hindsight-copilot init`: - merges a Hindsight HTTP MCP server into .vscode/mcp.json (servers.hindsight), JSON-safe (prints a snippet if the file is JSONC), and - writes a recall/retain rule into .github/copilot-instructions.md, which Copilot applies to every chat in the workspace. Resolves the ask in #1588. Mirrors the Zed/OpenHands MCP-config pattern. - hindsight_copilot package: config, mcp_config (.vscode/mcp.json writer), instructions (copilot-instructions.md rule), cli (init/status/uninstall) - 25 deterministic tests (mcp.json merge incl. preserving servers/inputs + JSONC fallback, instructions rule block) + gated requires_real_llm MCP handshake E2E - CI job, release registration (VALID_INTEGRATIONS + changelog generator), docs page, registry entry, icon (octicons), README row Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-23 16:39:12 -07:00
DK09876	aab7032071	feat(aider): add Aider integration (session-bracketing memory wrapper) (#2297) ... hindsight-aider wraps the aider CLI: recalls project memory before each session (injected via --read) and retains the transcript after. Bank per git repo.	2026-06-18 13:52:38 -07:00
DK09876	65862c4fef	feat(openhands): add OpenHands integration (native MCP config + recall/retain rule) (#2276) ... Long-term memory for OpenHands via native Streamable-HTTP MCP: hindsight-openhands init wires the Hindsight MCP server into config.toml + a recall/retain rule in AGENTS.md.	2026-06-17 12:46:20 -07:00
DK09876	539101af38	feat(zed): add Zed editor integration (MCP context server + recall/retain rule) (#2153) ... MCP-only Zed integration: hindsight-zed init wires the Hindsight MCP server into Zed's settings.json (via mcp-remote) plus a recall/retain rule in AGENTS.md. Validated end-to-end in real Zed.	2026-06-17 10:28:07 -07:00
Ben	1c9ba0e659	feat(composio): add Composio integration (Hindsight memory as custom tools) (#2180) ... * feat(composio): add Composio integration (Hindsight memory as custom tools) Exposes Hindsight retain/recall/reflect as Composio in-process custom tools via register_hindsight_tools(). The Hindsight bank for each call is the Composio session's user_id, so one registered tool set isolates memory per user automatically. Also ships memory_instructions() for pre-recall system-prompt injection (Composio doesn't auto-inject context). - hindsight_composio/: tools.py, config.py (dataclass + env fallback), errors.py. - tests/: 50 tests using a FakeComposio (mirrors the real tool decorator + SessionContext) + mocked Hindsight client — exercises the framework wiring. - CI: test-composio-integration job (uv build/sync/ruff/pytest) + path filter. - Gallery card + doc page + official Composio icon; release-integration.sh entry. * fix(composio): register in changelog generator + test memory_instructions - Add composio to generate_changelog.py INTEGRATIONS dict (release would otherwise fail at the changelog step; it was only in release-integration.sh). - Add TestMemoryInstructions covering formatting, max_results cap, empty/error fallback, tag passthrough, and missing-config error. * address review: Literal config types, typed generics, debug log, real-LLM E2E - Type budget as Literal[low|mid|high] and tags_match as Literal[any|all| any_strict|all_strict] across config + tools (matches autogen/continue) - Parameterize bare list -> list[Any] on register_hindsight_tools - _ensure_bank: logger.debug the swallowed create_bank failure so a real auth/network error is visible rather than only surfacing later on retain - Add requires_real_llm E2E bucket exercising retain/recall/reflect through the (input, ctx) tool call path against a live Hindsight server; exclude from PR CI via -m 'not requires_real_llm'	2026-06-16 15:56:53 -04:00
Evo	abc1439675	fix(skill-docs): convert all admonition keywords in the docs→skill generator (#2218) ... The MDX→skill converter in scripts/generate-docs-skill.sh only handled :::tip / :::warning / :::note, and each rule required an inline title. So :::info and :::caution admonitions — and any title-less opener (e.g. a bare :::note) — were left as raw `:::` markdown in the CI-enforced agent-facing skill mirror (skills/hindsight-docs/references/**), where the generic `:::\s*\n` cleanup then ate the closing fence and the admonition body bled into the following section. Most visibly, #2202 added a :::caution "shared vs [[]] vs []" warning to retain.mdx, which now renders as broken raw markdown in retain.md. Teach the converter every supported keyword (tip/note/warning/info/caution) with an optional inline title, mapping each to a blockquote (title-less openers fall back to the capitalized keyword). Regenerated the skill mirror; this also repairs pre-existing :::info/:::caution/title-less leaks across the core API reference docs. Note: source files that are plain .md (e.g. configuration.md) are copied verbatim by the generator rather than run through this converter, so their admonitions are unaffected here — happy to extend the converter to that copy path in a follow-up if desired.	2026-06-16 11:12:20 +02:00
DK09876	c05ab9103f	feat(continue): add Continue.dev integration via HTTP context provider (#2213) ... Adds hindsight-continue: Hindsight memory for Continue.dev via its native http context provider (@hindsight recall) plus an optional MCP-server + rules setup. Includes the adapter package, tests against Continue's HTTP contract + a gated E2E, CI job, release registration, docs, and registry entry.	2026-06-15 14:38:39 -07:00
Nicolò Boschi	f0802b826b	chore(ci): enforce unused imports/vars + advisory dead-code scan (#2144) ... * chore(ci): enforce unused imports/vars + advisory dead-code scan Enable ruff F401 (unused imports) and F841 (unused variables) -- previously ignored as "too noisy" -- across hindsight-api-slim, hindsight-dev, and hindsight-embed, and clean up the resulting violations. These are now blocking: lint.sh auto-removes them and the verify-generated-files CI job fails on any leftover diff. Add an advisory dead-code scan for what the linter cannot see -- whole unused Python functions (vulture) and orphaned files/exports/dependencies in the control plane (knip): - scripts/hooks/check-unused.sh runs both locally - new non-blocking check-unused-code CI job surfaces findings on PRs - hindsight-control-plane/knip.json tunes out toolchain false positives vulture stays advisory because its function/argument heuristics false-positive on FastAPI/SQLAlchemy/Pydantic patterns; knip can be flipped to blocking once the control-plane dead code (PR #2135) lands. * chore(ci): make knip blocking on unused files/deps; remove dead deps #2135 deleted tooltip.tsx but left @radix-ui/react-tooltip in package.json, and react-chrono / three were never imported. Remove all three, and declare @radix-ui/react-visually-hidden (used in directive-detail-modal but unlisted). With the control-plane tree now clean, the check-unused-code job runs `knip --include files,dependencies,unlisted` as a BLOCKING step. vulture and knip's unused-exports check (the shadcn/ui surface is kept intentionally) stay advisory.	2026-06-12 11:06:31 +02:00
Ben	f5a6c300f1	feat(agent-framework): Hindsight memory for Microsoft Agent Framework (no MCP) (#1989) ... * feat(agent-framework): add Hindsight memory integration via context provider Persistent memory for Microsoft Agent Framework (the successor to Semantic Kernel) without MCP. HindsightProvider is a ContextProvider whose before_run recalls relevant memories and injects them into the agent's instructions, and whose after_run retains the conversation. Reuses the LlamaIndex integration's client/config pattern and the hindsight-client Python SDK. Targets the agent-framework-core 1.x before_run/after_run + SessionContext contract (verified against the installed package since the API has churned). 15 unit tests subclass the real ContextProvider so drift fails loudly, plus a gated e2e. Includes CI job, release + changelog + docs wiring, and an icon. * chore(agent-framework): refresh lock to agent-framework-core 1.8.1 (verified no API drift) * fix(agent-framework): drop unused per-op timeout constants TIMEOUT_RETAIN/TIMEOUT_RECALL/TIMEOUT_BANK were defined but never used: the hindsight-client SDK sets one timeout on the constructor and has no per-call timeout argument, so per-op values can't be wired in. Keep the single constructor-level TIMEOUT_DEFAULT and document why. Addresses review feedback.	2026-06-11 13:57:20 -04:00
Sanderhoff-alt	c96106cc01	chore: remove dead code and stale config (#2135) ... Remove unreferenced backend helpers, stale UI/docs components, and unused imports across the API, control plane, clients, and integrations. Drop obsolete consolidated-observation helpers and unused scoring code, clean orphaned React/docs components, and remove stale Radix dependencies. Align release scripts, Helm docs, lockfiles, generated clients, and current API examples with the package and endpoint surface still in use.	2026-06-11 17:12:03 +02:00
DK09876	91d767cdcb	feat(cursor): add Hindsight memory plugin for Cursor (#866) ... * feat(cursor): add Hindsight memory plugin for Cursor Adds a complete Cursor integration using the plugin architecture (hooks, skills, rules). Automatically recalls relevant memories before each prompt and retains conversation transcripts on task completion. Modeled after the claude-code integration with Cursor-specific adaptations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(cursor): add integration docs, blog post, and sidebar entry Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(cursor): clarify plugin vs MCP modes, add hook diagnostics - Add plugin-vs-MCP comparison table near top of integration doc - Add "Verifying Plugin Hooks" section with state file commands - Add troubleshooting note: visible tool calls = MCP, not plugin - Write last_retain.json state file in retain.py for diagnostics - Add mode: plugin and query_length to recall state file - Fix test_settings_file_loaded to isolate from user config Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cursor): install path, always-write diagnostics, Cloud snippets - Add mkdir -p before cp -r in all install examples (first-run fix) - Add "fully quit and reopen Cursor" note to all setup flows - Recall/retain hooks now write status on every invocation (success, empty, skipped, error) not just on success - Fix docs to show ~/.hindsight/cursor-state/ default path - Add concrete Hindsight Cloud config snippet to Quick Start - Add Cloud option to blog post setup section Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cursor): add session field to dynamic bank IDs, add changelog - Support "session" in dynamicBankGranularity for per-conversation banks - Add changelog page for cursor integration - Add test for session-based dynamic bank ID Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cursor): sync integration README with cookbook/blog setup guidance - Add mkdir -p for plugin install path - Add "fully quit and reopen Cursor" instruction - Show Cloud as Option A, local as Option B, daemon as Option C - Match the setup flow documented in the cookbook and blog Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(cursor): add pip/uvx installer, fix review findings - Add hindsight_cursor package with CLI `init` and `uninstall` commands - Add pyproject.toml for PyPI publishing via existing release pipeline - Update README install path: `pip install hindsight-cursor && hindsight-cursor init` - Fix rule/skill files to describe plugin behavior instead of MCP tools - Add diagnostics on get_api_url failure paths in both hooks - Remove missing assets/avatar.png reference from plugin manifest - Add Cloud token retrieval guidance (Settings > API Keys) - Add test_cli.py with 8 tests for init/uninstall commands Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cursor): daemon timeout, config defaults, full config docs - Set daemonIdleTimeout default to 300s (was 0/infinite with no cleanup hook) - Fix retainEveryNTurns fallback from 1 to 10 in retain.py - Fix DEFAULTS: hindsightApiUrl="" and bankId="cursor" to match settings.json - Document all config settings in README (was missing ~15 entries) - Fix pytest version discrepancy in pyproject.toml - Fix plugin.json author to "Vectorize" for consistency Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(cursor): streamline setup with init flags, add Docker instructions - Restructure Quick Start around Cloud vs Local as two clear paths - Use hindsight-cursor init --api-url/--api-token for one-command setup - Add Docker run command for users without a local Hindsight server - Remove separate "configure" step that contradicted init behavior Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cursor): replace beforeSubmitPrompt with sessionStart + MCP beforeSubmitPrompt does not support additionalContext in Cursor's hook system — the old recall.py was silently ignored. This rewrites the architecture to use Cursor's native mechanisms: - sessionStart hook for ambient project-level recall (supports additionalContext) - MCP integration for on-demand recall/retain/reflect tools mid-session - stop hook for auto-retain (unchanged, works correctly) Also fixes Python floor (3.9 -> 3.10, pytest 9 requires it) and updates docs/blog to match the new architecture. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cursor): workaround broken sessionStart additionalContext Cursor's sessionStart hook accepts additionalContext output but silently drops it before the agent's composer handle is ready — a race condition acknowledged by Cursor staff in 2026-04, still present in 3.6.31 (verified 2026-06-02 with a marker-emitting test hook). Without a workaround the plugin's "auto-recall memories at session start" feature silently does nothing in every install. Per Cursor staff guidance (Dean Rie, thread 158452), the documented escape hatch is to write a workspace .cursor/rules/<file>.mdc with alwaysApply: true — the rules engine injects those reliably. Plugin- local rules dirs (~/.cursor/plugins/local/...) are NOT reliable per thread 159101. Implementation: - scripts/lib/rules_file.py (new): owns the workaround. Three helpers: * rotate_session_rules() — deletes any prior rules file at the top of each sessionStart so an empty recall doesn't leave stale memories from a previous session. * write_session_rules() — writes the .mdc with alwaysApply: true, an HTML comment that explains what the file is and links to the Cursor bug, and the recalled memories inside a <hindsight_memories> block (same wrapper the broken native path used, so the static rules guidance is unchanged). * ensure_gitignored() — idempotently appends the file path to <workspace>/.gitignore when the workspace is a git repo. No-ops otherwise. Matches both /-anchored and bare relative forms so we don't double-add against an existing entry. - scripts/session_start.py: rotates at the top, writes the fallback file after recall succeeds, gates both behind config flags (useRulesFileFallback, appendToGitignore, both default True). Still emits additionalContext to stdout below — when Cursor fixes the upstream bug, dropping the workspace write is the only code change needed; the same plugin works on the native path with no protocol rev. - scripts/lib/config.py: two new config keys + HINDSIGHT_USE_RULES_ FILE_FALLBACK / HINDSIGHT_APPEND_TO_GITIGNORE env overrides. - rules/hindsight-memory.mdc: tells the agent where recalled memories now appear (the new .cursor/rules/hindsight-session.mdc file) and notes that the file is plugin-generated and safe to delete. - tests/test_rules_file.py: 18 tests pinning the on-disk shape: frontmatter, alwaysApply, bug link, rotation, idempotent gitignore with both anchor forms, falsy workspace handling, write-error degradation. Why this design (vs. alternatives): - Just shipping MCP-only and documenting the limitation would repeat the OpenAI Agents notebook-10 Pattern-1 failure mode: the agent has to choose to call recall, and small models reliably skip it. Auto- inject doesn't depend on tool-call choice. - Reverting to beforeSubmitPrompt would mean a recall per turn instead of per session, and Cursor staff have signalled additional_context on that hook is unimplemented (forum 150707). - The workspace file is the price of Cursor's bug being open with no ETA. Mitigations: auto-rotate, auto-gitignore, in-file explanatory comment, config opt-outs. Verification: - Full suite: 74 passed (56 prior + 18 new). - Smoke end-to-end against a fresh git repo: rules file written with correct frontmatter, .gitignore appended cleanly with both an explanatory comment and the path entry, no duplicate-add on re-run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(cursor): adopt requires_real_llm bucketing + live E2E + lockfile + docs Aligns cursor with the standing test-bucketing convention from PR #1469 ("Split test suite into deterministic mock and real LLM buckets") that the other eight Python integrations already follow. Changes: - pyproject.toml: register the `requires_real_llm` marker so the live E2E suite is selectable as a discrete bucket (and excluded from the deterministic CI path via `pytest -m "not requires_real_llm"`). Add hindsight-client as a dev dep — the E2E driver needs it to seed and verify banks; the runtime plugin scripts still use stdlib only. - tests/test_e2e.py (new): four-test gated suite that drives the actual hook scripts the way Cursor does — JSON on stdin, env vars for config — against a live Hindsight server. Covers: 1. session_start writes the rules-file workaround with recalled content, appends `.gitignore`, and emits the forward-compat `additionalContext` to stdout. 2. empty-bank case: hook succeeds without writing a rules file. 3. opt-out: `useRulesFileFallback=false` produces no `.cursor/` or `.gitignore` mutations even when recall surfaces content. 4. retain end-to-end: drives `retain.py` with a JSONL transcript (the on-disk shape Cursor actually emits, not an inline messages array), then verifies the bank holds the fact via direct recall. Two non-obvious fixtures the suite needs: - `HOME` / `CURSOR_PLUGIN_DATA` redirected to tmp so the test doesn't touch the developer's real `~/.hindsight/cursor.json` or state. - `HINDSIGHT_BANK_MISSION` overridden to a focused mission that aligns with the seeded fixtures — the production default mission is broad boilerplate, fine for real users but too diffuse to reliably surface targeted test content within a deadline. - `HINDSIGHT_RETAIN_EVERY_N_TURNS=1` because retain.py batches every N turns (10 by default) and a single-shot test only has one turn. - uv.lock: committing per the convention every other Python integration follows. 258 KB, 29 packages resolved, `uv lock --check` clean. - README.md: new "How session memory reaches the agent" section documenting why the plugin writes `<workspace>/.cursor/rules/ hindsight-session.mdc` (Cursor's native `additionalContext` channel is broken, forum thread 158452, still open in 3.6.31). Captures the empirically-verified behaviour: Cursor blocks prompt submission until sessionStart returns, so every new agent's first prompt has memories, the rules file is regenerated each session, and the file is auto-gitignored. Two new config knobs (`useRulesFileFallback`, `appendToGitignore`) added to the Session Recall table. Verification: - Deterministic bucket: 74 pass / 4 deselected (the new gated E2E). - Live bucket (HINDSIGHT_API_URL=http://127.0.0.1:8888): 4 pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cursor): default to hosted backend + give each retain a distinct document_id V2 audit (2026-06-02) caught two real bugs in cursor that were missed by the V1 pass: 1) Goal-5 (Default to Cloud) FAIL — settings.json shipped hindsightApiUrl='' and the daemon path treated empty as "fall back to local daemon at 127.0.0.1:9077". Users following the docs ("just enable the plugin") never reached the hosted backend without explicitly passing --api-url. Every other integration's empty-config path lands on https://api.hindsight.vectorize.io. 2) The retain path used document_id=session_id in full-session mode, which silently upserts the same Hindsight document on every retain. The audit's 5-turn distinct-fact driver exposed this as "5-turn cloud → 1 topic surfaced" — earlier turns got overwritten because each retain rewrote the single per-session document with whatever transcript snapshot was current. Both are addressed below; the live test suite still passes against the local server and the new deterministic tests pin the cloud-default resolution + the unique-document-id derivation. Changes: - scripts/lib/config.py — add ``DEFAULT_HINDSIGHT_API_URL`` constant (``https://api.hindsight.vectorize.io``). Add ``useLocalDaemon`` flag (default ``False``) so self-hosters can opt back into the auto-managed daemon path. New env override ``HINDSIGHT_USE_LOCAL_DAEMON``. - scripts/lib/daemon.py — rewrite ``get_api_url`` resolution: 1. Explicit ``hindsightApiUrl`` wins. 2. A locally-running server on the configured port is used (preserves the "developer already started a daemon" path). 3. ``useLocalDaemon=True`` AND ``allow_daemon_start=True`` (retain path) triggers the auto-managed daemon. Recall path never starts a daemon on its own. 4. Otherwise → ``DEFAULT_HINDSIGHT_API_URL``. A failed daemon-start under (3) also falls back here rather than hard-erroring, so the plugin keeps working when ``hindsight-embed`` isn't on PATH. - scripts/retain.py — every retain now derives ``document_id = f"{session_id}-{int(time.time() * 1000)}"`` regardless of retainMode. The chunked-vs-full-session distinction at the doc-id layer was always a misfeature; full-session mode now means "the transcript ingested per retain may span the whole session", not "every retain writes the same document". - tests/test_daemon.py (new) — pin the four-tier resolution + env override + the source-shape of retain.py's document_id derivation. Verification: - Deterministic bucket: 81 pass / 4 deselected (74 prior + 7 new). - Live bucket: 4 pass / 0 fail against 127.0.0.1:8888. - Manual smoke for empty-config → returns ``DEFAULT_HINDSIGHT_API_URL``. - Live server still resolves to ``http://127.0.0.1:8888`` when healthy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cursor): parse Cursor 3.x role-nested transcript format retain.py's read_transcript only recognized two transcript shapes: - Flat: {role, content} - Type-nested: {type: "user"|"assistant", message: {role, content}} Cursor 3.6.31 writes a third shape to its stop-hook transcript: {"role":"user","message":{"content":[ {"type":"text","text":"..."}, {"type":"tool_use","name":"...","input":{...}} ]}} Top-level has `role` (not `type`), and `content` lives under `message` as a list of typed blocks (not at the top level as a string). The old parser's two branches both missed every line: `entry.get("type")` was None and `"content" in entry` was False. read_transcript silently returned [] for every Cursor 3 transcript, and retain.py bailed with status=skipped reason=empty_transcript on every stop hook. Visible symptom: auto-retain silently stops working under Cursor 3 even though the stop hook fires correctly and transcript_path points at a real, populated file (verified by reading ~/Library/Application Support/Cursor/logs/.../cursor.hooks.*.log — the input JSON includes a valid transcript_path that the parser then ignores). End users see recall continue to work (sessionStart writes the rules-file workaround) but new turns never get retained. Fix: - Add _normalize_blocks_to_text to flatten typed-block lists to a single string, inlining a compact [tool_use:<name>] marker so downstream Answer:/Thought: handling still sees coherent structure. - Recognize the role-nested Cursor 3 shape explicitly. - Keep flat and type-nested handling intact. Verified end-to-end against a real Cursor 3.6.31 transcript captured from ~/.cursor/projects/.../agent-transcripts/<conv>/<conv>.jsonl: read_transcript now returns the 15 messages it should (1 user + 14 assistant turns) instead of 0. Regression tests (3 added): - test_read_transcript_parses_flat_format pins the flat shape. - test_read_transcript_parses_type_nested_format pins the type-nested shape. - test_read_transcript_parses_cursor3_role_nested_with_block_content is the regression: fails on the pre-fix parser (returns []), passes now. Also asserts the [tool_use:Shell] marker survives. 14/14 tests in test_hooks.py pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(docs): drop missing image refs in cursor blog post The 2026-04-03 cursor-persistent-memory blog references /img/blog/cursor-persistent-memory.png in both frontmatter and inline markdown, but the image was never added to the repo. build-docs fails MDX compilation with "Markdown image with URL /img/blog/cursor-persistent-memory.png couldn't be resolved to an existing local image file". Strip the two references so the post renders. The prose stands on its own without an illustration; an image can be added in a follow-up PR if/when one is produced. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(cursor): sync openapi.json with main Branch carried an older snapshot of hindsight-docs/static/openapi.json that pre-dated main's addition of the OperationProgress schema. check-openapi-compatibility flagged the missing 'progress' field on GET /v1/default/banks/{bank_id}/operations/{operation_id} as a backwards-incompatible removal. Re-checkout main's openapi.json onto the branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(cursor): drop cursor-persistent-memory blog post The blog post was added as marketing for the Cursor integration but the accompanying illustration was never produced. Earlier commit 0e4b2568 stripped the missing image references so build-docs would pass; user prefers the blog post itself be dropped from the integration PR and authored separately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(cursor): ruff format scripts + generate docs-skill changelog verify-generated-files CI flagged drift in three cursor scripts (scripts/lib/daemon.py, scripts/retain.py, scripts/session_start.py) and a missing skills/hindsight-docs/.../integrations/cursor.md. - scripts: applied ruff format/check (3 files reformatted, all checks pass). - generate-docs-skill.sh produced the integrations/cursor.md changelog mirror. Format-only + a generated file regeneration; no behaviour changes. All cursor tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: re-trigger CI A previous push to this branch silently did not trigger a pull_request event in GitHub Actions, leaving the PR without a CI run for the latest HEAD. Push an empty commit to force a new event. * ci: empty commit to attach pull_request CI check to the PR head (Previous pushes did not auto-trigger pull_request workflow events for reasons internal to GitHub Actions; manual workflow_dispatch runs passed green but their checks don't roll up onto the PR. Re-poking the head to surface the green state on the PR.) * ci: trailing newline to force CI retrigger * fix(cursor): address review — drop dead code, register changelog + gallery - Remove compose_recall_query / truncate_recall_query from scripts/lib/content.py (ported from openclaw but unused — cursor only recalls at sessionStart) and their test; slice_last_turns_by_user_boundary stays (used by retain.py). - Add cursor to the INTEGRATIONS map in generate_changelog.py so the release changelog step resolves the slug. - Add the integrations.json gallery entry + icon and rely on the existing docs-integrations/cursor.md so check-integrations.mjs passes. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Ben <ben.bartholomew@vectorize.io> Co-authored-by: DK09876 <dk09876@DK09876s-MacBook-Pro.local>	2026-06-09 09:50:22 -07:00
Ben	b0f86f9c0d	feat(obsidian): Hindsight plugin for Obsidian (#1941) ... * feat(obsidian): add Obsidian plugin integration Sync an Obsidian vault into a shared Hindsight bank and chat with an agent grounded on your notes (citations link back to the source note). Obsidian stays the source of truth: one-way sync, conversation memory off by default. - TS plugin (esbuild → main.js): requestUrl HTTP client, incremental sync engine (hash/mtime gate, upsert/delete/rename, reconcile + orphan prune), reflect-backed chat view with citations + reasoning, settings + commands. - One shared bank ("obsidian") across vaults; implicit scoping via auto tags (vault:, folder: ancestors, created:/updated: date buckets) so recall can scope by any combo from the UI or an automation. document_id is vault-prefixed to avoid cross-vault collisions. - Tests (vitest, mocked obsidian module): sync upsert/delete/rename/hash-gate, auto-scope tags, client request shapes, and the §0.5 guard (no conversation retain when the toggle is off). - Wiring: test-obsidian-integration CI job + aggregate gate, VALID_INTEGRATIONS, changelog generator, integrations.json + docs page + changelog page + icon. Out of scope for v1: rename-proof frontmatter identity; BRAT/community-store release-asset attachment (release-integration.yml only npm-publishes today). * feat(obsidian): scoped chat filters, retrieved-notes, debug logging, branding - Chat scope filters (vault + folder dropdowns above the ask bar) build tag_groups (all_strict) passed to reflect; folder tags are hierarchical. - "Notes retrieved" list + per-step reasoning: reflect's based_on omits document_ids, so harvest them from the recall/expand tool outputs (incl. nested observation source_facts). New reflect-util with a unit test. - Debug logging toggle: logs the reflect request (with scope) and the retrieved note ids to the console for verifying filters. - "New chat" view action + command to reset the conversation. - Branding: real Hindsight logo (favicon) embedded as a data URI for the ribbon, chat header, empty state, and tab icon (via an SVG <image>). * feat(obsidian): chat output extras — copy, snippet previews, wikilink resolution - "Copy" action under each answer. - "Notes retrieved" now shows the matched text snippet per note (from the recall/expand tool outputs + observation source_facts), so you can see why a note was pulled without opening it. New retrievedNotesDetailed() + test. - Answers render with the active note as sourcePath, so [[wikilinks]] resolve. * feat(obsidian): auto-grow chat composer + frontmatter/client edge tests The composer textarea now grows with multi-line input up to a 240px cap, then scrolls. Adds unit coverage for the two previously untested pure layers: frontmatter.normalizeNote (no/blocklist/inline-flow frontmatter, created/date precedence, scalar metadata, unterminated block) and client edge paths (transport rejection, reflect tag_groups-vs-tags branch, retain tag omission). * ci(obsidian): attach BRAT install assets to the GitHub release Obsidian plugins install from GitHub release assets (main.js, manifest.json, styles.css), not npm. The release-integration workflow only npm-published the package, leaving the plugin uninstallable. Add an obsidian-only step that creates/updates the release for the tag and uploads the three files (idempotent on re-run), and grant the job contents:write. * chore(obsidian): fix generated-files drift (prettier + docs-skill mirror) Run prettier over the integration (README.md table/emphasis formatting and the new frontmatter.spec.ts array wrapping) and regenerate the agent-skill changelog mirror that generate-docs-skill.sh produces. Resolves the verify-generated-files CI check. * feat(obsidian): persistent sync-status indicator in the status bar Background, edit-triggered sync previously ran silently — only the manual 'Sync vault now' surfaced a Notice. Add an always-visible status-bar item that shows synced/syncing/error state plus a live 'last synced x ago' time, notes the pending-edit count, and triggers a sync on click. All sync paths (reconcile, debounced flush, single-note ingest, delete, rename) route through it. Pure label/tooltip logic is unit-tested (9 cases). * feat(obsidian): mirror sync status in the chat header Surface the same sync state in the chat panel's header (right-aligned), reusing renderSyncStatus with no brand prefix since the Hindsight wordmark is already shown. The plugin pushes updates to any open chat view whenever sync state changes, and clicking the pill triggers a sync. * feat(obsidian): show note count + pending in the sync indicator Replace the bare check mark with the tracked-note count and either the pending-edit count or the last-sync time (e.g. '✓ 412 notes · 2m ago', '✓ 412 notes · 3 pending'). Tooltip carries the full breakdown. Count comes from the local sync index; singular/plural handled. * feat(obsidian): explicit refresh button for sync (spins while syncing) The sync status was clickable text with no obvious affordance. Split it into an informational status label plus a dedicated refresh icon button (in both the chat header and the status bar) that triggers a sync on click and spins while a sync is in flight. * docs(obsidian): document the sync-status indicator in the README	2026-06-08 16:43:06 -04:00
Derek Bouius	6dc56498ce	feat(integrations): add oh-my-openagent (OMO) integration (#2018) ... * feat(integrations): add oh-my-openagent (OMO) integration Cloud-first Hindsight memory integration for the OMO agent harness. Provides automatic recall/retain via lifecycle hooks with support for both Hindsight Cloud (api.hindsight.vectorize.io) and self-hosted. - 5 lifecycle hooks: SessionStart, UserPromptSubmit, Stop, SubagentStop, SessionEnd - Always-apply rule for memory guidance - Config hierarchy: settings.json → ~/.hindsight/omo.json → HINDSIGHT_* env vars - Bearer token auth for cloud mode (hsk_* keys) - Interactive demo script for local dev testing - Full test suite (29 tests) * chore(ci): add OMO integration test job - Add test-omo-integration job to test.yml (pip + pytest pattern) - Add detect-changes output and path filter for omo * fix: apply lint formatting to OMO integration files * fix: fix demo importlib.util import and add mkdir to setup instructions - Import importlib.util explicitly (importlib alone doesn't expose .util) - Add mkdir -p for ~/.omo/hooks and .omo/rules in README copy instructions - Default demo API URL to localhost:8888 to match Docker compose port * docs: rewrite OMO README with cloud-first setup as default Simplify setup to 4 numbered steps with cloud as the primary path. Move self-hosted to an optional section. Add testing section. Clarify that rules are per-project while hooks/scripts are global. * chore: add omo to VALID_INTEGRATIONS in release script * fix: address release-blocking issues for OMO integration - Remove pyproject.toml (causes release workflow to mis-classify omo as a Python package and fail uv build). Move pytest config to pytest.ini. - Add IntegrationMeta entry in generate_changelog.py - Add integrations.json entry with internal doc link - Add docs page at docs-integrations/omo.md - Add omo.svg icon - Add "version": "0.1.0" to settings.json	2026-06-08 16:21:49 -04:00
Ben	66e58a23af	feat(cline): Hindsight memory integration via lifecycle hooks (#1956) ... * feat(cline): add Hindsight memory integration via lifecycle hooks (no MCP) Gives Cline persistent long-term memory without MCP, using its lifecycle hooks. TaskStart/UserPromptSubmit recall relevant memories and inject them via contextModification; TaskComplete/TaskCancel retain the task transcript. Cline hands hooks no transcript, so prompts are accumulated per-task in local state and retained at task end. Reuses the agent-agnostic core from the Codex integration (HTTP client, config, bank derivation, state, content helpers). Includes an install.py, 34 tests, CI job, and release/docs wiring. * refactor(cline): typed HindsightClineConfig instead of raw dict (review) Address the code-review should-fix: replace the raw `config` dict (known, enumerated keys) with a HindsightClineConfig dataclass per SKILL §5. load_config maps the camelCase settings.json/env keys onto snake_case fields; consumers read typed attributes. Also tighten type hints flagged in the review: ensure_bank_mission (client: HindsightClient, debug_fn: Callable[..., None] | None), _cast_env(typ: type) -> Any, debug_log(... ) -> None, parse_hook_input (raw: dict[str, Any]), and client _headers/_request dict parameterization. retain_metadata stays a dict (genuinely user-defined dynamic keys). * refactor(cline): parameterize retain() metadata dict type	2026-06-08 16:09:46 -04:00
DK09876	394d66e607	feat(integrations): add Haystack integration (#1256) ... * feat(integrations): add Haystack integration for persistent agent memory Add hindsight-haystack package providing Haystack Tool instances backed by Hindsight's retain/recall/reflect APIs. Uses async client methods with event-loop-safe sync wrapper to work correctly inside Haystack's agent runtime. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(haystack): use persistent event loop for async client calls aiohttp binds its session to the creating event loop, so asyncio.run() (which creates/destroys a loop per call) breaks on sequential calls. Switch to a persistent daemon-thread event loop with run_coroutine_threadsafe. Also removes unused per-operation timeout constants and adds _run_sync tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(haystack): add HindsightToolset with auto-recall/retain, fix review issues - Add HindsightToolset(Toolset) with auto_recall and auto_retain flags that automatically inject recalled memories into the system prompt before each turn and retain user/assistant messages after each turn - Fix _ensure_bank to retry on transient errors instead of permanently disabling bank creation - Fix reflect_on_memory to return structured_output JSON when response_schema is set - Truncate error messages to avoid dumping raw HTTP responses to agents - Extract _build_backend_kwargs() and _build_tools() as shared helpers - Add 20 new tests (60 -> 80 total) covering toolset, auto-recall, auto-retain, structured output, and bank creation retry Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(haystack): address review round 2 — max_recall_results, role metadata, _run_sync cleanup - Add max_recall_results param to HindsightToolset (default 10) to cap auto-recall prompt injection size, matching Pydantic AI pattern - Auto-retain now includes role + source metadata on messages, matching LlamaIndex's metadata pattern for distinguishable conversation turns - _recall_for_prompt now calls the API directly with result cap instead of going through the formatted string from recall_memory - Serialize/deserialize max_recall_results in to_dict/from_dict - Add tests for max_recall_results and role metadata (82 total) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(haystack): default to Cloud without configure(); add gated E2E + bucketing - resolve_client() now falls back to DEFAULT_HINDSIGHT_API_URL and the HINDSIGHT_API_KEY env var when configure() was never called (it previously raised "No Hindsight API URL configured"). Updated the unit test to assert the cloud-default + env-key behavior. Satisfies the "default to Cloud" goal. - Add a gated tests/test_e2e.py (retain/recall/reflect tools against a live Hindsight server), marked requires_real_llm; register the marker in pyproject; the test-haystack-integration CI job now runs the deterministic bucket (-m "not requires_real_llm"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(haystack): close owned clients at exit; run E2E client I/O on the bridge loop The tools run async client calls on a persistent background event loop. aiohttp sessions bound to that loop were never closed, surfacing as "Unclosed client session/connector" warnings. Track module-owned Hindsight clients (those created when the caller didn't pass client=) and close them on the loop via an atexit hook, then stop the loop. The live E2E now performs all client I/O through that same loop (acreate_bank/adelete_bank/aclose via _run_sync) and logs cleanup failures instead of swallowing them — zero unclosed-connector warnings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(haystack): sync openapi.json with main Branch carried an older snapshot of hindsight-docs/static/openapi.json that pre-dated main's addition of OperationProgress. Re-checkout main's openapi.json so check-openapi-compatibility passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(haystack): strip api_key from to_dict() so it doesn't leak to YAML _build_backend_kwargs was emitting the api_key in the serializable dict that to_dict() returns. Haystack pipelines get dumped to YAML for inspection, checkpointing, and sharing — a serialized key leaks into every dump. Reviewer (benfrank241) flagged this on #1256. Drop the api_key from the serialized backend_kwargs. resolve_client() already reads HINDSIGHT_API_KEY from the env var as a final fallback, so a redeployed pipeline picks the key back up from the host's environment rather than from the YAML. The test_tools_round_trip_serialization_with_client test previously asserted the leak — flipped it to assert the key is NOT present and added a json.dumps probe asserting the literal key value also doesn't appear under any other field name. Pre-fix, the test fails: AssertionError: api_key must not appear in serialized backend_kwargs — would leak to YAML pipeline dumps assert 'api_key' not in {'api_key': 'client-key', ...} 86/86 tests pass post-fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: re-trigger CI A previous push to this branch silently did not trigger a pull_request event in GitHub Actions, leaving the PR without a CI run for the latest HEAD. Push an empty commit to force a new event. * ci: empty commit to attach pull_request CI check to the PR head (Previous pushes did not auto-trigger pull_request workflow events for reasons internal to GitHub Actions; manual workflow_dispatch runs passed green but their checks don't roll up onto the PR. Re-poking the head to surface the green state on the PR.) * ci: trailing newline to force CI retrigger * fix(haystack): register in changelog/gallery + docs page + tidy tools Review follow-ups for the Haystack integration: 1. Add haystack to the INTEGRATIONS map in generate_changelog.py so the release script's changelog step resolves the slug (was missing, which would fail the release). 2. Add the integrations.json gallery entry, a doc page at docs-integrations/haystack.md, and an icon — required by check-integrations.mjs (forward: entry needs a doc page; reverse: a released integration must appear in the gallery). 3. Drop the inaccurate 'Raises: HindsightError' clause from create_hindsight_tools — resolution always succeeds (URL defaults to Cloud) so it never raises; the error type stays exported as the conventional public catch type. 4. Replace the _TOOL_DEFS dict-of-3-tuples with a frozen _ToolDef dataclass and drop the redundant method-name field (it equalled the dict key). * fix(haystack): use official Haystack logo for gallery icon Replace the placeholder glyph with the real deepset Haystack mark (teal #0EAF9C rounded square + white symbol), extracted as vector from deepset's own website source (deepset-ai/haystack-home site-logo partial). --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: DK09876 <dk09876@DK09876s-MacBook-Pro.local>	2026-06-08 14:52:38 -04:00
Salem Korayem	dbfe83a2ae	feat(integrations): add Cursor CLI integration (#1975) ... * Add .worktrees to .gitignore * feat(integrations): add Cursor CLI integration Four Cursor CLI hooks keep memory in sync automatically: - sessionStart — health check + daemon pre-start - beforeSubmitPrompt — recall relevant memories and inject as `additional_context` - stop — read the on-disk transcript, retain the conversation (fire-and-forget, async retain) - preCompact — surface which memories will survive the next context-window compaction The integration follows the same shape as the existing codex integration (Python hook scripts reading JSON from stdin, writing JSON to stdout) and the same config schema, so users with a codex setup can drop in cursor-cli with no new concepts. Project resolution prefers Cursor's `CURSOR_PROJECT_DIR` env var (common field in the hook runtime), then `workspace_roots[0]`, then `cwd` — avoiding the codex `session` default granularity since Cursor's `stop` hook is fire-and-forget. CI: - new `test-cursor-cli-integration` job in .github/workflows/test.yml - `cursor-cli` added to VALID_INTEGRATIONS in scripts/release-integration.sh Docs: - new top-level hindsight-integrations/README.md indexing every integration, with cursor-cli highlighted under "Coding agents & CLIs" 72 tests cover the four hook scripts, the bank-id derivation, the HTTP client, the cursor transcript reader, and the chunked-retain logic. All pass under `python -m pytest tests/ -v`. Ruff and shellcheck are clean. Co-Authored-By: opencode minimax-m3 high <noreply@anomaly.co> * fix(cursor-cli): derive bank id in session_start banner The session banner used a static `config.get("bankId") or "cursor-cli"` fallback, while recall.py / retain.py / pre_compact.py all called `derive_bank_id(hook_input, config)`. With `dynamicBankId: true` and `dynamicBankGranularity: ["project"]`, the banner reported the static default ("cursor-cli") while the other hooks targeted the derived bank (e.g. "korayem-cli-agents-hindsight"). Users and agents that trusted the banner then called `hindsight memory reflect cursor-cli` against an empty bank, while the hooks themselves were writing to the correct one. Mirror recall.py's pattern: import derive_bank_id, call it with the parsed hook_input, surface the resolved bank in debug logs so users can confirm parity with the other hooks. Tests cover all four acceptance criteria: - dynamicBankId true → derived bank in banner - dynamicBankId false + explicit bankId → static bank in banner - HINDSIGHT_BANK_ID env override → resolved through config loader - regression: previous tests still pass Co-Authored-By: opencode minimax-m3 high <noreply@anomaly.co> * refactor(cursor-cli): align implementation with codex/claude-code The cursor-cli implementation shipped several invented surfaces and patterns that drifted from the codex/claude-code reference. This commit removes the inventions and brings the script bodies back to near-parity with the references so future divergence stands out in a diff. Removed — invented user-facing surfaces: - session_start.py: the "Hindsight memory integration is active for this session. Bank: <id>" additional_context banner. The references' sessionStart is fire-and-forget with no additional_context. Banner output is where the bank-id display-mismatch bug lived, and the only consumer that "saw" the banner was the agent, which never asked for it. - pre_compact.py and its TestPreCompactHook class entirely. preCompact is observational in Cursor's spec — it cannot influence the compaction itself. The actual mechanism that preserves memory through compaction is the beforeSubmitPrompt recall that fires after compaction finishes. The "Hindsight preserved N memories" user_message was invented value with no reference equivalent. Restored — patterns from codex that were dropped: - session_start.py: debug_log for "Hindsight not running" path (was changed to a noisier print). - recall.py: import time, import write_state, LAST_RECALL_STATE const, and the write_state(...) block that drops the most recent recall payload to ~/.hindsight/cursor-cli/state/. Dead code in codex, but matching the reference for now keeps the diff focused on actual cursor-specific differences. - recall.py: `prompt = (hook_input.get("prompt") or hook_input.get("user_prompt") or "")` — kept the user_prompt fallback for defense in depth. - retain.py: "Exit codes" section in the docstring and the inline comments / blank lines that codex uses for readability. - lib/__init__.py: removed the cursor-cli-specific docstring to match codex's empty file. Kept — true Cursor-specific differences (justify in PR review): - session_start.py / retain.py / recall.py: docstrings mention Cursor, not Codex. - debug log key: conversation_id (Cursor's term) instead of session_id (codex's term). Cursor's `stop` hook carries conversation_id; codex's carries session_id. - session_id fallback chain: hook_input.get("conversation_id") or hook_input.get("session_id") or "unknown" — accepts both payload shapes. - template_vars includes conversation_id alongside session_id so retainTags / retainMetadata templates work either way. - retainTags default: ["{conversation_id}"] (codex is empty list) — convention is to tag the document with the source-of-truth id. - retainContext default: "cursor-cli" (was "codex"). - agentName default: "cursor-cli" (was "codex"). - bankMission / retainMission defaults: full text matching the Cursor CLI audience (codex leaves them empty). - USER_AGENT: "hindsight-cursor-cli/<version>" (was "hindsight-codex/<version>"). - PROFILE_NAME: "cursor-cli" (was "codex") in daemon.py — controls the hindsight-embed profile name. - bank resolution: CURSOR_PROJECT_DIR env var → workspace_roots[0] → cwd (codex only uses cwd). Cursor sets CURSOR_PROJECT_DIR on every hook. - VALID_FIELDS in bank.py adds "gitProject" as an alias for the project resolution. - recall output schema: Cursor's beforeSubmitPrompt wants {continue, additional_context}, not codex's {hookSpecificOutput: {hookEventName, additionalContext}}. Tests: - Removed TestSessionStartHook tests that asserted on the deleted banner. - Removed TestPreCompactHook class entirely. - test_session_start.test_no_output_when_server_reachable is the new mirror of codex's expectations: sessionStart emits nothing on stdout. Net: -296 lines, 68 tests passing, ruff + shellcheck clean. Co-Authored-By: opencode minimax-m3 high <noreply@anomaly.co> * fix(cursor-cli): flush memory at session end Add a Cursor sessionEnd hook that forces a final retain so short sessions are stored even when retainEveryNTurns skips per-turn retention. Also remove stale preCompact/banner docs and align the daemon idle-timeout fallback with the shipped config. Co-Authored-By: OpenAI GPT-5 Codex High <noreply@openai.com> * fix(cursor-cli): register integration in changelog generator cursor-cli was added to VALID_INTEGRATIONS and CI but missing from the INTEGRATIONS map in generate_changelog.py, which the release script reads when generating the changelog entry. Without it, the release would fail at the changelog step. --------- Co-authored-by: opencode minimax-m3 high <noreply@anomaly.co> Co-authored-by: OpenAI GPT-5 Codex High <noreply@openai.com>	2026-06-08 11:27:40 -04:00
DK09876	b708302187	feat(integrations): add Superagent safety middleware (#1128) ... * feat(integrations): add Superagent safety middleware for Hindsight memory Adds hindsight-superagent integration that wraps Hindsight retain/recall/reflect with Superagent Guard (prompt injection detection) and Redact (PII removal). - SafeHindsight middleware class with configurable guard + redact pipeline - Global configure() / per-instance config with env var fallbacks - CI job and release script entry - 54 unit tests + 10 e2e tests (all passing) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(superagent): default to Hindsight Cloud URL when no URL is configured Matches the pattern used by all other integrations — falls back to https://api.hindsight.vectorize.io instead of erroring. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(superagent): require superagent_api_key, update README defaults - resolve_safety_client now raises HindsightError if no API key is provided, matching actual safety-agent behavior (create_client() requires a key) - README: document superagent_api_key as required, hindsight_api_url defaults to Hindsight Cloud URL Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(superagent): disable broken fallback by default, add env var key resolution The safety-agent SDK's default fallback endpoint (superagent.sh/api/fallback) returns a 307 redirect that httpx doesn't follow for POST requests, causing all guard() calls to fail on cold starts. This change: - Defaults enable_fallback=False so the primary Cloud Run endpoint is used directly (60s timeout is sufficient) - Exposes enable_fallback and fallback_timeout in config/SafeHindsight for users who want to opt back in - Adds os.environ fallback for SUPERAGENT_API_KEY in resolve_safety_client so it works without calling configure() first - Fixes e2e redact test that was blocked by guard on recall query Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(superagent): require explicit guard_model, increase client timeout Superagent's hosted guard endpoints (Cloud Run Ollama) currently serve empty model lists, making the default superagent/guard-1.7b unusable. Update all examples to use guard_model="openai/gpt-4o-mini" and document the self-hosting alternative. Increase Hindsight client timeout from 30s to 120s to accommodate reflect's server-side LLM call. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(superagent): disable guard on retain, fix e2e tests for OpenAI guard General-purpose LLMs (gpt-4o-mini) over-classify PII content as security violations, blocking retain before redact runs. Disable guard on retain in all examples and default test helper. Fix e2e tests to use explicit guard_model and OpenAI provider instead of broken hosted endpoints. All 10 e2e tests now pass against live APIs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(superagent): switch guard/redact model to gpt-4.1-nano gpt-4.1-nano correctly distinguishes prompt injection from legitimate content (including PII), eliminating the need to disable guard on retain. Re-enables full Guard → Redact → Retain pipeline. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(superagent): add typed return values and py.typed marker Replace Any return types on recall() and reflect() with RecallResponse and ReflectResponse from hindsight-client. Add py.typed marker for PEP 561 type checker support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style(superagent): fix ruff line-length formatting in _client.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(superagent): add enable_redact_on_recall + lazy SafetyClient Two gaps surfaced by code review: 1. `enable_redact_on_recall` was missing. Guard was configurable on every op (retain/recall/reflect) but redact was wired only into retain. A memory like "John's SSN is 123-45-6789" stored from a non-safe path would come back verbatim through `recall()`. Added the option to redact each result's text on the read path. Default is False rather than True because every result triggers its own redact call (N results → N round-trips), unlike retain which is always 1 call. Callers who care about read-path PII opt in. 2. SafetyClient was resolved eagerly in `SafeHindsight.__init__`, raising if SUPERAGENT_API_KEY was missing even when every safety hook was disabled. Moved resolution behind a `_get_safety()` getter that constructs on first guard/redact call. Explicit `safety_client=` still wins and is stored directly, so the "supply your own client" path is unchanged. Tests: 62 pass (56 original + 3 redact-on-recall + 3 lazy-resolution). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(superagent): address review-agent findings — env fallback, race, concurrency, scope Addresses the 1 blocker + 8 should-fixes from the review-agent pass. Blocker: - resolve_hindsight_client() now reads HINDSIGHT_API_KEY env directly. The base hindsight_client.Hindsight doesn't fall back to the env var on its own, so the constructor-only path (no prior configure() call) was silently dropping the key. Fix: read os.environ.get(HINDSIGHT_API_KEY_ENV) as the third precedence step after explicit api_key and config.api_key. Should-fix: - Safety client config is now snapshotted at __init__ via snapshot_safety_config() and built lazily via build_safety_client() on first guard/redact call. A later configure() call cannot silently change what an already-constructed SafeHindsight will see. - Redact-on-recall (and the new retain_batch / redact-on-reflect paths) run under an asyncio.Semaphore bounded by `redact_concurrency` (default 5). Wide recalls no longer stampede the Superagent rate limit. - Added `enable_redact_on_reflect` — reflect's synthesised text is also LLM output derived from possibly-PII memories, so the same opt-in shape as redact-on-recall applies. Off by default. - Added `SafeHindsight.retain_batch(items)` wrapping aretain_batch with per-item guard + redact under the concurrency cap. Any item's GuardBlocked aborts the whole batch before any store. - Added `aclose()` + async context manager. Closes owned underlying clients (Hindsight, SafetyClient) but leaves caller-passed clients alone. - Pinned safety-agent to >=0.1.5,<0.2.0 and hindsight-client to >=0.4.0,<1.0 so a pre-1.0 minor upstream bump can't silently change the API. - Switched config-resolution precedence from `or`-chains to `_kw()` helper using `is not None`. Explicit empty list / 0 / False kwargs now override global config instead of being treated as "unset". - Tag merge in retain() now uses `dict.fromkeys(...)` instead of `set(...)` so order is preserved (call-tags first, then default tags, deduped). E2E tests: - TestE2EGuard block tests now actually assert that Guard blocks (with 3 retries to absorb model variance). Previously they silently passed if Guard returned "allow" — defeating the purpose. - Same fix for the bare-Superagent `test_guard_blocks_injection`. - Added E2E coverage for redact-on-recall, redact-on-reflect, retain_batch, and global-config-vs-per-instance-override precedence. Unit tests: - 15 new unit tests across 5 new test classes: TestSafetyConfigSnapshot, TestRedactConcurrencyCap, TestRedactOnReflect, TestRetainBatch, TestLifecycle, TestTagMergeOrder, TestEnvFallback. All passing; total 77 unit tests up from 62. README updated with new options, lazy-resolution clarification, batch and lifecycle sections. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(superagent): round-3 review-agent findings — E2E rigor, validation, observability Addresses 5 should-fixes, 2 nits, and 1 question from the round-3 review pass. E2E rigor (should-fix): - test_redact_strips_pii_from_stored_memory: previously passed silently if recall returned no results. Now polls via _recall_until_nonempty() so empty results fail the test. Same polling helper applied to every E2E that retains-then-recalls (redact-on-recall, redact-on-reflect, retain_batch, config precedence) so a non-indexed retain no longer silently turns an assertion into a non-assertion. - test_recall_clean_query / test_reflect_clean_query: now assert the stored memory's content actually surfaces in recall/reflect output, not just that the response shape is valid. - cleanup_banks fixture: extended suffix list to include every test class's bank (-redact-recall, -redact-reflect, -batch, -precedence) so the new E2Es don't leak banks. Code correctness (should-fix): - Validate safety_concurrency >= 1 in both SafeHindsight.__init__ and configure() — asyncio.Semaphore(0) would deadlock _redact_many() and the guard-batching path in retain_batch. Raises ValueError early. - Expand retain_batch to pass through every per-item field Hindsight.aretain_batch supports (metadata, document_id, entities, observation_scopes, strategy) and accept top-level document_id / document_tags kwargs. Previous narrow surface forced callers to fall back to the raw client for any of those fields. Naming + docs (nit): - Rename `redact_concurrency` → `safety_concurrency`. The same cap bounds both redact-many and the guard-batching loop in retain_batch, so the name "redact-only" was misleading. Public kwarg, config field, and internal attr all renamed; tests + README updated. - Align README requirements list with pyproject bounds: safety-agent >=0.1.5,<0.2.0 and hindsight-client >=0.4.0,<1.0. Observability (question → resolved): - Add `on_guard(scope, result)` callback invoked for every guard verdict (pass and block) so callers can log/observe non-block decisions without changing core flow. Scope is one of "retain"/"recall"/"reflect"/ "retain_batch". Sync or async callable accepted; async is awaited. Callback fires before GuardBlockedError raises on block, preserving observability for the block path too. Tests added: 12 new across TestSafetyConcurrencyValidation, TestOnGuardCallback, TestRetainBatchFieldPassthrough. Total: 87 unit tests (was 77 → +10 net after the renames). All passing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(superagent): round-4 polish — update_mode, retain_async, on_guard error containment Addresses 2 should-fixes and 1 nit from the round-4 review. retain_batch surface (should-fix): - Added "update_mode" to _BATCH_PASSTHROUGH_KEYS. Hindsight.aretain_batch reads item.get("update_mode") per item, so dropping it forced callers who wanted controlled upserts to fall back to the raw client. - Added top-level `retain_async: bool = False` kwarg. Hindsight supports background-processing the batch after the safety pipeline is done; the wrapper now exposes that knob. Guard + Redact still run synchronously before the call returns — only the underlying store is deferred. When the default False is used, the kwarg isn't forwarded so the client's own default wins. on_guard error containment (nit): - The callback is documented as observability "without changing the core flow," but a raised exception inside the callback previously took down the memory op. Wrapped the call in try/except with a WARNING log so observability failures stay observable instead of fatal. The log includes the scope and the exception type/message so an operator can spot a misbehaving callback. Block-path behaviour is unaffected — if Guard says block, GuardBlockedError still raises after the callback attempt. Tests: 93 unit tests pass (was 87; +6 net). New cases cover update_mode per-item passthrough, retain_async forwarding (and the don't-forward-on- default case), sync and async on_guard exception containment, and that a callback exception doesn't suppress a real block verdict. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(superagent): make E2E suite merge-clean — natural-language anchors, lifecycle Live E2E run with the Superagent key surfaced two reproducible failures plus aiohttp connector leaks. Fixes: 1. test_redact_strips_pii_from_stored_memory — previously queried for "What is Bob's contact info?", which deterministically misses after redact strips Bob's name and email from the stored content. A first attempt added a synthetic canary ("redact-pii-canary alpha bravo") alongside the PII, but Hindsight's fact extraction treats opaque identifier phrases as noise and drops them, so the canary itself didn't surface in recall either. Fix is to use natural-language project context ("Project Phoenix client onboarding") as the anchor — fact extraction materialises it as a real fact, vector search handles it cleanly, and the assertion verifies (a) the anchor is retrievable and (b) the PII is absent from the result. 2. test_redact_on_reflect_scrubs_synthesis — same root cause, same fix. Anchor on "Project Tango payment notes" instead of a synthetic canary or PII-laden query. The credit card sits secondary in the memory but isn't relied on for retrieval. 3. Unclosed aiohttp ClientSession / TCPConnector warnings — every test instantiated a SafeHindsight via _make_client() but never called aclose(). Added an autouse fixture that tracks every safe created via _make_client() and aclose()s them on test teardown. Idempotent; exceptions during cleanup are swallowed so they don't mask the test's own result. Result: 14/14 E2E pass in 74s (down from 127s due to fewer rerun attempts on the previously-failing paths) with no unclosed-session warnings. 93/93 unit tests still pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style(superagent): apply ruff format (fixes verify-generated-files CI) Same formatter drift as the other integrations: ruff check passed but ruff format (run by the verify-generated-files job via scripts/hooks/lint.sh) reflows manually-wrapped lines that fit within 120 cols. Formatting only — no behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(superagent): bucket E2E as requires_real_llm; PR CI runs deterministic only Mark the live E2E suite (real Superagent Guard/Redact + OpenAI + Hindsight) with a module-level requires_real_llm marker, registered in pyproject, mirroring the core test split from #1469. The test-superagent-integration job now runs -m "not requires_real_llm" (deterministic bucket: 93 tests); the real-LLM bucket (14 tests) is selectable via -m requires_real_llm. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(superagent): add deterministic retain->recall->reflect round-trip (mock bucket) Drives SafeHindsight end to end with mocked Hindsight + Superagent clients, asserting guard/redact-then-forward across all three ops — the in-CI / no-keys analog of the live round-trip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(superagent): remove dead resolve_safety_client resolve_safety_client at _client.py:87 was a convenience wrapper around snapshot_safety_config + build_safety_client, with a docstring saying "kept for backwards compatibility — combines snapshot + build into one call". As reviewer (benfrank241) flagged on PR #1128: there's nothing to be backwards compatible with — this is a new package. The middleware (SafeHindsight) uses snapshot_safety_config + build_safety_client directly. The function had no real callers. Drop: - The function itself from _client.py. - TestResolveSafetyClient class from tests/test_client.py (its 6 tests only exercised the dead wrapper). - The corresponding import. test_middleware.py::test_unsafe_path_does_not_resolve_safety_client stays — the "resolve" there is a generic verb describing whether the middleware needs to construct a safety client at all, not a reference to the deleted function. That test still verifies the lazy-construction semantics it always did. Test suite: 88 passed, 14 skipped (down from 88+6 = 94 passed; the 6 removed were the wrapper-only tests). Middleware coverage unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(superagent): ruff format/check fixes for verify-generated-files CI verify-generated-files flagged _client.py drift (2 trailing blank lines after the resolve_safety_client removal) plus 3 additional small lint findings ruff check could autofix. Running the full ruff format + ruff check --fix pipeline brings the diff to zero against what CI expects. No behaviour changes; format-only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: DK09876 <dk09876@DK09876s-MacBook-Pro.local>	2026-06-05 16:45:34 -04:00
DK09876	a933a417cd	feat(claude-agent-sdk): add Claude Agent SDK integration (#1582) ... * feat(claude-agent-sdk): add Claude Agent SDK integration with memory tools and hooks Adds hindsight-claude-agent-sdk package providing: - In-process MCP server with retain, recall, and reflect tools - Automatic memory hooks (auto-recall on prompt, auto-retain on stop) - Tool output retention via PostToolUse hooks - Global configuration and per-call overrides - 74 unit tests, CI job, and release script entry - Cookbook recipe for docs site Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(claude-agent-sdk): default to Cloud without configure(); add gated E2E + bucketing - resolve_client() now falls back to DEFAULT_HINDSIGHT_API_URL and the HINDSIGHT_API_KEY env var when configure() was never called (it previously raised). Updated the tools + hooks unit tests to assert the cloud-default + env-key behavior. Satisfies the "default to Cloud" goal for both create_hindsight_tools and create_memory_hooks. - Add a gated tests/test_e2e.py (retain/recall/reflect MCP tools against a live Hindsight server, stdlib urllib health check — no requests dep), marked requires_real_llm; register the marker; the test-claude-agent-sdk-integration CI job now runs the deterministic bucket (-m "not requires_real_llm"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(claude-agent-sdk): assert create_memory_hooks reads HINDSIGHT_API_KEY from env Mirrors the tools env-key test so hook construction's cloud-default + env-key path is covered, not just the no-key default. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: DK09876 <dk09876@DK09876s-MacBook-Pro.local>	2026-06-05 16:23:15 -04:00
Chris Bartholomew	d7a3aa5269	feat(llm): provider prompt-prefix caching — retain + consolidation + reflect (bank-agnostic, default-on) (#1936) ... * feat(gemini): add context-cache foundation (GeminiCacheManager + opt-in call() arg) Wraps the google-genai SDK's CachedContent API so callers can reuse a stable (system_instruction + response_schema) prefix across many requests. Cached input tokens are billed at a fraction of the standard input rate, which makes workloads with a fixed-prefix / small-user-message shape — fact extraction, structured tagging, classification — far cheaper to run. This PR is foundation-only: no caller is wired up yet. Default behaviour for every existing path is unchanged because `cached_content_name` defaults to `None` and the cache manager is never instantiated until a follow-up wires it in. What's here ----------- - `gemini_cache.GeminiCacheManager`: per-process map of prefix fingerprint → CachedContent resource name. Thread-safe via a single asyncio.Lock. Refreshes proactively at TTL minus a safety margin. Stable fingerprint normalisation strips auto-generated Pydantic schema titles so dynamically-built schema classes with identical shape hash to the same key (relevant for callers that rebuild the schema class on every request). - `gemini_llm.GeminiLLM.call(cached_content_name=...)`: new optional arg. When set, the SDK config drops `system_instruction` and `response_schema` (those live in the cache) and instead passes `cached_content` to GenerateContentConfig. When unset, behaviour is byte-identical to before. - `tests/test_gemini_cache.py`: 10 unit tests covering fingerprint stability, dict/list/Pydantic schema cases, get_or_create caching/recreate, "minimum token count" soft-fallback, transient SDK error soft-fallback, failed-create-doesn't-poison-cache, and the TTL refresh boundary. Failure handling ---------------- - Gemini rejects creates whose prefix is below the model's minimum cacheable size with a "minimum"-style error message. The manager catches this, logs at DEBUG, and returns None so the caller transparently falls back to a non-cached call. - Any other SDK error is logged at ERROR and also returns None — a bad create never crashes a request. Callers are required to treat None as "cache unavailable, use the normal path". Not in this PR -------------- - Wiring this into the fact-extraction pipeline (or any other caller) - A metric for cached-token volume Both will come in a focused follow-up so the foundation can land and be reviewed independently. * feat(gemini): wire retain fact-extraction to context cache; surface cached + thoughts tokens Follow-on to the foundation commit on this branch — without this, the cache manager is unreachable and the metric ignores half the cost surface. This commit makes the change actually do something when the flag is flipped on. What lands ---------- 1. Retain fact-extraction (engine/retain/fact_extraction.py) opts into the cache. The system prompt and response schema are fingerprinted and reused across calls; the user message is the only variable part on the wire. A cache lookup failure or "prefix too small" response from Gemini transparently falls back to the existing uncached path — caching is a soft optimisation, never a blocker. 2. New top-level flag HINDSIGHT_API_LLM_GEMINI_PROMPT_CACHE_ENABLED (also exposed as ``llm_gemini_prompt_cache_enabled`` on HindsightConfig). Defaults to False so upgrade-and-do-nothing is a no-op. Flipping to True opts every Gemini caller (currently only retain) into context caching. 3. Two new metrics: - hindsight.llm.tokens.cached_input — subset of input tokens billed at the cached rate. Lets dashboards split cache-hit vs cache-miss volume independently of total throughput. - hindsight.llm.tokens.thoughts — reasoning tokens emitted by Gemini 2.5+. Billed at the output rate by the provider but invisible to candidates_token_count, so absent from output-token dashboards today. Surfacing this is required for honest cost attribution. 4. Provider plumbing: GeminiLLM gains a ``gemini_prompt_cache_enabled`` kwarg and a ``get_or_create_cached_prefix(...)`` accessor that lazy- builds a GeminiCacheManager on first opt-in. LLMProvider / create_llm_provider / ConfiguredLLMProvider pass the flag through the standard plumbing alongside the existing safety_settings. Verification ------------ - ``uv run ruff check`` — clean - ``uv run pytest tests/test_gemini_cache.py`` — 12 tests including two new integration tests that pin (a) flag-off → cache manager never built, and (b) flag-on → manager lazy-built, second lookup served from in-memory cache, no extra SDK call. - ``uv run pytest tests/test_gemini_safety_settings.py`` — 13 tests still green (no signature drift; the NoOp metrics collector was updated alongside the real one). Rollout ------- - Land this commit. With the flag default-off, behaviour is identical to today: cache code paths exist but are never reached. - Flip the flag per-env. The metric goes non-zero on cached_input within a few calls. - Watch hindsight.llm.tokens.cached_input vs hindsight.llm.tokens.input to confirm cache-hit rate. What's deliberately NOT in this PR ---------------------------------- - Extending caching to other Gemini callers (reflect tool-call, consolidation). Same mechanism applies — copy two lines from the retain path. Leave for a follow-up so this lands in one focused PR. - Cross-pod cache sharing. Each pod warms its own cache. The cost of one extra full-price call per pod per fingerprint per TTL window is negligible relative to steady-state savings. * feat(gemini): extend context caching to the tool-calling reflect loop Adds caching support to the agentic tool-loop path. The reflect agent's ``system_prompt + tools`` is stable for the duration of a single reflect (and across reflects against the same bank), so caching them once and reusing the cache name across every iteration of the loop collapses the dominant input cost — the prefix repeated on every turn. Mechanism --------- 1. ``GeminiCacheManager.fingerprint(...)`` now accepts ``tools`` and includes the OpenAI-style tool list in the hash. A loop that swaps a tool gets a fresh cache automatically; a loop that doesn't, hits the cache deterministically. The tool list is serialised with sort_keys so upstream dict-reordering doesn't cause phantom cache misses. 2. ``GeminiCacheManager.get_or_create(...)`` accepts ``tools`` and converts the OpenAI-style entries into Gemini ``Tool`` / ``FunctionDeclaration`` shapes inside ``CreateCachedContentConfig``. The cached prefix now holds system_instruction + tools, so the subsequent ``call_with_tools(cached_content_name=...)`` invocation skips resending both. 3. ``GeminiLLM.call_with_tools(...)`` gains ``cached_content_name``. When set, ``system_instruction`` and ``tools`` are dropped from the per-request config (the SDK rejects re-sending them alongside ``cached_content``); ``tool_config`` (mode / allowed_function_names) stays per-request as it must. 4. ``GeminiLLM.get_or_create_cached_prefix(...)`` accepts ``tools`` and forwards them to the cache manager. 5. ``reflect/agent.py:run_reflect_agent`` looks up (or creates) the cached prefix ONCE per reflect — right after the ``system_prompt`` and ``tools`` are built — and reuses the returned cache name across every iteration of the agentic loop. The lookup is wrapped in a try/except so a cache-side failure can never block a reflect. 6. ``call_with_tools`` now extracts ``cached_content_token_count`` and ``thoughts_token_count`` from ``usage_metadata`` and threads them through ``metrics.record_llm_call`` — same as ``call()`` already does. Without this the new ``hindsight.llm.tokens.cached_input`` and ``hindsight.llm.tokens.thoughts`` counters would never report the reflect-side share of cached/thinking tokens. Tests (3 new on top of the 12 from earlier on this branch) ---------------------------------------------------------- - ``test_fingerprint_changes_with_tools``: adding a tool changes the fingerprint so a loop that adds a tool gets a fresh cache. - ``test_fingerprint_stable_under_dict_reordering``: dict-key order in the OpenAI-style tools list does NOT change the fingerprint. - ``test_get_or_create_passes_tools_to_create``: the ``caches.create`` call actually receives the tools in its config — without this the cache would silently lack the tool definitions and the first ``call_with_tools(cached_content_name=...)`` would 400. Verification ------------ - ``uv run pytest tests/test_gemini_cache.py tests/test_gemini_safety_settings.py`` → 28/28 pass (15 cache + 13 safety; the safety-settings suite doubles as regression on the ``call_with_tools`` signature change). - ``uv run ruff check`` on changed files — clean. Behavioural envelope -------------------- - Flag still defaults False — no caller is opted in by default. - When flag is True, both ``retain_extract_facts`` (from the earlier commit on this branch) and ``reflect_tool_call`` opt in. - A cache-side failure (transient SDK error, prefix too small, manager uninstantiated) returns None and the caller proceeds uncached. There is no path by which caching can break reflect or retain. * fix(gemini): make explicit prompt caching actually work end-to-end The caching paths could never produce a cache hit: - CreateCachedContentConfig was given response_schema/response_mime_type, which the google-genai SDK forbids (extra_forbidden) — so every cache create raised and soft-fell-back to an uncached call. Cache only holds system_instruction (+ tools); response_schema is a generation-time constraint and stays on the per-request GenerateContentConfig. - call() dropped response_schema when a cache was in use (assuming the schema lived in the cache — impossible). Keep it on the request; only system_instruction moves into the cache. Structured output is preserved. - cached_content_name was plumbed into the leaf GeminiLLM.call / call_with_tools but NOT through the LLMProvider wrapper, so the real call path raised "unexpected keyword argument 'cached_content_name'". Thread it through both wrappers, forwarding only when set (other providers untouched). With these, retain extraction caches the ~1.7k-token prefix at ~90%. * feat(gemini): cache consolidation prefix + gate reflect cache to auto turns - Consolidation: split the batch prompt into a stable system instruction (mission + rules + decision guide + output format) and a per-batch user message (facts + existing observations + capacity note). The system prefix is byte-identical across batches in a run, so it is cached and reused; the variable data and the per-batch response_schema stay out of the cached surface so it never busts. Measures ~30-40% cached/input per batch (the remainder is irreducible per-batch data). - Reflect: Gemini rejects cached_content alongside a per-request tool_config ("CachedContent can not be used with ... tool_config"). The forced-retrieval iterations set tool_config, so only the `auto` iterations can reference the cache. Gate cached_content_name on tool_choice == "auto"; forced iterations send the prefix inline. * test(gemini): per-operation cached-ratio test + consolidation split coverage - New tests/test_gemini_implicit_cache_ratio.py: measures cached/input token ratio per operation (retain, reflect, consolidation) against real Gemini via the LLM-request tracer. Dual mode: default records the implicit-cache baseline (~0% for this access pattern); HINDSIGHT_GEMINI_EXPLICIT_CACHE=1 asserts the explicit cache engages (cached_tokens > 0, per-op ratio floor). Gated behind HINDSIGHT_RUN_GEMINI_EVALS=1 + a Gemini key. - test_consolidation.py: unit test for the system/user prompt split (cacheable byte-stable prefix; data only in the user message). Fix the inline mock LLM callbacks to read facts from the user message(s) rather than messages[0], now that the stable instructions are a separate system message. * perf(consolidation): move stable observation-format note into cached prefix The "## INPUT FORMAT" boilerplate (the explanation of the observation JSON shape: id/text/proof_count/occurred_*/source_memories) was re-sent in every per-batch user message. It's stable, so move it into the cached system prefix (build_consolidation_system_prompt); the per-batch user message now carries only the variable facts + observations data. Lifts the cached/input ratio a couple of points without changing what the model sees. * feat(gemini): make cached prefix bank-agnostic (mission → user message) The retain and consolidation system prompts embedded the per-bank mission, so each distinct mission produced a different cache fingerprint → one CachedContent per bank. With many banks/missions that multiplies create + storage cost and cached-object count, and makes default-on uneconomical. Move the mission out of the cached prefix into the per-request user message: - retain: _build_extraction_prompt_and_schema now returns a bank-agnostic prompt; the mission rides in the user message via _retain_mission_preamble(). - consolidation: build_consolidation_system_prompt drops the mission param; the mission moves into build_consolidation_input (the user message). Result: the cached prefix is identical across all banks, so a single shared CachedContent serves every bank — cardinality drops from O(missions) to O(1) per operation, and the cost-inversion for many-low-volume-bank workloads goes away. Behavioral note: the mission now appears in the user turn rather than the system prompt. Validate mission-adherence against the accuracy benchmarks before flipping the global default on. Tests updated to assert the new location + cross-bank prefix sharing. * test(retain): assert different missions yield one shared cache prefix Extend the mission-relocation test to prove the payoff directly: two banks with different retain missions produce a byte-identical system prompt → the same cache fingerprint → a single shared CachedContent instead of one per mission. * test(retain): cacheable prefix invariant to per-bank free-text (concise/verbose) Parametrized over the concise and verbose modes: the cached system prompt must be byte-identical regardless of the retain mission (any value, incl. JSON/unicode/ long text) and custom instructions, so per-bank free-text can never fragment the shared Gemini cache. Structural toggles (causal/labels/language) are intentionally out of scope — they legitimately partition the cache via the fingerprint. * refactor(llm): make prompt-prefix caching a provider-interface feature Hoist caching out of Gemini-specific duck-typing into the LLMInterface contract, mirroring supports_batch_api(): - LLMInterface.supports_prompt_caching() -> bool (default False) and get_or_create_cached_prefix(...) -> str | None (default None), with docs on how explicit-cache (Gemini handle), automatic-cache (OpenAI), and inline-marker (Anthropic cache_control) providers each map onto the hook. - call()/call_with_tools() gain a provider-neutral cached_prefix handle (renamed from the Gemini-flavoured cached_content_name); the wrapper forwards it only when set so non-caching providers' signatures are untouched. - GeminiLLM implements supports_prompt_caching(); the retain/consolidation/reflect call sites gate on it instead of hasattr(). The engine already decides WHAT is cacheable (bank-agnostic system prefix), so a new provider only implements HOW — e.g. OpenAI can benefit with no code (stable leading prefix is auto-cached) or a thin override. * docs(models): add per-provider capability table (batch API, prompt caching) Adds a "Provider Capabilities" table to the LLM section of the models page showing which providers support the Batch API (OpenAI/Groq/Fireworks) and explicit prompt-prefix caching (Gemini/Vertex via CachedContent), with notes on OpenAI's automatic prefix caching and the bank-agnostic shared-cache design. Includes the regenerated skills/hindsight-docs mirror. * docs(models): drive provider capability table from llmProviders.json Replace the hand-written capability table with a data-driven one so adding a provider stays a single-file edit. The capability flags (batchApi, promptCaching) live in llmProviders.json — the existing single source of truth for the provider grid and default-models table — and a new LLMProviderCapabilities component (plus a matching renderer in generate-docs-skill.sh) renders them. Tool-calling dropped (not differentiating here). Keep flags aligned with supports_batch_api() / supports_prompt_caching() on the provider classes. * feat(llm): generic, default-on prompt caching knob Rename the Gemini-specific opt-in flag to a provider-agnostic, default-on knob, modelled on HINDSIGHT_API_RETAIN_BATCH_ENABLED: - HINDSIGHT_API_LLM_GEMINI_PROMPT_CACHE_ENABLED → HINDSIGHT_API_LLM_PROMPT_CACHE_ENABLED (config field llm_gemini_prompt_cache_enabled → llm_prompt_cache_enabled, kwarg gemini_prompt_cache_enabled → prompt_cache_enabled), single global knob (not per-op). - DEFAULT_LLM_PROMPT_CACHE_ENABLED = True. Safe to default on: the cached prefix is bank-agnostic (one shared cache) and creation soft-fails to an uncached call, so it never breaks a request. Providers that don't implement caching ignore the flag. - Resolve the flag for every provider (drop the gemini/vertexai restriction) so any future provider that implements supports_prompt_caching() picks it up. Docs: models page now says "on by default; disable with HINDSIGHT_API_LLM_PROMPT_CACHE_ENABLED=false". The per-operation ratio test sets the flag explicitly in both modes since the default is now on. Includes the regenerated skills/hindsight-docs mirror. * fix(gemini): fall back to uncached on a cached-request 400 A 400 from a generate request that references a CachedContent (expired/deleted cache, cross-project mismatch, cache+tool_config incompatibility, ...) was treated as a generic retryable error: the same cached request was retried, 400'd again, and the whole operation failed. The soft-fallback only covered cache *creation*, not the call that *uses* the cache. Now, on the first 400 while a cache is in use, call()/call_with_tools(): - drop the cache and rebuild the request inline (re-send system prefix + schema/ tools) so the request still succeeds, - invalidate the dead cache name (GeminiCacheManager.invalidate) so the next operation recreates it instead of reusing the bad name, - retry immediately (no backoff — it's a config switch, not a transient error). If the uncached retry also 400s it's a genuine bad request and errors normally. Supporting fix: system_instruction is now ALWAYS captured from the messages (it was skipped when cached), so the fallback has the prefix to inline; the config builder still omits it from the request while the cache carries it. New unit test covers the 400 → uncached-retry → invalidate path. Cached success path unchanged (real Gemini retain still 90.8%). * fix(gemini): bound the cache-create call with a timeout get_or_create holds the manager lock across the caches.create network call, which correctly dedups concurrent callers (a 10-chunk retain batch produces exactly one create, not ten). But with no timeout, a hung create would block every waiting chunk indefinitely. Wrap the create in asyncio.wait_for (30s default, configurable via create_timeout_seconds); on timeout it soft-fails to None and callers proceed uncached instead of stalling the batch. Unit test covers the timeout path. * style: ruff-format the prompt-cache config line (fixes verify-generated-files) * test: fix consolidation-scope-parallelism mock + metrics counter count - test_consolidation_scope_parallelism.py: the inline mock read facts from messages[0], which is now the (cached) system message after the consolidation prompt split — read the user message(s) instead. - test_metrics.py: mock_meter provided 5 counter mocks but MetricsCollector now creates 7 (the cached_input + thoughts counters), so create_counter.side_effect ran out (StopIteration at setup). Bump both fixtures to 7. --------- Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>	2026-06-04 14:11:49 +02:00
Nicolò Boschi	613a699e9f	fix(consolidation): eliminate duplicate observations via interleave dedup recall (#1907) ... Round-robin interleave fusion for consolidation dedup recall (guarantees the semantic-#1 'twin' a slot so the LLM updates instead of duplicating), unified 'reranking' strategy param (cross_encoder/rrf/interleave), case-sensitive exact-dup guard, obs-dedup tool + benchmark wired into the perf dashboard (English dataset). Near-dup observation rate 4% -> 0% on the English hermes transcript (1/10 and 1/4), coverage 89% -> 94%, no false merges.	2026-06-03 17:37:57 +02:00
Ben	c032a74f17	feat(google-adk): add Hindsight integration for Google ADK (#1862) ... * feat(google-adk): add Hindsight integration for Google ADK Implements google.adk.memory.BaseMemoryService so Runner-driven agents get persistent long-term memory automatically: - HindsightMemoryService — retain on session end, recall on search_memory, with per-(app_name, user_id) bank scoping via a configurable template - create_hindsight_tools — ADK FunctionTool wrappers for explicit hindsight_retain / hindsight_recall / hindsight_reflect 49/49 tests pass. CI job, release script, and changelog generator wired up. Docs page + integrations.json + banner + sidebar entry added. * feat(google-adk): add ADK icon from adk.dev * test(google-adk): add end-to-end smoke script with real Gemini Runner Exercises both integration patterns against the dev cloud: - Phase 1: HindsightMemoryService (automatic memory) — Runner saves session A via add_session_to_memory; session B's agent calls load_memory which routes through search_memory and gets the facts back. - Phase 2: create_hindsight_tools (explicit) — agent calls hindsight_retain directly in session C; session D's agent calls hindsight_recall. Both phases pass live against api.dev.hindsight.vectorize.io with gemini-2.0-flash. * fix(google-adk): apply repo ruff format to smoke_runner.py	2026-06-01 13:39:43 -04:00