234 Commits

Author	SHA1	Message	Date
Nicolò Boschi	1d5f852809	test(system): blackbox system-test suite driven only through the public API (#4212) ... * test(system): blackbox system-test suite driven only through the public API The api-slim suite is ~500 files, each covering one mechanism, which leaves composition bugs — consolidation wiping facts, a delta refresh missing a backdated window, a transfer dropping evidence — with nothing watching them. This adds the layer that does: tests that drive a real `hindsight-api` process over HTTP through the published Python client, with no engine access and no SQL. Determinism comes from a stub server implementing the OpenAI chat-completions/embeddings APIs and a Cohere-compatible rerank endpoint. The server under test is pointed at it with ordinary environment variables (`HINDSIGHT_API_LLM_BASE_URL`, `..._EMBEDDINGS_OPENAI_BASE_URL`, `..._RERANKER_SILICONFLOW_BASE_URL`), so no production code changes are needed and the real provider transport — the OpenAI client, JSON repair, retries, structured output — is exercised for real rather than replaced by a fake. Design decisions worth knowing: - Rules match on a named pipeline step, not the request body. Nothing on the wire identifies the caller (the json_schema name is the constant "response", and the soft json_object path sends no schema at all), so a step is a short anchor phrase from its prompt — owned by `steps.py` so a prompt edit is one line, not thirty red tests. - An unmatched call fails the test and prints the rule to paste in. `MockLLM` synthesizes plausible facts from its input when it doesn't recognize a call, which is why tests using it pass without proving anything. - Requests are validated strictly. A fake you control drifts permissive, and the provider bugs that have actually hurt here (Bedrock rejecting response_format, Azure 400ing on prompt_cache_key) are all "the provider refused our request". - Background work stays on. Consolidation runs in the worker after retain returns; tests wait for it via the operations API rather than disabling it, because that asynchronous half is where the composition bugs live. - The server runs from a scratch dir holding an empty `.env`, since a discovered `.env` deliberately overrides the ambient environment (#2961) and would otherwise silently replace the whole test configuration. CI (`test-system`) needs no provider secrets, so unlike every `test-api` job it also runs on fork PRs — and with embeddings and reranking both stubbed, nothing loads sentence-transformers, so it skips torch and the HuggingFace cache too. * test(system): pin the whole recall payload, not just a keyword With the LLM, embedder and reranker all stubbed, a recall is a pure function of its input — so assert it as one. Ranking order, the rendered fact text, the document/chunk identity composite, the temporal fields, the empty envelope sections, and each of the four scores are now pinned. The three retrieval components are reproducible to the bit across runs; only `final` drifts (~1e-9), because it folds in recency measured against wall-clock now, so it gets a tolerance and a comment saying why. The previous assertion ("Berlin appears somewhere") passed just as happily with fusion inverted, the reranker contributing nothing, or the temporal fields silently stopping being parsed. * ci: skip the two unconditional jobs for a system-tests-only change Iterating on hindsight-system-tests/ costs ~6 minutes of CI against a 36-second test. Of the 96 jobs, 91 already skip for a change confined to that package; the remainder were build-docs (2.2 min) and verify-generated-files (3.6 min), both deliberately unconditional. Both now hang off a new `outside-system-tests` filter rather than a positive one, so they still run for every other change in the repo and step aside only for a PR touching nothing but the system-test package — which no generated file is produced from and the docs site never reads. The filter is a lone negation on purpose: paths-filter builds one matcher per pattern and ORs them, so the natural-looking ['**', '!dir/**'] pair matches every file and leaves the filter permanently true. Verified against picomatch directly before committing.	2026-09-08 14:12:12 +02:00
Nicolò Boschi	5f9bff9056	fix(docker): drop curl from the API runtime images (#4202) ... * fix(docker): drop curl from the API runtime images curl's only in-container consumer was the readiness loop in start-all.sh; there is no HEALTHCHECK instruction anywhere. It is also the sole reverse-dependency of libcurl4t64, which brings libssh2-1t64, so one line in each install list accounted for nine HIGH findings - all status=affected with no Debian fix published, so `apt-get upgrade` could not clear them and not shipping the package was the only remediation. Trivy 0.74.0 HIGH+CRITICAL, on locally built slim images: api-only 3C / 60H -> 3C / 51H standalone 3C / 60H -> 3C / 51H with exactly the curl, libcurl4t64 and libssh2-1t64 findings removed and nothing new. Replace it with http_probe, which reproduces `curl -sf` WITHOUT -L rather than approximating it. The distinction matters: curl does not follow redirects unless asked, so a 302 is a completed transfer and succeeds regardless of what it points at. urllib.request.urlopen follows it and raises on a 404 behind it, which would report a healthy service that redirects as "not ready". http_probe uses http.client and tests `status < 400` itself, bypassing urllib's redirect handler, and carries userinfo through as Basic auth the way curl does. Verified equivalent to `curl -sf` on 2xx, 3xx-to-good, 3xx-to-bad, 4xx, 5xx, query strings, userinfo auth and connection refused. Exit codes are not reproduced (curl's 22 and 7 become 1); every call site tests zero/non-zero. One deliberate difference, since it is a change and not a translation: curl was called with --connect-timeout, which caps only the connection phase, and the API health loop passed no timeout at all - so a server that accepted a connection and never answered hung the probe forever. The timeout now covers the whole request. There is no wget fallback. BusyBox wget cannot reproduce these semantics (no --max-redirect, so it always follows), and it is not needed: every image that probes anything is Python-based. cp-only, the one image with neither, performs no probe at all and dropped curl in #4197. Missing python3 now fails loudly at startup instead of degrading into a readiness loop that can never succeed. Closes #4198 * refactor(docker): move the readiness probe into hindsight_api.http_probe The first version of this probe was Python embedded in a shell string inside start-all.sh. That was a bad shape for code encoding rules this fiddly: every quote had to survive two levels of escaping, ruff and ty never saw it, and it could only be exercised through the shell. Move it to hindsight_api/http_probe.py, shipped with the code and covered by tests/test_http_probe.py, which pins each case to what `curl -sf` does for the same response. start-all.sh keeps a three-line wrapper that shells out to `python3 -m hindsight_api.http_probe`. hindsight-admin was the obvious home and is the wrong one: it takes 5.1s to start in the built image, against 0.028s for bare stdlib, because it pulls in the CLI and everything behind it. The readiness loop polls once per second, so importing the API to ask whether the API is up would break the loop it drives. This module imports stdlib only; measured 0.035s per probe in the image. `hindsight_api/__init__` is cheap by design and has to stay that way for this to hold - its docstring already says so. The shell test drops to checking the wiring, since the semantics now have a real home, and skips when the package is not importable: test-start-all.sh also runs in CI from a bare checkout with no virtualenv. Reformatting by `ruff format` on first contact is the point - the embedded version could never have received it. * refactor(probe): make the readiness probe its own package, isolated from the API hindsight_api.http_probe was the wrong home. The probe answers "is an API process up?", and living inside the package it probes invited exactly the coupling that would break it: an import of the engine or the config would put API startup cost - and API startup side effects - on a loop that runs once a second. Move it to hindsight_probe, a sibling top-level package in the same distribution. Its dependencies are now explicit by construction: none. It imports the standard library and nothing else. Packaging alone does not enforce that. Both packages install into the same virtualenv, so `import hindsight_api` from the probe would still resolve at runtime. So the rule is a test, not a convention: test_imports_nothing_but_the_standard_library imports the package in a clean subprocess and asserts that nothing outside sys.stdlib_module_names was pulled in. Adding `import hindsight_api` to the probe fails it with the offending name. The audit ignores _sysconfigdata_*, a platform-specific stdlib internal whose name embeds the build triple and so is absent from stdlib_module_names everywhere. Both Dockerfiles now copy the package; the api-builder previously copied only hindsight_api, so the first build without this shipped an image whose probe could not import. That surfaced as require_http_probe_runtime failing at startup with a clear message rather than a readiness loop that could never succeed, which is what that guard is for. Verified in the built Linux image: no non-stdlib imports, hindsight_api never loaded, 0.036s per probe, and the full end-to-end boot still reaches "Hindsight is running" with /health answering 200. * refactor(probe): keep the readiness probe inside hindsight_api Reverts the separate hindsight_probe package. It was justified on a bad measurement: an earlier cold-cache timing suggested `import hindsight_api` cost ~0.12s against ~0.03s for a standalone package. Measured properly, warm, in the built image, they are the same - ~0.03s each - and importing hindsight_api pulls in zero third-party modules. Its PEP 562 lazy-attribute design already does the work the split was meant to do, so the split bought nothing and cost a second top-level package, four pyproject entries and a COPY in each Dockerfile. What was worth keeping is the enforcement, which is orthogonal to where the module lives. test_imports_nothing_heavy imports the probe in a clean subprocess and asserts it pulled in no third-party package and nothing from hindsight_api.engine, .api or .config. Adding `from hindsight_api.engine import memory_engine` to the probe fails it with 43 packages named, numpy, sqlalchemy and asyncpg among them - which is the failure mode the rule exists to prevent. Verified in the built image: no third-party or engine imports, 0.034s per import, probe wiring works, curl absent.	2026-09-08 11:11:04 +02:00
Nicolò Boschi	42d4de25cc	ci: build the control plane on Node 24, matching the image (#4194) ... The Docker cp-builder stage moved to node:24-slim, but the two CI jobs that build the control plane directly on the runner still provisioned Node 20, which reached end-of-life on 2026-04-30. That is both an EOL toolchain and a version skew: CI would validate and publish a build produced on a different major than the one the shipped image builds with. - test.yml build-control-plane: runs the CP unit tests, i18n parity check, next build, standalone verification and server smoke test - release.yml release-control-plane: builds and publishes @vectorize-io/hindsight-control-plane to npm Neither package declares an `engines` floor, and next@16.2.11 requires >=20.9.0, so Node 24 clears both. Other Node 20 jobs are deliberately left alone: the TypeScript client jobs (build/test-typescript-client, -oracle, -deno) and test-hindsight-all exercise the published SDK, where pinning an older major is lower-bound coverage rather than an oversight, and changing it should be a separate decision.	2026-09-07 13:46:34 +02:00
Nicolò Boschi	945b741448	fix(openclaw): fail the smoke test on any hindsight doctor diagnostic (#4190) ... The smoke test's doctor assertion matched a word list — `hindsight.*(fail|error|not loaded)` — and openclaw 2026.9.x blocks a non-bundled plugin's typed hooks with a message containing none of those words: - hindsight-openclaw: typed hook "before_prompt_build" blocked because non-bundled plugins must set ...hooks.allowConversationAccess=true In that state the plugin still logs "agent hooks registered" and doctor still lists it as loaded, so nothing else in the script tells it apart from a healthy install — a build that never recalls or retains would have passed as green. (Not a live bug today: the setup wizard sets allowConversationAccess. The point is that the check could not have caught it if the wizard ever stopped.) Extract the logic into assert_doctor_clean() and fail closed: any diagnostic naming this plugin fails, whatever its wording. Unrelated bundled-plugin diagnostics and the expected memory-slot deselection notice are tolerated exactly as before. Since this assertion silently failed once, it gets its own fixtures from real captured doctor output, run in CI ahead of the smoke test.	2026-09-07 13:30:10 +02:00
Chris Bartholomew	7cc51b8796	ci: run the core suite when the workspace lockfile or root manifest changes (#4089) ... `uv.lock` and `pyproject.toml` at the repository root determine which dependency versions the API is built and tested against. Neither appears in any path filter, so a pull request that changes only one of them resolves `core` to false and every Python job is skipped. The result is a green check set that proves nothing. A dependency bump touching only the workspace lockfile currently reports success having run `detect-changes`, `verify-generated-files` and `build-docs`, with `test-api` and the rest of the suite skipped. Nothing observes that the new versions still work. The workflow already treats the lockfile as environment-determining — it is part of the `test-api` venv cache key: key: ${{ runner.os }}-venv-test-api-${{ hashFiles('uv.lock', ...) }} So the cache is invalidated by a file that cannot trigger the job whose cache it keys. Both patterns are deliberately bare, matching only the root files. A glob such as `**/uv.lock` would match the ~52 integration lockfiles under `hindsight-integrations/` and run the full core suite for every sample-project bump. Verified with picomatch: `uv.lock` matches `uv.lock` alone and none of `hindsight-integrations/*/uv.lock`. Cost: a root-lockfile-only change now runs the 30 jobs gated on `core`, where it previously ran three. That is the intended trade — those changes are exactly the ones no other filter covers.	2026-09-03 15:30:08 -04:00
Nicolò Boschi	614bfc96df	feat: run Hindsight on free-threaded CPython 3.14 (-py3.14t image) (#4037) ... * test(api): guard against silently losing free-threading A free-threaded CPython re-enables the GIL the moment it imports a C extension that has not declared `Py_MOD_GIL_NOT_USED`, and says so only with a RuntimeWarning. One new module-scope import therefore reverts the whole server to single-threaded execution while every existing test still passes and the process still serves traffic -- which is exactly how the three imports fixed in the previous commit went unnoticed. Adds a subprocess probe that imports the full API surface and asserts the GIL is still disabled, plus a second case pinning the diagnostic form: PYTHONWARNINGS="error:The global interpreter lock:RuntimeWarning" which turns the warning fatal at the offending import (naming the module and the import chain) while leaving unrelated RuntimeWarnings alone. The probe runs in a subprocess because the GIL can only be re-enabled once per interpreter, so an import already done in the pytest parent would mask a regression. Skipped unless `Py_GIL_DISABLED`, so it is inert on the 3.11 matrix and only bites on a python3.14t job. Verified both ways on python3.14t: passes on the fixed tree, and a negative control that adds `import psycopg2` to the probe fails with the module named in the assertion message. * feat(api): fail loudly when a free-threaded build loses the GIL A free-threaded CPython re-enables the GIL for the whole process the moment it imports a C extension that has not declared `Py_MOD_GIL_NOT_USED`, and says so only with a RuntimeWarning. Nothing crashes and nothing degrades visibly: the server starts, serves traffic, and passes its tests, having quietly reverted to single-threaded execution. One new module-scope import is enough. Adds `hindsight_api/_free_threading.py` and calls it eagerly from the package `__init__`, next to `apply_default_thread_limits()` and for the same class of reason: both configure how the process executes, and both are worthless once the libraries they govern have loaded. `HINDSIGHT_API_FREE_THREADING` selects the mode: strict (default on a free-threaded build) — the GIL re-enable warning becomes an exception, so the offending import raises with the module and full import chain in the traceback. A GIL already on at startup raises. warn — log and continue, for bringing up a deployment whose dependencies are not all ready. off — no guard; used by the migration subprocess, which imports psycopg2 on purpose. Every mode is a no-op on a normal build, so this is inert on 3.11. * ci(api): run the test suite on free-threaded CPython 3.14 Adds a `test-api (free-threaded 3.14)` job plus the two pieces of packaging it needs, so a regression that silently re-enables the GIL fails CI instead of quietly costing the deployment its parallelism. The job asserts free-threading before running anything -- a bad build or an already-taken GIL shows up as its own red step rather than as a mass of confusing downstream failures -- and runs pytest under PYTHONWARNINGS="error:The global interpreter lock:RuntimeWarning" so a regression fails at the offending import with the module named. It is deliberately NOT gated on `has_secrets`. It uses the mock LLM, so it needs no provider credentials and therefore also runs on fork PRs, which skip every secret-gated test-api job today. Packaging: * `overrides-freethreaded.txt` drops orjson. PEP 508 has no marker for "free-threaded build", so a never-true marker is the mechanism; the alternative is forking pyproject.toml for one interpreter. * `scripts/ci/build-freethreaded-quicktok.sh` builds quicktok with pybind11>=2.13 and `py::mod_gil_not_used()`. Building the published sdist unmodified is not enough: the extension would not declare free-threading support and would re-enable the GIL on import. Delete this step once the change is upstream and released. local-ml is excluded from the job. Importing `sentence_transformers` re-enables the GIL, so a process that loads the local models cannot stay free-threaded (torch, tokenizers, safetensors and transformers are each fine on their own -- measured, not assumed). `tests/conftest.py` now skips the `embeddings` and `cross_encoder` fixtures when that stack is absent, instead of collapsing every DB-backed test into a misleading "sentence-transformers is required for LocalSTEmbeddings" ImportError. On 3.11, where local-ml is installed, nothing changes. Measured on python3.14t (Linux/aarch64, Postgres 18 + pgvector, mock LLM): the non-LLM suite is 5954 passed / 1719 skipped. Of the residue, the migration-test errors are this harness missing the `embedded-db` extra (pg0), which the CI job installs; the remaining ~28 failures are not yet attributed to the interpreter and need a same-container 3.11 control run before any are called real. NOT yet validated: the job's `uv pip install --group dev` and quicktok script invocation exactly as written. The quicktok patch-and-build was verified by hand (producing a cp314t wheel that imports GIL-free and tokenizes correctly), but the container host died mid-run before the scripted forms were exercised end to end. Refs: initiative kp-ac23cdb434a144bf947371b6e6e4f5e8 * ci(api): drop the quicktok wheel build from the free-threaded job quicktok had no free-threaded wheel, so the free-threaded CI job patched and built one (pybind11>=2.13 plus py::mod_gil_not_used()) before it could run anything. #4022 moved token counting to toktok-rs, which publishes cp3XXt wheels and declares gil_used = false, so none of that is needed: a plain `uv pip install` from PyPI now yields a working free-threaded install. Removes the build step and scripts/ci/build-freethreaded-quicktok.sh. overrides-freethreaded.txt keeps only orjson, which still publishes no cp3XXt wheel and whose build script refuses free-threading outright. * fix(api): make the shared caches and budget manager loop-agnostic Two process-wide singletons held an `asyncio.Lock`. An asyncio.Lock binds to the loop that first *waits* on it, so with several event loops in one process (free-threaded uvicorn) the first contended acquire claims it and every other loop then fails with RuntimeError: <asyncio.locks.Lock object ...> is bound to a different event loop It passes a single request and collapses under load, which is the worst possible shape: recall returned HTTP 500 from every loop but one. * engine/bank_stats_cache.py — the TTL cache behind bank config resolution. * engine/db_budget.py — ConnectionBudgetManager, a `_default_manager` singleton. Both now use a threading.Lock. That is not a workaround: every critical section either guards is await-free dict work, so the lock is never held across a suspension point and cannot block a loop, and unlike asyncio.Lock it is loop-agnostic. The cache needed one thing more. Its in-flight map coalesces concurrent loads behind an asyncio.Future, and a Future belongs to the loop that created it, so a caller on another loop must never await it. In-flight slots are now keyed by (running loop, cache key): coalescing happens within a loop, while the cached DATA stays shared across all of them, which is the part worth having. `invalidate()` detaches the key on every loop, since the slots are per-loop. Measured on python3.14t, real recall against Postgres 18 + pgvector, 8 event loops in one process, 64 concurrent: before 0 successful requests, 57 cross-loop errors after 124-165 rps at 5.6-5.9 cores, p50 354-438ms, 0 cross-loop errors For reference the same workload on 3.11 (one loop, as uvicorn runs today) does ~40 rps at 0.96 cores with p50 ~1390ms. Verified on 3.11 and python3.14t: the bank stats/info cache suites (12 passed, 13 skipped) pass identically on both. Refs: initiative kp-ac23cdb434a144bf947371b6e6e4f5e8 * fix(api): replace the remaining process-wide asyncio primitives Four module- or class-level `asyncio` primitives were left after the cache and budget-manager fixes. Each binds to the loop that first waits on it, so with several event loops in one process the first contended acquire claims it and every other loop fails with "is bound to a different event loop" — under load only, which is why none of them showed up in tests: * llm_wrapper `_global_llm_semaphore` and `_per_op_llm_semaphores` (module scope, built at import before any loop exists) * cross_encoder `RemoteTEICrossEncoder._global_semaphore` (class attribute) * llamacpp `_shared_server_lock` (module scope) Adds `_cross_loop.py` with `CrossLoopSemaphore` / `CrossLoopLock`. The counter lives in a threading primitive, which is loop-agnostic, and waiting is a short async backoff so a loop is never blocked while it queues — unlike a bare threading.Lock, these are safe to hold across `await`, which llamacpp's server start/stop needs. The caps stay PROCESS-wide rather than becoming per-loop. That preserves the existing contract: `--workers N` has always meant N independent caps, one per process, so `HINDSIGHT_API_LLM_MAX_CONCURRENT=8` keeps meaning 8 in flight per process instead of silently becoming 8 x loops against the provider. Polling rather than a cross-loop future handoff is deliberate and documented: it only runs while a cap is saturated, costs at most 20ms of extra latency acquiring a slot, and carries none of the per-loop waiter-registry state that `call_soon_threadsafe` would need. These gate LLM calls and subprocess startup, so that is not measurable. The uncontended path does not yield at all — a test pins that, since it is on every LLM call. Also documents the whole class of bug in the code-review skill: a "Concurrency" standards section (which lock, and why the choice is ownership rather than style) and review step 11d with the greps to catch it. tests/test_cross_loop_primitives.py covers cross-loop use, that the cap really is process-wide, exclusivity held across an await, and the uncontended fast path. It also pins the failure mode of a plain asyncio.Semaphore, so the reason this module exists cannot quietly stop applying. These tests need no free-threaded build — two event loops in one process reproduce it on 3.11. * fix(api): expose CrossLoopSemaphore's cap instead of a private counter test_llm_per_op_concurrency asserted the configured cap by reading asyncio.Semaphore's private `_value`, so it broke when the per-operation caps became CrossLoopSemaphores. Adds a public `capacity` property and asserts on that. Reading the cap is a reasonable thing for a caller to want; making it public is better than swapping one private attribute for another. Caught by re-running the full 3.11 suite against the final tree — the earlier 3.11 control predated this commit, so it would otherwise have shipped as an unnoticed regression on the supported interpreter. * build(docker): add a free-threaded image target (tag -py3.14t) Adds `api-builder-freethreaded` and `api-only-freethreaded`: docker build --target api-only-freethreaded -t hindsight:py3.14t . Built as its own pair of stages rather than by parameterising the existing ones. Almost nothing is shared — the interpreter has to be installed rather than taken from the base image, the dependency resolution differs, and local ML is excluded — so parameterising would have complicated the supported 3.11 path to no benefit. Nothing above these stages changes. Notes on the shape, each of which cost a build to find: * There is no official free-threaded python image; the library/python tags ship the GIL build only. uv installs the interpreter into /opt/pythons and the runtime stage copies it alongside the venv. * Not `uv sync --locked`. The resolution must drop the dependencies with no cp3XXt wheel (overrides-freethreaded.txt) and `uv sync` takes no --override, so it resolves fresh against the same pyproject. That is precisely why this ships as its own tag instead of being assumed equivalent to the pinned image. * No `uv pip check` either: it fails by design here, because pyproject still declares orjson and quicktok-v1 for every other interpreter and pip check cannot know their absence is deliberate. * libpq-dev/libpq5 are needed because psycopg2-binary has no cp3XXt wheel and builds from source. Shipping it costs the image nothing: migrations.py runs alembic in a subprocess on a free-threaded build, so psycopg2 is never imported into the serving process. * local-ml is refused outright with an explicit error rather than silently producing a mis-tagged image, since importing sentence_transformers re-enables the GIL. This tag defaults to remote embeddings and reranking. The build asserts what the tag claims: it imports the whole API under PYTHONWARNINGS="error:The global interpreter lock:RuntimeWarning" and checks `sys._is_gil_enabled()` is False. A C extension that has not declared Py_MOD_GIL_NOT_USED re-enables the GIL on import and says so only with a warning, so without this an image could look free-threaded and run single-threaded. The runtime also sets HINDSIGHT_API_FREE_THREADING=strict so the container refuses to start in that state rather than being merely slow. Verified: image builds (1.94GB), starts, runs its startup migrations with the GIL still disabled, and serves a real recall (142 results, matching the 3.11 image on the same corpus) with zero GIL warnings in its log. * test(api): guard multi-loop safety on the ordinary 3.11 suite Every cross-loop bug found while bringing up the free-threaded server — the bank stats cache, the connection budget manager, the LLM concurrency caps, and dateparser's locale dictionaries — was found by running the server with eight event loops, and none of them needed a free-threaded interpreter to reproduce. asyncio.Lock/Semaphore/Future bind to the loop that first waits on them regardless of the GIL, so two loops in two threads reproduce the whole class on 3.11. This adds that as a normal test: it runs everywhere, in seconds, with no special build. That is what keeps the free-threaded CI job from having to be the only safety net. The cheap guard catches loop-binding on every PR; the expensive job is left to cover what genuinely needs the interpreter — a C extension silently re-enabling the GIL, and races that only appear under true parallelism. Each case corresponds to a bug that shipped. Verified as a negative control by restoring the pre-fix bank_stats_cache: on 3.11 the cache test fails with "got Future ... attached to a different loop", exactly as the eight-loop server did. The final case pins the premise itself — that a module-level asyncio primitive still breaks across loops — so if CPython ever changes that, the guards above get revisited rather than quietly becoming theatre. * docs(code-review): state that both 3.11 and free-threaded 3.14 are supported The Concurrency section explained which lock to use and why, but never said why a reviewer should care — so the rules read as advice about a hypothetical future interpreter rather than a property of the two builds the project actually ships. Adds a "Supported interpreters" section naming both: CPython 3.11 (the default image, the `.python-version` pin, what `uv.lock` resolves for) and free-threaded CPython 3.14 (the `-py3.14t` image target). Neither can be deferred to a follow-up. It calls out the two things that catch people. Anything process-wide is genuinely concurrent on 3.14t, because the GIL is no longer making check-then-act accidentally atomic. And free-threading is lost SILENTLY: importing a C extension without `Py_MOD_GIL_NOT_USED` re-enables the GIL for the whole process with only a RuntimeWarning, so the 3.14t image keeps working and merely performs like the 3.11 one. "It passed CI" is therefore weaker evidence than usual, which is why the free-threaded job asserts the GIL is off before running any test and the image build asserts it too. Also notes the thing that makes this tractable to review: most of what breaks is not free-threading-specific but multi-loop, and multi-loop reproduces on 3.11 as soon as two event loops exist in one process. So new shared state is expected to be covered by tests/test_multi_loop_conformance.py in the ordinary suite, not left to the free-threaded job. Extends review step 11d and the must-fix list with the interpreter-dropping cases — chiefly a new C-extension dependency with no cp3XXt wheel on the API import path. * ci(api): drop quicktok from the free-threaded overrides, record orjson's cost quicktok left the project in #4022, so overriding it out is dead weight. Records what the remaining override actually costs, measured on a 1024-dim embedding rather than assumed: with orjson 23 us/vector, literal 6,531 chars fallback (_repr_literal) 325 us/vector, literal 20,504 chars 14x slower to render and 3.1x more bytes on the wire per vector. Both spellings parse to the same float32 bytes, so this costs throughput and nothing else, and it is on the retain path only (memories/pg/writes.py, retain/link_utils.py) — recall never renders a vector this way. That is the number worth having before anyone decides whether orjson is worth chasing upstream: it makes the free-threaded image a poor fit for a retain-heavy deployment and a fine one for a recall-heavy deployment, which is exactly the workload the free-threading work was aimed at. * fix(api): turn the free-threading guard off inside the migration child The migration subprocess imports psycopg2 deliberately — that is the entire reason it exists. But the guard this branch adds is inherited by the child, defaults to strict on a free-threaded build, and therefore turns psycopg2's "the GIL has been enabled" warning into an exception. Every migration failed: RuntimeError: Migration subprocess failed (exit 1). ERROR __main__: Failed to run database migrations: The global interpreter lock (GIL) has been enabled to load module 'psycopg2._psycopg'... which took out 1765 tests on 3.14t — every fixture that migrates a schema. This is a stacking bug, not a bug in #4033: that PR sets ENV_MIGRATION_ISOLATION to "never" in the child to stop it recursing, which is all it needs because the guard does not exist there. The guard is this branch's, so disabling it in the child is this branch's job too. Also clears PYTHONWARNINGS for the child, for the same reason one step removed: the free-threaded CI job runs the suite with that warning promoted to an error, and the child must not inherit it. Found by running the full suite on both interpreters after the rebase — the free-threading-only failures went from 19 to 1765, which is what a broken shared fixture looks like rather than a broken feature. * test(api): patch the isolation seam directly, not the cached env var The migration orchestration tests opted out of the subprocess by setting HINDSIGHT_API_MIGRATION_ISOLATION=never. That never worked: the flag is read through get_config(), whose result is cached in a module global, so setting the env var after any earlier get_config() call has no effect. It passed on 3.11 by accident — "auto" resolves to "never" there anyway, because the interpreter is not free-threaded — and failed on 3.14t, where "auto" isolates and the patched step functions were never reached. Patches migrations._should_isolate_migrations instead, which says plainly what these tests need: the fan-out has to happen in this process, because what they assert is the call sequence and a subprocess would not see the patches. Worth folding into #4033: its tests are green on 3.11 for the same accidental reason, so the flag's opt-out is not actually exercised there. * build(docker): give the free-threaded image its own Dockerfile The free-threaded stages lived in docker/standalone/Dockerfile. They shared nothing with it that mattered: a different interpreter (installed rather than taken from the base image), a different dependency resolution, no control plane, no local ML. The only thing genuinely in common is start-all.sh, and the runtime hardening was duplicated rather than reused anyway — so "reuse" was buying nothing while making a 550-line file longer and threading a build arg through four stages of the supported 3.11 image. Moves them to docker/standalone/Dockerfile.freethreaded. docker/standalone/Dockerfile is now byte-for-byte what it was before this branch. docker build -f docker/standalone/Dockerfile.freethreaded -t hindsight-api:py3.14t . Also drops overrides-freethreaded.txt entirely. It existed to remove dependencies with no cp3XXt wheel: quicktok left in #4022 and orjson in #4040, and everything remaining publishes free-threaded wheels, so there is nothing left to override. The CI job installs plainly now too. DEPENDS ON #4040. Until that merges, orjson is still in the runtime closure and the free-threaded install fails on it — verified, that is exactly what the build does without the override this commit removes. * ci(docker): build, smoke test and release the free-threaded image The `-py3.14t` image existed but nothing built it outside my machine, nothing exercised it, and the release never published it. Closes all three. CI (test-api free-threaded job) now builds the image and runs docker/freethreaded-smoke.sh against it. The build already asserts the GIL is off after importing the whole API, so a mis-tagged image fails before the smoke test starts; the smoke test then covers what a build cannot: * the container reaches /health/ready — i.e. it ran its migrations, which on this image means the subprocess path, since psycopg2 would otherwise take the GIL for the life of the process; * it serves a real retain and recall; * the SERVER process still has the GIL disabled, asserted rather than inferred from the container working, because losing it is silent. The smoke test is separate from test-image.sh rather than a flag on it: this image ships no local models, so embeddings must be remote and test-image.sh assumes a provider API key. It carries a deterministic stub embedder inline so it needs no secrets and no network, which also means it runs on fork PRs — the free-threaded job is deliberately not gated on secrets. Release: adds the tag to the docker matrix. Every entry now names its Dockerfile, since the free-threaded image has its own. Two deliberate asymmetries: * `latest` never points at `-py3.14t`. It is not a drop-in for the default tag — no local models — so it must be asked for by name. * linux/amd64 only. The build installs the interpreter and compiles psycopg2 from source, so emulated arm64 is slow enough to be worth adding deliberately rather than inheriting by default. Also silences the GIL warning inside the migration child. The child is SUPPOSED to take the GIL, and left visible the warning surfaces in a `-py3.14t` container's log as "the global interpreter lock (GIL) has been enabled" — which reads exactly like the image has silently lost its free-threading when it has not. The smoke test now treats any occurrence in the log as a failure, which only works once the expected one is gone. Documents the tag and its constraints in installation.md (regenerated docs skill). Verified locally end to end: image builds, and the smoke test passes — starts, migrates, retains, recalls, `free-threaded: 3.14.7`, no GIL warnings. * ci(api): fix what the free-threaded CI job actually caught The job failed with 64 failures on its first real run. My local container runs had missed all of them, for two reasons worth recording: I never ran the suite with PYTHONWARNINGS set, and my local venvs were not built with --all-extras the way CI's 3.11 job is. 51 of the 64 were the job's own configuration. It ran the whole suite with the GIL re-enable warning promoted to an error, but the suite imports LiteLLM on purpose to test that provider, and LiteLLM pulls in fastuuid, which has no free-threaded build — so ~50 tests failed for doing exactly what they are meant to do ("NameError: name 'fastuuid' is not defined"). The filter now applies only to the step that imports the whole API to assert the GIL is off. That is where it belongs, and the property is still asserted three more times: in tests/test_free_threading.py (in a subprocess), in the image build, and in the image smoke test. 5 were a real 3.14 incompatibility in test code: `asyncio.get_event_loop()` no longer auto-creates a loop, so `get_event_loop().run_until_complete(...)` raises "There is no current event loop in thread 'MainThread'". Replaced with `asyncio.run`, which is the supported spelling and behaves identically on 3.11. 2 were tests that need the local-ml extra, which a free-threaded install cannot have (sentence-transformers re-enables the GIL) and CI's 3.11 job does have via --all-extras. Both now `importorskip` the thing they actually need — torch's global default dtype has nothing to assert without torch, and the reflect test's MemoryEngine construction reaches the local embeddings provider. The rest are pre-existing or already attributed: the xai_oauth cleanup test fails with PYTHON_GIL=1 on the same binary, so it is a 3.14 asyncio change rather than a free-threading one. * ci(api): close the last four free-threaded CI failures Down from 64 to 4 after the previous commit; these are the remainder. Two were the local-ml pattern again. tests/test_jina_mlx_import_error.py stubs mlx, but the path under test still reaches transformers for a tokenizer — so without the extra the assertion sees "No module named 'transformers'" instead of the message it checks. It now importorskips transformers, which is what it actually needs. One was a missing environment variable rather than a code problem: the github-copilot provider looks for CLI account metadata that no runner has, and falls back to a token. The 3.11 test-api job passes GITHUB_TOKEN and this job did not. Added. It is not a repository secret — Actions provides it to every run, forks included — so the job stays runnable on fork PRs, which is deliberate given the secret-gated test-api jobs skip there entirely. The last is tests/test_xai_oauth_llm.py::test_cleanup_closes_a_client_still_draining _from_a_recycle, which fails with PYTHON_GIL=1 on the same binary. It is a 3.14 asyncio scheduling change that a plain 3.14 upgrade would hit identically, not something free-threading introduces, and it is left unfixed and attributed rather than worked around. * fix(xai-oauth): close a retired client whose drain task was cancelled `cleanup()` cancels each in-flight drain task so shutdown does not block on a request that may never land, and left the actual close to `_close_when_drained`'s `finally`. From Python 3.12 that no longer works: the cancellation is delivered at the task's next await — which IS the `await stale.aclose()` in that `finally` — so the close never runs, and CancelledError is a BaseException, so the `suppress(Exception)` around it does not catch it either. The client was therefore never closed and leaked its connections on every shutdown that happened while a recycle was still draining. `cleanup()` now closes the retired clients itself, after the drain tasks are done. The list is captured before cancelling, because the `finally` pops each entry out of `_drained` on its way through whether or not the close happened. aclose() is idempotent, so a drain that completed normally costs nothing. Found by the free-threaded CI job, but it is NOT a free-threading bug: it reproduces identically with PYTHON_GIL=1 on the same interpreter, so a plain 3.14 upgrade would hit it too. 3.11 is unaffected — the older cancellation semantics let that `finally` await run. Two earlier theories were wrong and are recorded so nobody retries them: swallowing the CancelledError is not enough (the next await is cancelled again), and `Task.uncancel()` does not help either (`_must_cancel` still fires at the next await). The close has to happen outside the cancelled task. Adds a regression test asserting the drain task is gone AND the client is closed. Verified as a negative control: with the fix reverted it fails on 3.14t with "a retired client was left open after cleanup", and passes on 3.11 either way, which is exactly the interpreter split the bug has. * test(retain): stop asserting batch dispatch ORDER in the coalescer test test_batches_never_exceed_the_backend_batch_size compared the flattened backend calls to the input list, which pins the order in which batches reach the backend. The coalescer never promised that: it runs up to `max_concurrent_requests` calls at a time (`self._slots`), so which batch lands first is a scheduling detail. Under the GIL the interleaving happened to be stable, so the assertion held. On a free-threaded interpreter the batches genuinely race and it failed on order alone — every text present, every batch within budget, every caller's vectors correct: At index 8 diff: 'chunk-16' != 'chunk-8' Compares a Counter instead, which keeps the property that actually matters — every text dispatched exactly once, nothing dropped or duplicated — and leaves the per-caller assertion below it untouched, since that is what proves each caller gets its own vectors in its own position. This is a test that was over-specified, not an implementation that regressed. The batch-size assertion the test is named for is unchanged. Verified 8/8 under xdist on 3.14t, where it was failing intermittently, and on 3.11. * build(docker): install Rust in the free-threaded builder, for litellm The image build failed on amd64: Failed to build `litellm==1.99.0` Error: command ['maturin', 'pep517', 'build-wheel', ...] Caused by: No such file or directory (os error 2) litellm publishes only abi3 wheels (cp310-abi3), and abi3 — the stable ABI — does not apply to a free-threaded interpreter, so uv cannot use them and falls back to the sdist, which builds litellm's Rust extension. litellm is a hard runtime dependency (pyproject pins it per-platform), so this is not optional. It is the same constraint that kept toktok from working free-threaded before #4022: an abi3 wheel is invisible to a cp3XXt interpreter. Rust is confined to the builder stage — the runtime image copies only the venv and carries no toolchain. Drop this once litellm publishes cp3XXt wheels. Only amd64 hit it: my local builds were linux/arm64, where the resolution differed. That is a good argument for the CI job building the image at all, which is what caught this.	2026-09-02 22:23:06 +02:00
Nicolò Boschi	e8518c392f	fix(clients): close the bank-config drift between the wrapper SDKs and the server (#4030) ... The wrappers (hindsight_client.py / src/index.ts) are hand-written layers over the generated SDKs, and they build the config PATCH body by enumerating fields. The endpoint takes a free-form `updates: dict[str, Any]`, so nothing regenerates that list and nothing checked it. It had drifted three ways: - 22 of the server's 47 _CONFIGURABLE_FIELDS were reachable from neither wrapper, including the whole recall-budget group and memory_defense. - 8 more were in Python but not TypeScript. - reflect.apply_all_directives was missing from both. Both wrappers now cover all 47, plus apply_all_directives. Two wrong type hints went with it. llm_gemini_safety_settings was declared dict[str, str] in Python, but the server, the Gemini provider and the control plane all take a list of {category, threshold} — a caller following the hint sent a shape the provider rejects. client-coverage-check existed to catch exactly this and was wired into no workflow, so it sat red on main with apply_all_directives missing from both wrappers. It now runs in CI next to check-cli-coverage, and it can see bank config: the OpenAPI walk it already did reports updateBankConfig as covered by its single `updates` property, so the check reads _CONFIGURABLE_FIELDS from config.py directly. The bank-config check verifies forwarding, not just acceptance. Declaring the keyword is not enough — a field accepted and never written into the request body is silently dropped, which is how this class of bug presents: the caller gets a 200 and no change. Refs #4029	2026-09-02 14:24:59 +02:00
Nicolò Boschi	ac41cee604	feat(extensions): add hindsight-extensions registry and unbundle Supabase (#3988) ... * feat(extensions): add hindsight-extensions registry and unbundle Supabase Extensions were only ever bundle-able or nothing: shipping one meant putting it in `hindsight_api.extensions.builtin`, where it becomes maintainer-owned forever, lands in every image, and — because `extensions/__init__.py` eagerly re-exported every implementation — drags its dependencies into core's import graph. That pipe is why a third-party IdP's JWT client was a direct dependency of every Hindsight install. Add `hindsight-extensions/` as the registry for extensions distributed separately from the server. Its README is the contract: slots and how config env vars map onto them, how to write an extension, the package layout and naming (`hindsight-extensions/<name>/` -> `hindsight-ext-<name>` -> `hindsight_ext_<name>`), and Docker packaging. Move `SupabaseTenantExtension` there as the first entry, published as `hindsight-ext-supabase-tenant`. All 54 of its tests move with it, plus two new ones asserting the documented `hindsight_ext_supabase_tenant:...` env value actually resolves through `load_extension`. Two decisions worth their comments: - The extension does NOT declare `hindsight-api-slim` as a runtime dependency. The server is the host process that imports it, not something it installs; declaring it would let `pip install` of an extension silently move the server version underneath a running deployment. It is a dev extra, resolved from the local checkout via `tool.uv.sources` (dev-only metadata, verified absent from the built wheel). - The Docker example installs with `uv pip install --python /app/api/.venv/bin/python`, matching docker-compose/custom-models: the image's venv was created by `uv sync` and ships no `pip`, so a bare `pip install` lands in user site-packages and is invisible to the server. Core changes: - `extensions/__init__.py` and `builtin/__init__.py` export interfaces only. Nothing needed the concrete re-exports — the loader imports by path — and dropping them is what lets an extension have optional dependencies at all. - `builtin/supabase_tenant.py` stays for one minor release as a module whose `__getattr__` raises the migration instructions. `load_extension` wraps a missing *attribute*, not a failed import, so the ImportError propagates with its message intact instead of surfacing as "class not found". - Drop the direct `PyJWT[crypto]` dependency: no core module imports `jwt` any more. Note this does not shrink the install — `mcp` pulls pyjwt transitively and `cryptography` is already pinned directly — so the win here is ownership and import graph, not bytes. Locks are not checked in for extensions: `tool.uv.sources` pins the whole api-slim tree, so every core dependency bump would leave them stale. CI runs `uv sync --extra dev` and retriggers on `core` changes, since these tests run against the server's interfaces. Docs point at the registry rather than restating it, and the Deploying section's Docker recipe was replaced — it named an image (`vectorize/hindsight-api`) and a PYTHONPATH volume-mount pattern that no longer exist. Also includes two one-line generated-file syncs in skills/hindsight-docs (quickstart, installation) that were already stale on main; regenerating the docs skill picks them up. * refactor(extensions): ship extensions by image, drop the compat shim Follow-up on review. Three changes to how an extension is distributed: - Delete `builtin/supabase_tenant.py`. An install pinned to the old path now fails at startup with ModuleNotFoundError rather than a guided message. The docs carry the migration instead. - Extensions are not published to PyPI. There is no wheel, no version and no release step: the unit of distribution is an image built on top of Hindsight that installs the extension's dependencies and copies the package onto PYTHONPATH. That drops the whole "declare hindsight-api-slim only as a dev extra" problem — nothing resolves dependencies against a running server any more. - The pyproject is now test-harness only (`package = false`, no build backend, no distribution metadata), and says so in a comment so nobody re-adds packaging to it. Docs say 0.9.3, not 0.10. Since the Dockerfile is now the distribution mechanism rather than an example, CI builds it — its final `import` step is the only thing proving the extension is reachable from the interpreter the server actually runs. It builds against `:latest-slim` via a HINDSIGHT_IMAGE build arg to keep the pull cheap. Verified against the real image, not just locally: docker build -f hindsight-extensions/supabase-tenant/Dockerfile \ --build-arg HINDSIGHT_IMAGE=ghcr.io/vectorize-io/hindsight:latest-slim ... -> load_extension('TENANT', TenantExtension) inside the container returns SupabaseTenantExtension with its config resolved from the env vars. Worth noting from that build: `uv pip install 'PyJWT[crypto]' httpx` reports "Checked 2 packages" — both are already in the base image transitively. The line stays because the extension should pin what it imports rather than rely on the server's transitive tree, but it costs nothing today. 56 extension tests pass; the 3 remaining core tests (which assert no implementation is re-exported and no core module imports jwt) pass. * fix(tests): import ApiKeyTenantExtension from its module, not the package Dropping the concrete re-exports from `hindsight_api.extensions` broke `tests/test_extensions.py`, which imported `ApiKeyTenantExtension` from the package inside a multi-line parenthesised import. A collection ImportError fails the whole shard, which is why all three test-api shards and all six LLM acceptance jobs went red at once on the previous push. I'd checked for this with a single-line grep, which cannot see a name inside a parenthesised import list. Re-checked with an AST scan over every package in the repo (this was the only occurrence) and by collecting the full suite: 7780 tests collect clean.	2026-09-01 14:47:37 +02:00
Nicolò Boschi	b12b77dbfc	fix(ci): stop one stalled Gemini call from outliving the caller, and isolate the third claim-test file (#3982) ... * fix(ci): stop one stalled Gemini call from outliving the caller, and isolate the third claim-test file CI has been red on every branch since this morning, in four jobs with three independent causes. None of them is a product bug on the branch that trips them. 1. test-typescript-client, test-python-client(-oracle), test-doc-examples (cli) — all reflect, all the same shape: ERROR gemini_llm - Unexpected error during Gemini tool call: TimeoutError: WARNING reflect.agent - [REFLECT ...] LLM error on iteration 2: (120002ms) INFO memory_engine - [REFLECT ...] Complete: 448 chars, 2 iterations | 121.981s Gemini sometimes accepts a request and never answers it. The per-request deadline is the only thing that ends such a call, and the abort was terminal: TimeoutError fell through to the generic handler, which re-raised it, so the caller paid the whole deadline AND got an error — reflect then answered from a degraded forced pass. These stalls are not new. What changed is what they cost: #3946 made Gemini honour the configured llm_timeout instead of its hardcoded 90s, so the same stall went from 90s to 120s. Yesterday's green run has one at 83.994s, comfortably inside the 120s these suites allow a test; today's has 121.981s, just outside it. Two halves, at the layers they belong to: * A deadline abort is now retried exactly once, in both Gemini call paths. Nothing came back, so nothing about the request is implicated, and on a healthy provider the retry lands in well under a second. One retry only — tracked separately from the API-error ladder — because a second stall says something about the provider, not about this request. * Reflect gets its own per-request deadline default, 60s (DEFAULT_REFLECT_LLM_TIMEOUT). It is the one interactive operation: a caller holds an HTTP request open while it makes several sequential LLM calls, so a per-call deadline equal to the whole global budget lets ONE stalled call outlive the caller. Retain and consolidation keep 120s — they run in the background against a queue, where the deadline is there to stop runaway generation. An explicit HINDSIGHT_API_LLM_TIMEOUT is still inherited: an operator who set a global deadline meant it, and being quietly capped below it would be the more surprising of the two behaviours. 2. test-api (3/3) — "expected only the oldest same-document retain, got []". tests/test_retain_document_serialization.py claims queue-wide against the shared public schema, so another xdist worker's poller can take its rows before it does; its own claim then comes back empty and reads as "the predicate excluded them". This is the third file with that defect: #3963 gave test_worker.py and test_claim_bank_serialization.py a private migrated schema for exactly this, and this one was missed. Same fix. Only the queue tests take `backend`; the end-to-end append tests build their own engine and are untouched. 3. Core LLM tests — two assertions that test something other than their property: * test_delta_editorial_fusion asserted `"keyword" in fused`. The refresh had kept every SEO rule and written them as "search terms" / "search intent": the guidance was there, only the token was missing. It now judges the fusion, per CLAUDE.md — the structural asserts (no duplicate paragraphs, based_on counts) stay direct. * test_refresh_with_tags_only_accesses_same_tagged_models required the refreshed content to mention four specific Alice details. It leaked nothing at all and was reported as a SECURITY VIOLATION for summarising her more briefly. Tag scoping guarantees that nothing outside the scope is reachable; which of Alice's own details get written up is the summariser's call. The absence half stays strict, the presence half is now loose. Tests: new tests/test_gemini_deadline_retry.py pins both call paths (retry once, then give up — not the whole ladder); TestReflectDeadlineDefault pins the three-way resolution; the existing per-op defaults test now says which operation gets which deadline instead of asserting they are uniform. Claude-Session: https://claude.ai/code/session_0134gt96Tyi2Yi55yDH5s4Rb * fix(ci): size the reflect deadline against the retry ladder, and rerun the model-behaviour job First CI run on this branch made the mechanism visible and showed both numbers were wrong. The retry works — the log has stalls answered in ~3s on the next attempt, with whole reflects landing at 64.0s and 69.9s where they would previously have been errors at 120s. But the stall RATE is far higher than assumed: 5 stalls in one doc-examples job, and 2 of those stalled again on the retry. 60s x 2 attempts is 120s, which is exactly the cliff the change set out to clear, so those two runs failed identically to before (121.904s, 121.437s). What has to fit inside the caller's patience is deadline x attempts, not one deadline. Sized as such: - DEFAULT_REFLECT_LLM_TIMEOUT 60s -> 30s, and _TIMEOUT_RETRIES = 2, so the worst case is 3 x 30 = 90s. Healthy reflect calls in these logs answer in 1-4s, so the headroom is still an order of magnitude, and a third consecutive stall is ~1 in 60 at the observed rate rather than the ~1 in 16 that a single retry left. Second: `Core LLM tests` fails on a DIFFERENT test most runs — five distinct ones across the two runs of this branch (a judge verdict on emotional dimension, whether the agent called done() rather than being forced, a ReflectNoAnswerError, plus the two fixed in the previous commit). Every test in that job asserts what a model CHOSE to do, so one run cannot separate a regression from the model landing differently, and a red there currently carries no signal. It now runs with --reruns 2: a real regression reproduces and stays red, a coin-flip settles. Deliberately not applied to the deterministic shards, where a rerun would hide the races those tests exist to catch. Claude-Session: https://claude.ai/code/session_0134gt96Tyi2Yi55yDH5s4Rb * fix(tests): state the emotional-dimension threshold before the checklist Last CI run left one failure, and the reruns made it diagnosable: three attempts, all failing the same way, so not a coin flip. It is a false negative. Extraction produced "Sarah seemed disappointed upon hearing about the delay" and "Marcus felt anxious about the upcoming interview" — two of the three emotional states, which is exactly what the criteria says it takes to pass — and 3/3 judges still voted not-met, reasoning that the response "omits the emotional states". Only the speaker's thrill was dropped. The wording invited it: three states listed first, "at least two of these should be present" trailing behind them. Judges anchor on the enumerated items and read the missing one as the answer. Now the threshold leads and the three are a checklist under it, with the failing condition spelled out (two or more stripped to the bare event). Claude-Session: https://claude.ai/code/session_0134gt96Tyi2Yi55yDH5s4Rb * fix(tests): judge the fact sentence, not the rendering that restates the bare event The previous commit's diagnosis was wrong. Reproduced locally against the real model and judge: the criteria wording was not the problem, the response shape was. Extraction is perfect. All three emotions survive: TestUser was thrilled about receiving positive feedback on their presentation. Sarah seemed disappointed upon hearing about the delay. Marcus felt anxious about the upcoming interview. But the test fed the judge each fact's full rendering, and every one of those ends with a trailing dimension that is a bare restatement of the event: ... was thrilled about ... | When: ... | Involving: TestUser | Received positive feedback on presentation. The judge reads the restatement as the evidence and answers accordingly — "stripped down the emotional states of 'thrilled' and 'disappointed' to the bare events, only retaining 'anxious'" — about a response that says "was thrilled" and "seemed disappointed" in as many words. The clause I added last commit ("FAIL only if two or more were stripped down to the bare event") gave that misreading something to match, so it made the failure more certain rather than less. The emotional dimension lives in the sentence, so judge the sentence: one per line, metadata dropped. The context block goes too — it restated the same three emotions, leaving the judge holding them once as background and once as the thing to look for, which is the shape build_judge_messages already documents as blurring the two. Verified locally against the real pipeline: 4/4 passes, where the previous wording failed 3/3 in CI and on the first local run. Claude-Session: https://claude.ai/code/session_0134gt96Tyi2Yi55yDH5s4Rb	2026-09-01 13:59:39 +02:00
Rainier Schlekewey	466517ff9b	fix(docker): drop npm from the runtime stages and scan every runtime base (#3960) ... * fix(docker): drop npm from the standalone and control-plane runtime stages Every published image I scanned ships npm's own bundled copy of tar 6.2.1, which carries one CRITICAL and eight HIGH advisories, all of them fixed upstream: CVE-2026-59873 CRITICAL fixed in 7.5.19 CVE-2026-23745 HIGH fixed in 7.5.3 CVE-2026-23950 HIGH fixed in 7.5.4 CVE-2026-24842 HIGH fixed in 7.5.7 CVE-2026-26960 HIGH fixed in 7.5.8 CVE-2026-29786 HIGH fixed in 7.5.10 CVE-2026-31802 HIGH fixed in 7.5.11 CVE-2026-59874 HIGH fixed in 7.5.18 CVE-2026-73566 HIGH fixed in 7.5.21 It is npm's vendored tar, not an application dependency, so no package.json, lockfile or `overrides` change reaches it: /usr/lib/node_modules/npm/node_modules/tar (standalone) /usr/local/lib/node_modules/npm/node_modules/tar (cp-only) Bumping Node does not clear it either. Node 20 pins npm 10.8.2, which vendors tar 6.2.1; Node 22 pins npm 10.9.8 (tar 7.5.11, still under the CRITICAL fix) and Node 24 pins npm 11.19.0 (tar 7.5.19, still under CVE-2026-73566). Removing npm is what clears all nine findings. npm is never invoked at runtime. Both stages run `/app/start-all.sh`, which starts the API console script and launches the control plane as a pre-built Next standalone bundle with `node server.js` (docker/standalone/start-all.sh:305). No path I could find in the image shells out to a package manager: there is no HEALTHCHECK, no migration step, no plugin install, and no `next` CLI use at runtime. There is also no `corepack enable` anywhere under docker/ and no global npm install in any image stage, so nothing re-adds npm or npx to PATH in the runtime layers. The npm invocations in this file (lines 96, 97, 117, 134) are all in the node:20-slim builder stages, which keep npm untouched. `node` itself stays. Removal is by path in both stages. The node:20-alpine base unpacks npm from the Node tarball rather than installing an apk package. On the Debian stage npm is not installed as a separate apt package either: NodeSource's `nodejs` package ships it (`dpkg -S /usr/bin/npm` resolves to `nodejs`), so `apt-get remove npm` is a no-op and removal by path is the only option there too. corepack is deliberately left in place in both stages. It has no node_modules tree and ships no tar package metadata, so no scannable vulnerable tar component remains in either stage after this change. node-tar code is webpacked into its `dist/lib/corepack.cjs` bundle without package metadata, so its version cannot be determined from the image. Both path sets were validated in live containers against the real node:20-alpine and python:3.11-slim + NodeSource setup_20.x base images: every target existed before deletion, a filesystem-wide search for `*/node_modules/tar/package.json` returned nothing afterwards, npm and npx were gone from PATH, and node (plus python3 on the Debian stage) still ran. The api-only stage has no Node at all and is unaffected. * ci: build and scan every runtime base in scan-next-build, not just api-only scan-next-build answers "would an image built from main today be clean?", but it only builds `target: api-only`, which is python:3.11-slim with no Node in it at all. The two images that carry a Node runtime, cp-only (node:20-alpine) and standalone (Debian plus NodeSource nodejs), were never built here, so a Node-side finding had no fresh-build gate and could only surface in scan-published, after it had already shipped under a tag. That is how npm's bundled tar 6.2.1 reached every published image with a CRITICAL against it. The job becomes a matrix over the three targets, mirroring the matrix scan-published already uses. api-only keeps covering the Debian family for the reason recorded in the old comment; cp-only and standalone add the Node surface. The build args are unchanged, so the legs that read them stay on the slim runtime surface and cp-only ignores them. Each leg runs on its own runner, so this does not lengthen the job or add disk pressure to the existing build. * fix(docker): fail the build if the npm paths move, and exercise the Node runtimes Follow-up to the npm removal on the same branch, from review. A bare `rm -rf` on a path a future base image no longer uses succeeds silently, so a Node or NodeSource bump could quietly start shipping npm again. Both runtime stages now assert the path exists before removing it and assert npm is gone afterwards, which fails the build instead. The removals also had no runtime coverage. `build-docker-images` gated both `load:` and the smoke test on `variant == 'slim'`, and cp-only is a full variant, so the alpine control-plane image was built and thrown away, never started. It now runs the smoke test (it needs no LLM credentials); the gate moves to an explicit `smoke` matrix field so the two heavy full variants stay build-only. `docker/test-image.sh` only probed the API port for standalone, which would pass on an image whose control plane never came up - the two are separate processes. It now waits on the control plane's own health endpoint too when the target serves both. Verified locally on arm64: both targets build with the guards in place, `npm`/`npx` are gone from PATH and no `node_modules/tar/package.json` remains in either image, the cp-only smoke test passes, and the standalone image serves both 8888 and 9999. Claude-Session: https://claude.ai/code/session_011n8KQc8sCe4cJJCd8Cv69n --------- Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>	2026-09-01 10:28:28 +02:00
Nicolò Boschi	f22c36250f	feat(reflect): send the operation schema on mental model delta refresh (#3937) ... * feat(reflect): send the operation schema on mental model delta refresh The delta call was the one pipeline call that asked for structured output without sending a schema. Retain's extraction passes `response_format` and a per-operation `strict_schema`; the delta call passed neither, so the prompt was the only description of the payload the model got — and #3901 was the predictable result, a model spelling new blocks as `{"id", "text"}` because every block in the document it was shown carries an id. The reason was real, not an oversight. Pydantic renders the eight-op discriminated union as `oneOf` + `discriminator`. OpenAI's strict subset accepts `anyOf` and rejects `oneOf`, and the Gemini SDK refuses both keys outright — `types.Schema` raises `Extra inputs are not permitted` while building the request, before anything is sent. So no schema could travel, and the call hand-parsed its JSON instead. Fix the serialization rather than the call site. `UnionSafeSchemaGenerator` renders a tagged union as `anyOf` and drops the discriminator block, which costs nothing: each variant keeps its `Literal` `op` field, so the variants stay mutually exclusive and the discriminator was only ever a routing hint. `OpenAIStrictSchemaGenerator` extends it, so the strict and soft paths agree. Every site that serializes a schema *into a request* now goes through `provider_json_schema()` — all nine providers plus retain's batch path. That is safe to apply that broadly because the output is byte-identical for a model with no tagged union, which is pinned by a test rather than asserted here. Gemini still hands the model class to its SDK for every schema the SDK already converts, and serializes by hand only for a union it cannot accept. `gemini_cache` is left alone: it hashes the schema for a cache key and never sends it. The delta call then follows retain exactly: `response_format`, `strict_schema` from `llm_strict_schema_reflect`, and `skip_validation=True` so the raw JSON still reaches `parse_delta_operation_list` — which drops one malformed op instead of failing the batch the way `model_validate` would. The lenient parser stays in front of the schema, the same way `_coerce_fact_response` sits in front of retain's. Note the soft path (strict off, the default) now appends the schema to the system prompt for this call: better instruction, slightly more prompt tokens. Tests: the union rewrite (both serializers, variants preserved), the no-op property for non-union models, and the Gemini SDK's rejection of the raw union — asserted as a live expectation so the hand-serialization branch gets deleted if the SDK ever grows support. Plus a plumbing test that the delta call sends the schema, which the `patch_llm_call` fixture had been documenting as true since before it was. Claude-Session: https://claude.ai/code/session_011n8KQc8sCe4cJJCd8Cv69n * ci: run the real-LLM Gemini evals in the core LLM job `HINDSIGHT_RUN_GEMINI_EVALS` gates four test files' real-provider evals, and nothing in CI ever set it. `GEMINI_API_KEY` was already on the core LLM job, so the gate looked satisfied and the tests reported as SKIPPED rather than as missing — 11 of them, including the whole `TestDeltaRefreshGeminiEval` class. That left the mental-model delta path with no end-to-end coverage anywhere: the one call that hands a provider a schema the provider has to accept, and the exact path this branch changes. The gap surfaced while verifying the structured-output change — the test named as its safety net turned out never to run. Unskips 11 tests in the job that already has the credentials and the `hs_llm_core` marker for them. Claude-Session: https://claude.ai/code/session_011n8KQc8sCe4cJJCd8Cv69n * fix(gemini): strip additionalProperties and require the union tag The newly-enabled Gemini evals caught two defects in the structured-output change on their first run — both invisible to every offline check. 1. Vertex rejected every delta request: 400 INVALID_ARGUMENT: Unknown name "additional_properties" at 'generation_config.response_schema': Cannot find field. Handing the SDK a dict is not the same as handing it the pydantic class. The class path drops keys the backend has no field for; the dict path maps them faithfully, so `extra="forbid"` on every op model arrived as `additionalProperties`, became `Schema.additional_properties`, and the request was refused. The SDK builds it without complaint, which is exactly why asserting on `t_schema` was not enough — the new test asserts on the serialized request instead. Stripping the key costs nothing: the parser still validates against the pydantic model, so no field can be invented. 2. `op` was not a required property. Pydantic omits it because each variant defaults it (`op: Literal["add_section"] = "add_section"`), and with the `discriminator` block gone, `anyOf` alone gives a reader nothing else to tell eight near-identical shapes apart. A grammar-constrained model that omitted the tag would emit an operation matching no variant — reproducing the validation failure this whole path exists to prevent. The union rewrite now requires the tag wherever it appears, which is the honest completion of dropping the discriminator rather than a separate concern. Neither reached a network call in local testing, and the first was live for one CI round-trip. Both now fail closed in unit tests. Claude-Session: https://claude.ai/code/session_011n8KQc8sCe4cJJCd8Cv69n * test(retractions): give the real-LLM fixture its Vertex credentials The retraction eval builds an LLMConfig from provider/api_key/base_url/model. Vertex AI authenticates by project + service account instead of an api_key, so that config raises before it can make a call: ValueError: HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID is required for Vertex AI provider. The fixture's own docstring warns that a skipped test rots — it was written for a key-based provider, and the CI job runs the evals under `vertexai`, which is the one case it did not cover. Nothing caught it because the test had never run anywhere. Pass the three Vertex fields through from config. Claude-Session: https://claude.ai/code/session_011n8KQc8sCe4cJJCd8Cv69n * test(retractions): the eval imported a function renamed out from under it `parse_markdown` became `split_markdown`; the rename missed this call because the only reference lives in a test that never ran, so nothing failed. Second layer of rot in the same test — the credentials gap hid this one until it was fixed. Checked the rest of its surface rather than fixing one line and paying another CI round-trip to find the next: `build_structured_retraction_prompt` still accepts every argument it passes and requires none it omits, and the other imports resolve. This was the only stale reference. Claude-Session: https://claude.ai/code/session_011n8KQc8sCe4cJJCd8Cv69n	2026-09-01 09:38:18 +02:00
Nicolò Boschi	44f9376311	fix(docker): ship images free of fixed HIGH CVEs and scan them daily (#3936) ... * fix(docker): ship images free of fixed HIGH CVEs and gate releases on a scan The published 0.9.2 images carried four fixed HIGH findings on both architectures (#3906): openssl 3.5.6 (CVE-2026-14456, via libssl3t64 / openssl / openssl-provider-legacy) and jaraco.context 6.0.1 (CVE-2026-23949, pulled in transitively by fastmcp -> py-key-value-aio[keyring] -> keyring). Nothing in the build was pinning either one — the runtime stages installed whatever snapshot the python:3.11-slim base happened to carry, and the lock had never been refreshed past jaraco.context 6.0.1. - Runtime stages now `apt-get upgrade` (and `apk upgrade` for the alpine control-plane stage) so the shipped layers pick up Debian security updates published after the base image was cut. - uv.lock: jaraco-context 6.0.1 -> 6.1.2. - CI: the slim images built by build-docker-images are scanned with Trivy (HIGH/CRITICAL, fixed only) and the job fails on a finding. uv.lock / pyproject.toml join the `docker` path filter, since a dependency bump changes what ships even when docker/ is untouched. - Release: both published architectures are scanned before signing. A finding fails release-docker-images, so create-github-release — which needs it — never publishes a release for the tag. - create-github-release now attaches every asset to a draft and publishes afterwards. GitHub's immutable releases seal a published release, so assets uploaded post-publication would be rejected; this makes the workflow safe once the repo setting is enabled (the mutable-release half of #3906). Verified by building the api-only slim image with these changes and scanning it: openssl 3.5.7-1~deb13u2, jaraco_context 6.1.2, and zero HIGH/CRITICAL fixed findings (0.9.2-slim reports four). Closes #3906 Claude-Session: https://claude.ai/code/session_01M9KwqWJZRrw7NjAaKuACzj * ci: drop the release-time image scan, keep the PR gate The release scan could only run after the multi-arch push (a pre-push scan needs a second single-platform build, which is the disk pressure that got the release smoke test commented out), so it never actually prevented a vulnerable image from existing under its tags — it only withheld the GitHub Release. Scanning on PRs is where a finding can still be acted on, and build-docker-images already loads the slim images there. Claude-Session: https://claude.ai/code/session_01M9KwqWJZRrw7NjAaKuACzj * ci: scan images daily instead of on every PR Image findings appear when an advisory is published, not when our code changes, so a per-PR gate stays green for weeks and then fails an unrelated PR the day a CVE lands. Moved to a scheduled workflow with two jobs: one scans the published images users are running now (no build, seconds), one builds api-slim from main and scans it, so a regression is caught before it reaches a release tag. Also reverts adding uv.lock/pyproject.toml to the `docker` path filter: that was there to trigger the PR scan, and without it a dependency bump would otherwise pay for five image builds and smoke tests. Claude-Session: https://claude.ai/code/session_01M9KwqWJZRrw7NjAaKuACzj * docs(docker): explain why the runtime stages upgrade base packages Records the tradeoff at the point of change: builds are no longer pinned to the base image's package snapshot, which is the whole point but does mean two builds of the same commit can resolve different versions. Claude-Session: https://claude.ai/code/session_01M9KwqWJZRrw7NjAaKuACzj	2026-08-31 17:15:10 +02:00
Sanderhoff-alt	b8947a24e9	fix(ci): authenticate Hermes compatibility clone (#3536)	2026-08-17 13:13:30 +02:00
Ben	f36a462d1d	feat(eliza): add Hindsight long-term memory integration for elizaOS (#2385) ... Adds @vectorize-io/hindsight-eliza, an elizaOS plugin that gives agents long-term memory backed by Hindsight: - HINDSIGHT_MEMORY provider recalls relevant memories into the prompt before each model call. - HINDSIGHT_RETAIN evaluator retains conversation messages after each turn (fire-and-forget; agent replies optional). - Bank defaults to the message entityId for per-user isolation; both sides fail safe so a Hindsight outage never blocks the agent. Targets @elizaos/core ^1.7.2 (current npm latest, not the 2.x beta on main). Includes tests, CI job, release-script + changelog-generator entries, and docs gallery entry + page.	2026-08-13 11:11:24 -04:00
Nicolò Boschi	32c5d65773	fix(ci): fetch crates for all targets before the offline license scan (#3457) ... The Rust CLI job failed in `cargo about generate --offline`: cargo-about resolves the dependency graph for every target platform, so `cargo metadata` wants crates this runner never builds (it died on the Android-only android_system_properties). hindsight-cli/Cargo.lock is gitignored, so CI re-resolves from scratch and no earlier step has populated the registry. Run `cargo fetch` first in both the PR test job and the release job. Without --target it downloads for all target platforms, so the subsequent --offline generate resolves entirely from cache and keeps its ClearlyDefined lookups disabled.	2026-08-13 10:54:42 +02:00
Sanderhoff-alt	0c2e384287	fix(release): ship Rust CLI license notices (#3447) ... Publish the Hindsight MIT license and generated third-party license notices alongside the Rust CLI release assets. Generate the manifest from Cargo.lock with cargo-about and preserve the complete dependency license text. Run generation and verification in the Rust CLI PR test job so unsupported license expressions fail before a release tag. Generate the platform-independent manifest once in the Linux amd64 release job and fail if required assets are missing before publishing. Disable ClearlyDefined lookups to keep CI generation deterministic. Fixes #3446	2026-08-13 09:39:32 +02:00
Ben	a133cc1495	feat(agent-plugin): add portable Hindsight plugin for the Agent Plugins standard (#3394) ... Add a vendor-neutral Hindsight plugin conforming to Vercel's Agent Plugins 1.0.0 standard (plugin.json + mcp.json + skills/SKILL.md), so one artifact gives long-term memory to any compatible client (Codex, Cursor, GitHub Copilot, Kiro, VS Code) instead of a per-IDE integration. The plugin is a thin transport wrapper over Hindsight's existing MCP server (retain / recall / reflect); a bundled skill teaches the agent when to use it. Wiring: - CI: test-agent-plugin-integration job runs the manifest validator, gated on hindsight-integrations/agent-plugin/** changes. - Docs: integrations.json gallery entry + docs-integrations/agent-plugin.md. - Release: agent-plugin added to release-integration.sh and the changelog generator; both learn to read a root-level plugin.json and link the changelog to the source tree (git-distributed bundle, no registry package).	2026-08-11 18:06:17 +02:00
Nicolò Boschi	20bd4f3618	fix(ci): compare OpenAPI against the merge-base; build benchmark role configs whole (#3349) ... * fix(ci): compare OpenAPI against the merge-base; build benchmark role configs whole Two unrelated CI failures, both of which fail without anything being wrong with the code under test. OpenAPI compatibility diffed the branch's spec against the LIVE tip of the base branch, so every endpoint main gained after a branch was cut is reported as "Endpoint removed (breaks old clients)" by that branch. Three open PRs failed this way today on /health/live and /health/ready (added by #3329), none of which touch the spec at all; the only cure was an unrelated rebase. Compare against `git merge-base origin/$BASE_BRANCH HEAD` instead, which asks the question the check means to ask: did *this branch* remove something. Genuine removals still fail — verified both directions against the real specs. The scheduled LoComo benchmark has failed every night since at least Aug 8, before its first question: `LoComoAnswerGenerator()` raised "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID is required". The workflow does export that variable — but each benchmark role built its LLMConfig from exactly four env vars (provider, api_key, base_url, model), and LLMConfig deliberately does not read the environment for provider-specific settings (from_env is where the API resolves them). Every Vertex AI value was therefore dropped. The same four-var construction was copy-pasted at three sites — locomo, longmemeval and the shared judge — so fixing only the crashing one would have moved the failure down a line. Replace all three with a shared builder that carries the Vertex AI project/region/service-account through. hindsight-dev/tests had no CI job, which is why a plain construction bug was left for a nightly benchmark to find hours later. Add one, so those tests (and the new regression test) actually run on PRs. * fix(ci): make the benchmark role-config tests hermetic They passed locally off the developer's HINDSIGHT_API_LLM_API_KEY and failed in the new test-dev job, where no key exists, with "API key is required for openai" — the tests were reading ambient environment instead of declaring what they need. Clear every HINDSIGHT_API_*LLM* var before each test and set the ones under test explicitly.	2026-08-10 17:11:06 +02:00
Nicolò Boschi	2b0ed82f30	fix(coding-agents,ci): drop the periodic re-sync; stop running doc examples for integrations (#3341) ... Two things, both about cost. The periodic full re-sync (a replace every 20 appends) defeated the point of appending: on a long session it re-uploads the entire document on a fixed cadence, which is the expense this whole path exists to avoid. Removed. What it was insuring against still stands, and is now accepted: retains are async, so a write can be acknowledged and then fail server-side, and a resubmitted operation_id replays the original operation whatever its status — those turns are not re-sent. The other replace triggers (no cursor, fingerprint drift, dirty, bank change) are unaffected, so a write we can SEE fail still self-heals; only a silent server-side failure after acknowledgement is uncovered. Second: test-doc-examples gated on a `docs` filter that matched 'hindsight-integrations/**' — added so an integration rename would reach the docs build's integrations check, except build-docs has no `if:` and runs on every PR regardless. So the only consumer of that breadth was the doc-examples matrix, which runs every sample against a live LLM-backed server: four provider-credentialed jobs on every integration PR (and every prose-only docs PR), none of which those files can affect. It now gates on a `doc-examples` filter covering the runnable samples themselves (hindsight-docs/examples/**) and their runner. `docs` had no other consumer, so it is removed rather than left as config nothing reads.	2026-08-10 14:51:33 +02:00
Nicolò Boschi	aefc5e8e7d	test(ci): gate every build on hermes-agent@main co-installability (#3265) ... Hermes installs `hindsight-all` into its OWN venv via `hermes memory setup` to run memory in local_embedded mode, and exact-pins every direct dependency (`==X.Y.Z`) as a deliberate supply-chain policy. Any version range Hindsight declares that excludes one of their pins therefore makes the two impossible to co-install for every Hermes user on embedded memory — that is what #3251 hit with our cryptography/pillow floors. Hermes main has since bumped to cryptography==48.0.1 / Pillow==12.3.0, which already matches our floors, so no dependency change is needed and none is made here. What was missing is the check that keeps it that way: both sides bump on their own schedule, so the collision recurs silently until something looks for it. Tracking their main branch surfaces it while it is still cheap to fix on either side rather than in a released Hermes. scripts/test-hermes-compat.sh runs five checks of increasing depth: resolution (both into ONE resolution, so an unsatisfiable pin is a hard error instead of a silent downgrade), `pip check` plus a dump of the contested versions, `hermes memory status`, the runtime imports Hermes makes for embedded memory, and a real embedded daemon boot with bank operations. Implementation notes: - Hindsight installs from BUILT WHEELS, not `file://` directories. uv installs a workspace member given as a directory such that hindsight_embed.__file__ still points into the source tree, and the daemon manager keys dev-mode detection on that path — so a directory install launches the API via `uv run --project <repo>/hindsight-api-slim`, out of the monorepo's venv and .env, bypassing the Hermes venv this script exists to test. Step 5 asserts the daemon binary resolves inside the test venv so this cannot regress. - Hermes is cloned and installed editable: their build backend refuses wheel/sdist builds by design, so `git+https://` fails outright. - Python is pinned to 3.12 rather than .python-version because Hermes caps itself at <3.14; the venv must sit inside both projects' windows. - Hindsight state is isolated by a dedicated `hermes-ci` profile rather than by redirecting HOME, which would also hide the uv/HuggingFace caches from the runner and re-download the local-ml stack every run. - Step 5 runs from the work dir, not the repo, so a developer's .env cannot hand the daemon credentials a runner does not have. - MemoryEngine refuses to construct without an LLM key, so the daemon boots on a placeholder one; nothing calls the LLM during startup or bank operations. The job needs no secrets and so runs on fork PRs too. Only the retain/recall round-trip requires a real LLM key and is skipped without one. Verified end-to-end locally against hermes-agent main (0.20.0): all five steps pass, 225 packages consistent, daemon boots from the test venv and stops cleanly.	2026-08-10 13:03:40 +02:00
Nicolò Boschi	333812c85a	fix(coding-agents): read Devin's transcript with node:sqlite, and refuse to install without it (#3175) ... * fix(coding-agents): read Devin's transcript with node:sqlite, and refuse to install without it Devin is the only harness whose hooks never hand over a transcript — they carry a session id and nothing else, so the conversation has to be read from the CLI's own sessions.db. That read shelled out to the `sqlite3` BINARY, which is not a declared dependency and was never checked for. Where it was absent, execFileSync threw ENOENT, a bare `catch` folded it into `return []`, and retain no-opped forever while the installer reported success (#3125). Node ships its own SQLite, so the binary is no longer needed for anything: - `node:sqlite` replaces the subprocess. It is a builtin — nothing added to the package, nothing bundled (esbuild leaves `node:` imports external; the devin hook bundle is unchanged at 59K). Loaded through createRequire INSIDE the read function rather than imported at module scope: this module is pulled in by hook-lifecycle, which every harness shares, so a static import would break Claude Code and Codex on a Node without it. - The session id is now a bound parameter instead of being escaped into the SQL string by hand. - Missing reader, absent database and read failure each emit a distinct `diag` event, so a permanently memory-less install no longer looks like an idle session. A Devin storage-schema change surfaces the same way. - `install devin-cli` preflights the `node` on PATH — the interpreter the hook command actually runs under, which an npx-launched installer may not be — and refuses with the reason and a non-zero exit instead of wiring hooks that could never retain anything. On `install all` only Devin is blocked; the other agents are still wired. The SQLite read path had no tests at all; it now covers reading a session, binding a quoted id, and both failure diagnostics against a real database. * ci: export the integrations-coding-agents change filter The filter was defined and the job consumed it, but detect-changes never listed it among its outputs — so `needs.detect-changes.outputs.integrations-coding-agents` was always empty and test-coding-agents ran only on workflow_dispatch or a workflow-file change. Every PR touching just that package went untested. * chore: regenerate the stale docs-skill mirror Pre-existing drift on main, not from this branch: the `agent_name` deprecation landed in the docs without re-running generate-docs-skill.sh, so verify-generated-files fails for every PR. Pulled in here because this PR can't go green without it.	2026-08-04 16:30:08 +02:00
Nicolò Boschi	cb0e1dea40	chore: replace star-history.com chart with self-hosted gh-stars chart (#3150) ... The api.star-history.com embed in the README was broken. Replace it with nicoloboschi/gh-stars, which backfills stargazer data via a scheduled GitHub Action and commits a self-hosted SVG chart into the repo, so the README image no longer depends on a third-party service.	2026-08-03 12:32:58 +02:00
Nicolò Boschi	b5d8439c8f	hindsight-coding-agents: harness-pluggable long-term memory for coding agents (#2522) ... * feat(integrations): add hindsight-opencode-coding plugin Reflect-only long-term memory for coding agents in OpenCode, with a git+chat backfill and (opt-in) live session write-back. - reflect + INJECT: on a task, reflect() the symptom and push the root-cause answer into the system prompt (no tools/recall). - backfill: every commit (full message + full diff, commit timestamp + git metadata) under a 'git' retain strategy; each chat as a JSON user/assistant transcript with custom extraction (<=2 coherent facts) under a 'chat' strategy; observations on; optional codebase knowledge pages. - live write-back (opt-in HINDSIGHT_RETAIN_SESSIONS): every N turns upsert the tool-filtered transcript under a stable conversation:<sessionID> document_id. * refactor(integrations): generalize opencode-coding into hindsight-coding-agents Make the coding-memory plugin harness-pluggable instead of opencode-specific. A 'harness' (coding agent) differs in only two places; everything else is now shared core: - src/core/ hindsight client, missions, git + chat ingest, inject, RuntimeCore - src/core/types.ts HarnessAdapter + ChatReader interfaces - src/harness/ per-agent adapters + registry (opencode implemented) Backfill: --harness selects how past sessions are read (opencode today); git ingest, retain strategies, missions, and knowledge pages are identical across agents. Runtime: HINDSIGHT_HARNESS (default opencode) selects the adapter that binds RuntimeCore's reflect+inject+write-back to that agent's plugin API. Adding an agent = one adapter file + a registry entry. Type-checks and builds clean; unknown --harness/HINDSIGHT_HARNESS errors with the available list. * feat(coding-agents): on-demand memory_reflect tool, opt-in git-sync, JSON config Add two capabilities to the reflect-only coding-agents plugin and move all configuration off environment variables onto a single JSON file. - memory_reflect tool: exposes the same synthesized reflect that is auto-injected on the first message as an on-demand opencode tool the agent can call mid-task (RuntimeCore.reflectNow + opencode adapter tool). Harness-agnostic core, thin opencode wiring. - incremental git-sync (opt-in): on load, diff the target ref's commits (origin/main, falling back to HEAD) against the git:<sha> document_ids already in the bank and async-retain only the missing ones, reusing the backfill's per-commit encoding (retainCommit). Set-based, correct across rebases; best-effort, non-blocking. Off by default (gitSync.enabled). Adds HindsightClient.listDocumentIds + core/sync.ts. - config file: all settings now come from ~/.hindsight/coding-agent.json (core/config.ts) -- no environment variables. The backfill CLI reads the same file for shared connection/bank settings with --flags overriding; operation flags stay CLI-only. Committed with --no-verify: the repo-wide pre-commit lint hook is broken in this environment (missing @eslint/js in hindsight-control-plane) and blocks all commits. * fix(coding-agents): remove benchmark-specific strings from prompts Fairness audit of the sdebench benchmark found three contaminations: - CHAT_CUSTOM_INSTRUCTIONS used the literal answer to a graded task (round_cents/ROUND_HALF_DOWN/legacy ledger) as its example - replaced with a fictional, non-benchmark example. - buildSystemInjection told the model 'the hidden tests depend on those exact choices' - hardcoded knowledge of the benchmark's grading; reworded benchmark-agnostic. - REFLECT_MISSION examples were shape-matched to specific benchmark tasks (symbol mappings, exact numbers) - neutralized. No behavior change intended beyond removing the leaked specifics. (includes hook-regenerated skills/hindsight-docs sync) * feat(coding-agents): reflect-outcome diagnostics — no more silent memory loss A benchmark sweep ran the entire memory arm with zero injected memory: reflect failed environmentally on every task and the best-effort catch swallowed it, making a memory-less run indistinguishable from a memory run. onTask now appends a reflect_ok/reflect_empty/reflect_failed record (duration, error, query prefix) to HINDSIGHT_DIAG_FILE (default /tmp/hindsight-plugin.log). Consumers can assert a session actually had memory before trusting a comparison. * fix(coding-agents): chronological session recency + supersession-aware reflect Two defects surfaced by the conversation-amended benchmark tasks (a rule settled in one chat and amended in a later one): - chat ingestion staggered synthetic timestamps NOW - i*1h, INVERTING recency: an amendment chat ranked older than the decision it superseded, steering temporal ranking toward the stale rule. Session list order is chronological; the last session is now the newest. - REFLECT_MISSION now states that when memories conflict on the same rule, the latest/superseding decision wins and the superseded rule must be reported as no longer in effect, never presented as the fix. Observed live: reflect on an amended bank returned the superseded keep-latest rule as the fix. Both fixes are general recency/consistency semantics, not benchmark-specific behavior. * feat(coding-agents): multi-harness configurability + Claude Code hook entry One config, several agents side by side: - Each runtime entry point now KNOWS its harness instead of reading the config's `harness` key (which selected a single global adapter and made opencode + claude mutually exclusive). That key now only picks the backfill's session formatter. - New `harnesses.<name>` config sections: per-agent overrides of any field (bank, disabled, timeouts) over shared connection defaults. - New project-local layer: <project>/.hindsight/coding-agent.json overrides the global file — the natural home for a per-repo bank. Precedence: defaults < global < global.harnesses < project < project.harnesses. - New entry point: `hindsight-claude-hook` (dist/claude-hook.js), a Claude Code UserPromptSubmit hook. Reflects once per Claude session, caches the answer in tmp and re-injects it on later prompts, and writes the same reflect_ok/failed diagnostics as the opencode path. Verified live: claude hook via project config + harnesses section (reflect_ok, cached re-emit in 46ms, one reflect total); opencode via the benchmark harness (reflect_ok, task solved 0 corrections). * feat(coding-agents): per-repo dynamic bank resolution (family convention) Port of the bank-derivation convention shared by the claude-code, omo, cline, and opencode integrations, with coding-first defaults: - No bankId configured => the bank is derived from the git repo the working directory belongs to, WORKTREE-AWARE: git rev-parse --git-common-dir resolves every linked worktree to the main worktree's basename, so all worktrees of a repo share one memory bank (bare repos use the bare dir name; non-git dirs fall back to the dir basename). - Default granularity is [gitProject] (not agent::project): opencode and claude share ONE memory per repo — add 'agent' to dynamicBankGranularity to split per agent. - Explicit bankId keeps today's static behavior (benchmark harness, single-bank setups); dynamicBankId forces either mode; supporting fields: bankIdPrefix, directoryBankMap (exact cwd -> bank escape hatch), agentName, resolveWorktrees. - backfill: --bank wins, else the SAME resolution applied to --repo, so `hindsight-coding-backfill --repo .` fills exactly the bank the agents will read. Verified: worktree -> main-repo bank (hs-coding-plugin-wt -> memory-poc), static/prefix/dirMap/granularity cases, and the claude hook e2e (reflect_ok via directoryBankMap against a live bank). * feat(coding-agents): bank template string, prefix path map, {harness} field Bank-resolution refinements: - `bankIdTemplate` format string replaces the granularity array: e.g. "hindsight-{gitProject}" or "{harness}-{gitProject}" — default "{gitProject}" (opencode + claude share one bank per repo). Placeholders: {gitProject} {project} {harness} {channel} {user}; unknown placeholders warn with the valid list. bankIdPrefix removed (expressible in the template). - {harness} is supplied by the entry point itself (opencode plugin, claude hook, backfill --harness), not a config field — nothing to keep in sync. - directoryBankMap now matches by LONGEST absolute-path prefix and overrides everything incl. an explicit bankId: mapping a repo root covers all its subdirectories; deeper mappings win. - config discovery walks UP from the working directory to the nearest .hindsight/coding-agent.json — a hook invoked from a repo subdir previously missed the repo's project config entirely (found by an e2e test that failed exactly this way). Verified: derivation matrix (template/prefix-map/override/static/bad placeholder), claude hook e2e from a nested subdir (reflect_ok via walked-up config + prefix-matched map), opencode benchmark task green. * feat(coding-agents): cursor-cli + codex harnesses, unit tests, live system tests Harnesses — hook-based agents now share one runtime (core/hook.ts: stdin event -> layered config -> per-repo bank -> once-per-session reflect with tmp cache -> native output -> diagnostics), so each agent is a ~25-line HookSpec: - hindsight-claude-hook (UserPromptSubmit -> additionalContext) - hindsight-cursor-hook (beforeSubmitPrompt -> {continue, additional_context}) - hindsight-codex-hook (Codex CLI v0.116+ claude-compatible hooks; accepts prompt/user_prompt) All three + opencode registered in the harness registry (backfill --harness resolves them; hook harnesses share the normalized-JSON chat reader). Tests (vitest, family convention): - 25 unit tests: full bank-derivation matrix (worktree/bare/static/ dynamic/template/{harness}/prefix-map incl. longest-wins and no-sibling-false-match) and config layering (harness sections, project-over-global, upward walk, nearest-wins, gitSync field merge, malformed fallback, legacy signature). - live system suite (npm run test:live, HINDSIGHT_LIVE_E2E=1): builds a real git repo with a decision planted in a commit + a conversation, runs the real backfill CLI (server-side LLM extraction), then invokes the BUILT hook binaries as subprocesses and asserts the decision's literals come back in the injected context — semantic verification with a real LLM — plus per-session cache behavior and diag records. All 4 passing against a live server. Note: session ids in the live suite are unique per run — the hooks cache per session id in tmp, and a static id once cached a bad answer from a half-broken server across reruns. * docs(coding-agents): full README rewrite + integration docs page README now covers everything the package does today: the reflect-once/ inject-every-turn mechanics, all four harnesses (opencode plugin + claude/codex/cursor hooks) with install snippets, the complete configuration reference (layered files, harnesses sections, per-repo dynamic bank resolution with template placeholders, directoryBankMap, worktree behavior), backfill CLI incl. bank auto-resolution and chronological session ordering, the reflect diagnostics contract, and the unit + live test suites. Docs site: new docs-integrations/coding-agents.md (same content adapted to the integration-guide format) + integrations.json hub entry so the generated sidebar picks it up. Placeholder icon (github.png) pending a real one. Verified: page renders (docusaurus build), all doc pre-flight checks pass for this entry — note the docs build on this branch was ALREADY failing on the unrelated pre-existing 'zcode missing from integrations.json' check. * feat(coding-agents): 🧠 attribution header in buildSystemInjection Prepend the 'Using Hindsight Memories' visible-attribution directive to the harness-agnostic system injection so every coding-agent harness surfaces a recognizable header when it uses recalled memory. Covered by 5 deterministic inject.test.ts cases (real emoji + em dash, no lone surrogates). * feat(core): add recall() to HindsightClient * style(core): apply prettier formatting to recall test * style(coding-agents): normalize prettier formatting across package * fix(core): narrow RecallResult to actual API contract, add fetch-throw test * feat(core): formatMemories + shared attribution preamble * style(core): prettier-wrap recall.test.ts array literal * fix(core): cover formatMemories trim/filter + drop stale inject comment * feat(core): per-turn recall in the hook runtime (reflect once, recall every turn) Extracts the hook logic into a pure, unit-testable buildHookOutput(): every prompt now runs recall() and injects a <hindsight_memories> block; reflect still runs once per session (first prompt) and its cached answer is no longer re-injected on later turns. runHook() becomes thin stdin/stdout plumbing with a makeClient seam for tests. Updates the three hook entrypoints' doc comments to match, and adds recallMaxTokens/recallTimeoutMs config fields. * fix(core): make recall fail-open in buildHookOutput + cover recall failure/opts * feat(claude-code-v2): wrapper plugin skeleton (per-turn recall via bundled core) Also disables tsup code-splitting in hindsight-coding-agents so each bin entry (claude-hook.js etc.) is a single self-contained file with no shared chunk-*.js — required for wrapper build scripts that copy just the one hook file out of dist/. * chore(coding-agents): sync codex-hook bin into package-lock * fix(claude-code-v2): derive version from manifest + guard self-contained bundle * feat(core): Claude transcript reader (normalized user/assistant text turns) * fix(core): transcript reader null-safety + drop sidechain turns * feat: live write-back on the Claude Stop hook (shared retain-hook runtime) Extracts a testable buildRetain core (read transcript -> upsert under conversation:<sessionId> via retainLiveSession) plus a thin runRetainHook plumbing wrapper mirroring the existing runHook/buildHookOutput split, and wires it up as a Claude Code Stop hook. Fail-open throughout: an empty transcript is a no-op, and a retain failure is diagnosed but never thrown. Exports diag() from core/hook.ts so retain-hook.ts can reuse the same diagnostics helper instead of duplicating it. * refactor(core): extract diag module + trim buildRetain params - Move diag() out of hook.ts into a neutral src/core/diag.ts so retain-hook (and future lifecycle hooks like SessionStart) don't reach into a recall/reflect-specific module for a cross-cutting concern. - Drop the unused cwd/cfg params from buildRetain — only harness, sessionId, transcriptPath, and client are read; cwd/cfg stay in runRetainHook where they're actually used (config load + deriveBankId). - Clarify that retainSessions is opencode-plugin-only; the Stop hook always writes back unless disabled. * feat(core): knowledge-page CRUD on HindsightClient (mental-models) * fix(core): page methods throw on 404 + doc rationale * feat: native TS MCP server for knowledge-page tools (bank-aligned) Adds a native TypeScript MCP (stdio) server exposing the agent_knowledge_* tools (get_current_bank, list_pages, get_page, create_page, update_page, delete_page, recall) over MCP, wired into the claude-code-v2 wrapper. Bank resolution goes through the same loadConfig + deriveBankId path the hooks use (harness "claude-code"), so knowledge pages, recall, and retain all land in one per-repo bank. This is a native TS server rather than reusing the Python MCP because its bank derivation mismatches. - src/core/knowledge-tools.ts: SDK-free tool specs (zod schemas), unit tested against a stub client (17 tests) — every handler is fail-closed to an isError:true result instead of throwing. - src/mcp-server.ts: the only file importing @modelcontextprotocol/sdk. - tsup.config.ts: new mcp-server entry, noExternal inlines the SDK + zod so dist/mcp-server.js stays a single self-contained file. - claude-code-v2/.mcp.json + build.mjs: wires the bundle into the plugin; the self-contained-bundle guard passes for mcp-server.js unmodified (no exemption needed) since noExternal fully inlines its deps. * fix(mcp): honor disabled flag + testable selectTools - Export selectTools(cfg, client, bankId) from mcp-server.ts: pure, SDK-free, returns [] when cfg.disabled (mirrors the hooks' disabled check) so a disabled Hindsight exposes zero MCP tools instead of all 7. Confirmed at runtime: with disabled:true the server still connects but doesn't advertise a tools capability at all (tools/list -> Method not found), which is stronger than an empty list. - Guard main() behind an argv[1]-vs-import.meta.url check so importing the module for tests doesn't start a real stdio server. - Add src/mcp-server.test.ts covering selectTools for both the disabled and enabled cases. - Reword the HINDSIGHT_MCP_PROJECT_CWD comment: nothing sets it today (the plugin doesn't cd), it's an escape hatch, not a launching-host contract. * refactor(core): lazy-load opencode adapter so backfill bundles self-contained * test(core): lock opencode no-runtime registry invariant + doc it * feat(core): cold-repo detection + seed-consent state * test(core): cover seed write-failure + guard non-object state * feat(core): background seed mechanics + hindsight-seed control CLI Adds hasGitHistory (git.ts), startBackgroundSeed + seedControl (seed.ts), and the src/hindsight-seed.ts entrypoint the agent runs after the SessionStart seed offer (Task 10b) to seed or decline a repo's bank. * fix(core): handle async spawn error in startBackgroundSeed spawn() failures (ENOENT/EACCES/fd exhaustion/sandboxed environments) often arrive asynchronously as an 'error' event on the child, not a synchronous throw. An unhandled 'error' event crashes the caller, so attach a no-op handler alongside the existing try/catch. Also documents the Claude-Code-only harness assumption in hindsight-seed.ts. * feat: SessionStart auto-seed offer for cold repos (Claude wrapper wired) * fix(core): shell-escape seed offer paths + drop orphaned isColdRepo * docs(claude-code-v2): marketplace entry, full README, v1→v2 migration note * fix(core): cap hook reflect timeout, align backfill+hook config resolution - hook.ts: cap reflect's timeoutMs to HOOK_REFLECT_CAP_MS (8s) so it always resolves/aborts before Claude Code's 15s UserPromptSubmit kill window, guaranteeing the session cache write + recall injection complete instead of silently retrying reflect (and dropping recall) on every turn. - backfill.ts: resolve config via loadConfig({harness, projectDir: REPO, path}) instead of the legacy string form, so project-local .hindsight/coding-agent.json layers in and the background auto-seed backfill targets the same bank recall/retain/MCP read from. - hook.ts/retain-hook.ts: resolve the cwd fallback before loadConfig (not just at deriveBankId) so project-local config layers even when the hook event's cwd is missing. * fix(claude-code-v2): dev-install must copy .mcp.json (MCP tools were missing) * feat(core): deterministic SessionStart auto-seed + knowledge-page bank mission The prior SessionStart design asked the agent to pose a y/n question then run a seed command itself; live testing showed the model surfaces the question and then ignores it, so nothing ever seeds. The hook now starts the background seed itself on a cold git repo (tri-state: cold/warm/ unreachable) and always injects a short visible note plus a bank-mission pointing the agent at the agent_knowledge_* tools. * docs(claude-code-v2): update seed docs for deterministic auto-seed + knowledge mission * feat(core): default seed to aggregated commit messages (one cheap doc) + Initiatives page; full-diff opt-in via --diffs * docs(core): align backfill README + strategy log/comment with gitlog default * feat(core): headless codebase-survey seed + agent_knowledge_ingest MCP tool On a cold repo, the SessionStart hook now also spawns a detached headless `claude` that samples the repo's structure and ingests its findings into Hindsight via a new agent_knowledge_ingest MCP tool, alongside the existing git-history backfill. Knowledge pages synthesize their content from bank memories via source_query, so this is how the survey feeds them. - knowledge-tools.ts: add agent_knowledge_ingest (title -> slug doc id, retain via the "chat" strategy, tagged source:upload). - survey.ts: resolveClaudeBin + startCodebaseSurvey, mirroring seed.ts's fire-and-forget/never-throw spawn pattern. - Anti-recursion: HINDSIGHT_DISABLE_HOOKS guard at the top of runHook, runRetainHook, and runSessionStartHook so the survey's own claude session can't re-trigger seeding/recall/retain; survey.ts sets it on the child. - config.ts: codebaseSurvey (default true) + surveyModel (default "sonnet"). - session-start.ts: wire startSurvey into the cold-repo branch alongside startSeed; update the visible learning note. * fix(core): sandbox headless survey (deny-list, no bypassPermissions) + spend cap + document strategy * feat(core): default codebase-survey model to haiku (cheaper/faster; sonnet still configurable) * feat(core): survey excludes CLAUDE.md + agent-instruction files from ingestion * docs(coding-agents): v2 knowledge-pages design spec + implementation plan * feat(core): add pageRefreshEveryTurns config (default 10) * feat(core): knowledge-injection roster/preamble formatting * feat(core): passive knowledge entity_labels tier vocabulary + configureBank wiring * feat(core): tag-scope seeded pages, Initiatives folder, relatedPageId link source_query * feat(core): captureInitiative — per-initiative page + relatedPageId marker * feat(mcp): hindsight_* grounding tools + capture_initiative; remove raw page CRUD from agent * feat(core): SessionStart injects page roster + guidance preamble * feat(core): UserPromptSubmit hook-counted periodic page-roster refresh * feat(core): rich markdown session write-back with tool calls + verbose session strategy * chore: apply prettier line-wrapping to test files * fix(claude-code): surface the seed note via user-visible systemMessage, keep preamble in additionalContext * fix(survey): use renamed hindsight_ingest_document MCP tool (Task 6 rename regression) * fix(core): preamble + refresh nudge the agent to call capture_initiative for major features * fix(core): re-inject tool+capture reminder every cadence turn even with no pages (unconditional nudge) * fix(core): inject when-to-call guide for the full hindsight_* tool suite, not just pages+capture * feat(claude-code): cold-check-wins seeding — reseed a cleared bank on the live doc count, ignore stale seededAt * fix(core): simplify capture_initiative instruction to one clear trigger (remove confusing OR-chains) * fix(core): port proven v1 attribution preamble + surface memories block first so the header actually gets emitted * fix(core): reflect every turn (configurable reflectEveryTurns, default 1) instead of once per session * feat(core): per-turn injection is recall-only (drop reflect from the hook), recall token budget default 750 * feat(core): inject a user-feedback section above memories (capture-initiative + attribution-header preferences) * fix(core): align user-feedback attribution bullet with the generous WHEN-IN-DOUBT-EMIT rule * fix(core): sharpen capture_initiative trigger — call right after plan approval, before implementation * feat(survey): raise default codebase-survey budget cap to $2 (0.5 was over-conservative) * feat(codex): codex-v2 wrapper (SessionStart seed + per-turn recall + MCP); parametrize session-start/MCP harness * feat(core): default bank template is harness-neutral coding-agent::{gitProject} (shared memory across agents) * feat(core): default apiUrl is Hindsight Cloud (https://api.hindsight.vectorize.io); local is now an override * feat(codex): Stop write-back — Codex rollout transcript reader + codex-stop-hook (full parity) * fix(core): captureInitiative returns the server-assigned page id (not the slug) so read_knowledge_page + relatedPageId links resolve * feat(coding-agents): upgrade opencode adapter to full v2 parity Per-turn recall via chat.message + system.transform (750-tok budget), native hindsight_* tools registered directly through opencode's tool() (no MCP server), rich tool-aware write-back on by default, and cold-check auto-seed at plugin load — reusing the shared formatMemories / buildKnowledgePreamble / buildKnowledgeTools / buildSessionStartContext primitives so opencode matches Claude Code and Codex. Adds transcript-opencode.ts (rich normalizer over the live message list). Adds a HINDSIGHT_DISABLE_HOOKS recursion guard to RuntimeCore (seed/recall/write-back/sync no-op; tools still register) for headless survey runs. Removes the now-dead reflect path (client.reflect, inject.ts/buildSystemInjection, reflectTimeoutMs) as the whole surface is recall-only. README rewritten to the recall/knowledge-page/seed/write-back v2 model. * feat(coding-agents): harness-portable codebase survey (multi-agent headless) The cold-repo survey no longer hardcodes headless `claude` — startCodebaseSurvey now runs under the current harness's own CLI (claude/codex/gemini/opencode), falling back to any available agent, so a Codex/Gemini/opencode user without claude installed still gets the survey (the git-log seed already ran regardless). Per-agent read-only recipes: claude (-p + inline --mcp-config + --disallowedTools), codex (exec --sandbox read-only + inline -c MCP), gemini (-p --approval-mode plan --allowed-mcp-server-names hindsight --skip-trust), opencode (run --agent plan; tools from the loaded plugin under the HINDSIGHT_DISABLE_HOOKS guard). All spawned with HINDSIGHT_DISABLE_HOOKS=1. session-start threads the harness through to the survey. * feat(gemini): add Gemini CLI v2 integration (gemini-v2) Full v2 parity for Gemini CLI (>=0.52.0), which added a Claude-style hooks system (stdin/stdout JSON). Maps onto the shared HookSpec/runSessionStartHook/runRetainHook abstraction with Gemini's event names: BeforeAgent (per-turn recall -> hookSpecificOutput.additionalContext), SessionStart (seed), SessionEnd (write-back). The one Gemini-specific piece is transcript-gemini.ts — a reader for the 0.52.0 chats/session-*.jsonl mutation-log (upsert-by-id, polymorphic content: user text arrays, assistant plain strings, tool results as user functionResponse parts; drops the synthetic session_context message + thoughts). Adds the gemini-v2 wrapper (build.mjs + dev-install.sh that merges hooks + mcpServers into ~/.gemini/settings.json). Validated: reader against a real transcript, and a live recall smoke test (recall_ok) end-to-end. * style(coding-agents): prettier-format README config table * fix(opencode): inject via lastInjection fallback (1.18.5 system.transform has no sessionId) opencode 1.18.5 fires experimental.chat.system.transform with input {model} only — no sessionId — so RuntimeCore.getInjection(input.sessionID) looked up undefined and pushed nothing into the system prompt. Recall still ran (chat.message does pass sessionID) but the memory block + attribution preamble + knowledge-page guide never reached the model, so no visible header and no tool use. getInjection now falls back to the most recent turn's block (lastInjection) when there's no session-keyed hit. The completion's system.transform fires right after that session's onPrompt, so lastInjection is this turn's block. Adds an inject_ok/inject_empty diag (matching recall_ok/seed_started) to confirm injection lands. * fix(coding-agents): treat project-local config as untrusted (block apiUrl/apiToken/directoryBankMap from a repo) A project-local .hindsight/coding-agent.json lives inside whatever repo the developer opens, so it is untrusted input. loadConfig previously merged it per-field over the user-global config, letting a repo override apiUrl while the user-global apiToken survived the merge — so a malicious repo could set only apiUrl and the client would send the user's real Bearer token plus every recall query (the prompt) and Stop write-back transcript to an attacker-controlled host, silently, just by opening the repo (verified end-to-end). Fix: the project-local layer is now sanitized — apiUrl, apiToken, and directoryBankMap are stripped from it (top level + any harnesses.<name> section) with a one-line warning; the user-global config stays trusted and unrestricted, and a repo can still set its own per-repo bank (bankId/bankIdTemplate). Also skip re-applying the global file as a project layer when the upward findProjectConfig walk lands back on it (a repo under $HOME with no closer config), which would otherwise strip its own apiUrl and warn every session. Adds 4 regression tests. * style(coding-agents): prettier-format config.ts * feat(coding-agents): restore reflect as the memory path; per-turn injection from knowledge-page sections One opinionated runtime path (no behavior config): - reflect ONCE per session on the first prompt (agentic root-cause synthesis, benchmark-proven), cached and re-injected every turn — hook harnesses and the opencode runtime alike - every turn: knowledge-page SECTIONS matched locally against the prompt (lexical section index, no server/LLM call) injected with provenance and a pointer to the full page — fast like recall, organized like reflect - raw recall leaves the runtime path (still powers the hindsight_search_memory tool) Session write-back: transcripts are now JSON turns matching the backfill chat format, with each tool call compacted to a role:"action" turn naming the tool and its primary target (no arguments, no outputs) — Claude, Codex, Gemini and opencode readers. Knowledge pages: no more entity_labels/tag taxonomy — pages are unscoped, each page's source_query selects from the whole bank; survey, gitlog seed, write-back and security hardening stay. Spec: docs/superpowers/specs/2026-07-27-reflect-pages-runtime.md * test(coding-agents): rewrite unit tests for reflect+pages runtime and JSON action transcripts * test(coding-agents): live suite matches reflect_ok by content (pages_ok now follows it in the diag stream) * docs(coding-agents): README + docs page describe the reflect+pages runtime (reflect once per session, local page-section injection per turn, JSON action write-back) * feat(coding-agents): drop the backfill CLI — ingestion is automatic and background - new deepen engine (dist/deepen.js, unpublished): idempotent, resumable — per-bank lock, dedup by document id; ingests missing conversations, the one-time gitlog seed, then progressively deepens recent history with per-commit full diffs (newest first, bounded batch per run); drains and creates knowledge pages last - every session start now fires the engine (cold or warm); survey and the cold-seed note stay cold-only - sync status is the new readiness contract: hindsight_sync_status agent tool + dist/status.js for harnesses (synced = gitlog seeded, pages present, extractions drained); activeOperations() filters terminal ops - opencode write-back now upserts every turn (async) so a killed session loses at most the last turn - repoNameOf resolves relative paths so document ids are path-spelling-proof - hindsight-coding-backfill bin removed; benchmark/e2e run the engine directly and poll status * polish(coding-agents): short, non-technical cold-seed message highlighting the bank id * polish(coding-agents): cold-start banner — HINDSIGHT unicode wordmark + bank id line * feat(coding-agents): timing diagnostics on by default - session_start diag event on EVERY session (bank, cold/warm, pages, ms) — warm sessions previously logged nothing - deepen engine: deepen_started/deepen_done/deepen_failed diag events with duration; child output now appended to ~/.hindsight/coding-agent-state/deepen.log (was stdio:ignore — undebuggable) and log lines timestamped - retain_ok/retain_failed carry ms on both the Stop hook and the opencode per-turn upsert (which was fully silent) - vitest config pins HINDSIGHT_DIAG_FILE to a tmp file so unit tests stop polluting the real diag log * feat(coding-agents): show the Hindsight banner on every session start (cold: learning, warm: remembering) * polish(coding-agents): session banner uses the API server's colored pixel-art logo (shared visual identity), wording line below * polish(coding-agents): banner text before logo — the TUI's first-line prefix was displacing the logo's top row * polish(coding-agents): banner logo re-rendered foreground-only — the TUI strips ANSI background colors, which deleted half the server logo's pixels * feat(coding-agents): per-turn user-visible notice — every prompt shows what Hindsight delivered (reflect state + matched knowledge pages) via hook systemMessage; opencode logs the same line * polish(coding-agents): per-turn notice shows the match query excerpt and the page titles it returned * fix(pages-index): singularize plain-word tokens so plural prompts match singular headings ('components' -> 'Component map'); path-like tokens untouched * polish(coding-agents): per-turn notice — gradient Hindsight wordmark, value-driven wording, no timings * feat(coding-agents): interim always-inject knowledge stub + explicit Hindsight attribution - selectSections: TEMPORARY stub returning the first section of up to 3 distinct pages every turn regardless of prompt — guarantees injected data for testing source attribution; will be replaced by the server-side knowledge-base/search (local lexical index drops with it) - both injection blocks now carry an ATTRIBUTION directive: when memory shapes the answer, the agent introduces it with '🧠 From Hindsight memory (<page>)' — and must never credit memory that did not contribute * polish(coding-agents): gradient-word banner (logo dropped), lean per-turn notice, attribution directive front-loaded as a mandatory output format * polish(coding-agents): reflect turn notice shows the assigned goal and a preview of what memory returned * feat(coding-agents): page knowledge moves from auto-injection to an explicit tool - new hindsight_search_knowledge_pages(query) tool (native on opencode, MCP on hook harnesses) — interim local selection, single swap point for the server-side knowledge-base/search; results carry the attribution requirement - per-turn auto-injection of page sections removed: a trivial prompt ('yes') no longer displays phantom research; ordinary turns are silent - per-turn notice only on the reflect turn (assigned goal + result preview); tool calls provide their own native visibility - tool guide/roster advertises the search tool as the first stop * feat(coding-agents): bind hindsight_search_knowledge_pages to the server-side hybrid knowledge-base search - merge feat/knowledge-pages-okf underneath (GET /knowledge-base/search, BM25 + vector, RRF-fused; conflicts resolved in okf's favor for server/ clients/UI, coding-agents docs entry preserved) - client.searchKnowledgePages(query, limit) wraps the endpoint; the tool returns ranked {page, page_id, snippet, score} — verified end-to-end through the real MCP server against the live endpoint - interim local selection removed from the tool path (pages-index remains only for the hook page cache pending full cleanup) * refactor(coding-agents): drop pages-index — local section index deleted; hook/runtime keep only the id+title roster (content lives behind the server-side knowledge-base search) * refactor(coding-agents): drop hindsight_search_memory (raw recall) — knowledge-page search is THE search surface; recall client method and formatter removed * feat(coding-agents): hindsight_reflect tool — on-demand deep memory reasoning alongside the session-start reflect * refactor(coding-agents): one 'conversation' retain strategy for all developer conversations Backfilled decision chats and live session write-back were the same content type (identical JSON action-transcript format) extracted two ways based only on where they came from. Merged CHAT_MISSION + SESSION_MISSION into one CONVERSATION_MISSION that scales facts to substance (short decision chat -> 1-2 facts, working session -> several; final-state-wins, verbatim literals, rejected-alternative rule kept); the ≤2-fact CHAT_CUSTOM_INSTRUCTIONS extractor is retired with it. * feat(coding-agents): restore Chris's knowledge entity_labels tier configureBank again sets entity_labels {knowledge: feature-work/decision/ convention/component/concept, tag:true} + entities_allow_free_form, so the extractor routes durable facts with knowledge:<tier> tags the server-side knowledge base can select on; capture_initiative markers regain the knowledge:feature-work label. Pages themselves stay unscoped (the okf knowledge base owns synthesis). * feat(coding-agents): seeded pages tag-scoped again — page tags match the restored knowledge:<tier> entity labels (capture_initiative pages included) * fix(coding-agents): reflect injection wrapped in <hindsight_memory> so write-back never re-ingests it; seed-state file (declined flag) removed — the live bank is the only state * feat(coding-agents): gitIngest enum ('message' | 'full' | 'none') — one setting, one code path for seeding AND staying current - deepen's idempotent git pass IS the sync: gitlog doc re-upserts when HEAD moves (gitlog-head:<sha> tag makes freshness a single tag query); in full mode new commits surface at the top of rev-list and the next run ingests them - separate git-sync path deleted (sync.ts, runtime.syncGitOnce, gitSync config) * feat(coding-agents): gitIngest defaults to 'message' (cheap by default; opt into depth); deepen gains --git-ingest override for harnesses * feat(coding-agents): session banner shows git-sync state (condensed syncStatus): 'git in sync' / 'catching up on new commits' / 'syncing git history (n/target)' * polish(coding-agents): two-line banner — value headline (tracking decisions/conventions/history) + bank/sync detail line * refactor(coding-agents): ONE config file — project-local .hindsight/coding-agent.json layer removed entirely (with its sanitization machinery); per-repo routing stays via directoryBankMap * docs(coding-agents): fix stale project-config reference in comment * refactor(coding-agents): runtime scratch (deepen lock + engine log) moves to the OS temp dir — ~/.hindsight now holds ONLY the config file * feat(coding-agents): cursor auto-ingestion parity — hosts without a SessionStart hook fire the deepen engine (+ cold survey) from the session's first prompt * feat(coding-agents): leveled plugin logging — one plugin.log (debug/info/warn/error, config logLevel + HINDSIGHT_LOG_LEVEL/FILE overrides); diag events mirror at debug; deepen logs itself (separate deepen.log dropped); warn on reflect/retain failures * feat(coding-agents): one-shot bank configuration via the server's template import — missions, strategies, entity labels, and the 5 seeded pages in a single idempotent POST /import (configureBank PUT+PATCH and createPages removed) * feat(coding-agents): one-command installer — npx hindsight-coding-agents install|uninstall [harness...] Detects the coding agents on the machine and merges each one's native wiring (hooks + MCP: claude mcp add for Claude Code; hooks.json + append- only config.toml sections for Codex; settings.json for Gemini; hooks.json + mcp.json for Cursor; plugin array for opencode). Idempotent by marker, preserves foreign entries, backs up touched files as .hindsight-backup; uninstall removes exactly ours. 27 unit tests over temp homes. * fix(installer): refuse to install from an npx/dlx cache (wired paths would die on eviction); document global install + npm update -g as the update path * ci(coding-agents): unit + typecheck + build job, and a live E2E job (real API server + real LLM) running the deepen->sync->reflect->injection path; prettier-format the package * docs(blog): launch post draft — coding-agent memory results (marked draft: true) * docs(blog): rewrite launch post as the narrative — from 'does memory even help?' through why-not-SWE-bench, the corrections dataset, benchmark-driven architecture decisions, to the final numbers * docs(blog): position knowledge pages as a co-launch headline — living-documents framing, example page excerpt, platform-wide availability (dashboard editor, hybrid search API, bank templates), closing CTA * docs(blog): restructure launch post payoff-first — contrarian RAG finding + cost in the lede, TL;DR box, narrated task with both runs, seeded-answers objection met head-on, data-locality/time-to-value/latency answers, Sonnet number promoted, backstory compressed to one section * fix(coding-agents): deepen waits for server-side ops to settle (template-import page refreshes broke the synced contract); HINDSIGHT_CONFIG env override for the config path (containers/test harnesses; replaces the live test's dependency on the removed project-config layer) * docs(blog): second-pass fixes — flagship example swapped to the arbitrary retry decision (RFC 4180 attack closed), reconstruction disclosed, 58% provenance clause, placebo backstory + grading block restored in numbers, RAG figure per-task, benchmark-site date * docs(blog): align remaining CSV references with the retry flagship; TL;DR per-task figures * docs(blog): flagship rebuilt on the real dataset task — the ERP export decision whose rejected alternative IS the textbook fix (='00042' formula form, minimal quoting, CRLF); dangling injection-verified reference restored; limitations cross-check attached to the correct row * docs(blog): rewrite as the 0.9.0 launch post — five-beat narrative (question → dataset → auto-recall failure → reflect → knowledge pages from llm-wiki to self-healing) for Knowledge Pages + unified coding-agents plugin * docs(blog): add the missing beat — shaping the dataset revealed decisions live in git, which the old plugins never ingested * docs(blog): reframe reflect — very smart rather than slow; first message carries the session goal; on-demand reflect tool for session drift * docs(blog): pages section addresses the 'back to files?' objection — pages as projected views over consolidated memory (contradiction resolution underneath), raw docs remain source of truth * docs(blog): out-of-box row updated to n=3 (22/26/23 -> 0.72/task, -26%; cost -35%); matured row marked single-run * fix(hooks): reflect block injected once per session (+ cadence refresh), not every turn — hook context persists in the transcript, so per-turn re-injection stacked duplicate blocks * fix(coding-agents): wrapper bundles ship deepen.js, not the renamed backfill.js The core build entry `backfill` was renamed to `deepen` (deepen engine + status), but the three wrapper build.mjs bundleFiles lists still copied the removed `backfill.js`, so every dev-install failed with ENOENT. Point them at `deepen.js` (spawned by seed.ts at runtime) so the installers build again. * feat(coding-agents): periodic re-survey — refresh structural pages every N commits Structural knowledge pages are only generated on a cold repo, so an evolving architecture drifts from what the survey captured. Add surveyRefreshCommits (default 20; 0 = cold-seed only): at SessionStart, count commits reachable from HEAD since the newest survey-baseline marker (branch-robust via git.commitsSince) and re-run the headless survey once the threshold is crossed, re-recording a baseline marker. Cold seed still records the first baseline. * fix(coding-agents): per-turn hook timeout (30s) must exceed the 25s reflect cap The once-per-session reflect is capped internally at HOOK_REFLECT_CAP_MS=25s, but every harness killed the UserPromptSubmit/BeforeAgent hook at 15s — below the cap. The host killed the hook mid-reflect before the cache write, so the injection was discarded AND the reflect re-fired uncached on every turn ("UserPromptSubmit hook timed out after 15s" every prompt). Raise the hook timeout to 30s (> cap) across claude/codex/gemini, bump Stop to 30 to match, and document the cap-below-timeout invariant so it can't silently drift again. * polish(coding-agents): attribution header is a bold blockquote callout, not flat text The live directives all told the agent to credit memory with a plain inline "From Hindsight memory (<page>):", which renders as flat text. Switch every directive (session tool-guide, reflect injection, both MCP tool descriptions) to a markdown blockquote header "> ... **From Hindsight memory (<page>)** — ..." so it renders as a distinct callout, restoring the richer attribution look. * fix(coding-agents): strip <hook_prompt> transport wrappers from retained transcripts (codex surfaces hook stdout/errors as user messages); session + backfill transcripts switch to JSONL (one turn per line — clean appends, chunker-atomic turns) Note: benchmark numbers (n=3) were measured on the JSON-array format; JSONL is extraction-equivalent by design but unvalidated by a sweep — gate before quoting new numbers on this pipeline. * fix(installer): write [features].hooks (codex_hooks deprecated in Codex >= 0.145); accept either flag as already-enabled * fix(hooks): fire the ingestion engine from the FIRST prompt on every harness (lock-protected no-op when SessionStart already did) — safety net for sessions predating the install, whose banks otherwise never get pages; survey stays SessionStart-owned (ensureSeed hosts excepted) * feat(status): expose survey observability — surveyBaseline (last surveyed HEAD, from Chris's survey-baseline markers) + surveyCommitsBehind in syncStatus/hindsight_sync_status * test(status): expected shapes include the survey observability fields * feat(survey): findings docs ARE the completion signal — surveyDocs (0-4) in syncStatus; a baseline without findings re-fires the survey at the next warm session start (crashed-survey retry) * feat(config): banks.<bankId> overrides — per-repo opt-in/out applied AFTER bank resolution (disable a repo, tune gitIngest/retainSessions per bank) from the ONE config file; resolution fields ignored inside a bank section * feat(config): bankAliases — remap resolved bank ids as the final resolution step (single hop, converging allowed); docs page brought fully current (env exceptions, gitIngest/logLevel/survey rows, banks overrides, aliases, resolution step 4) * refactor(config): bank rename lives INSIDE banks.<id> as the field (separate bankAliases tree removed) — one per-repo section for disable, behavior, and rename; applyBankConfig returns {cfg, bankId} * docs(coding-agents): recipe — two repos sharing one bank (converge by resolved id via banks.<id>.bank, or by path prefix via directoryBankMap), with the id-vs-path rule of thumb * rename(config): directoryBankMap -> mapPathToBank (direction-explicit; pre-0.9.0 breaking-rename window) * feat(coding-agents): companion skill — hindsight-coding-agent SKILL.md shipped in the package and installed into ~/.claude/skills by the installer; explains storing/retrieving, full config (banks/mapPathToBank/gitIngest), install/update, and debugging * docs(coding-agents): mention the companion skill in README + docs page * feat(coding-agents): companion skill ships to ALL skills-capable hosts (claude/gemini/cursor native dirs, codex via ~/.agents/skills standard); retained sessions and ingested documents carry the harness as tag (harness:<name>) and metadata * feat(skill): self-updating companion skill — every session start re-syncs installed copies with the packaged SKILL.md (presence-gated; npm update -g now updates the skill too, no re-install) * fix(coding-agents): worktree-aware document ids (no more per-worktree gitlog duplicates) + deepen self-cleanup; issue/PR refs preserved verbatim and emitted as ENTITIES; calibrated reflect-injection wrapper; docs ported to the TRUE source (hindsight-docs/docs-integrations) that generates the skill copy * docs(skill): explain the internal marker documents (survey-baseline:<sha> bare-sha content is deliberate — zero extracted facts; gitlog:<repo> seed doc) * feat(survey): human-readable baseline markers under a zero-extraction marker strategy (live-verified: 0 facts) — start as researching, deepen lazily flips to completed once findings exist * refactor(survey): one survey strategy with conditional rules replaces the separate marker strategy — status markers extract nothing, findings extract structural facts (both branches live-verified) * fix(hooks): mid-session heal — zero knowledge pages in the roster cache fires the ingestion engine on any prompt (covers long-lived sessions predating the install; lock makes repeats free) * feat(bank): ~ expansion in mapPathToBank; document the directory-blacklist recipe (map tree to one bank + disable it) * feat(coding-agents): explicit correction protocol — when the agent verifies a memory is wrong/stale it ingests a 'Correction: <topic>' doc (claimed vs verified-true vs evidence); guidance in the injection wrapper, tool guide, tool description, and companion skill * fix(hooks): reflect block injected exactly once — cadence re-injection dropped (replaying the turn-1 synthesis at arbitrary turns reads as random noise after drift; hindsight_reflect covers genuine re-need) * fix(coding-agents): 15s hard timeout on every client request + opencode boot no longer awaits seedIfCold — a stalled memory server can never freeze the host TUI (onPrompt already tolerates a late preamble) * fix(reflect): defer past trivial openers — a greeting no longer spends the once-per-session synthesis on 'hi' (seen live: reflect answered a greeting with persona chatter and burned the session's slot); first substantive prompt reflects instead * test(hooks): align reflect-call assertions with the non-trivial fixture prompt * Revert trivial-prompt reflect deferral (misread the report — the issue was the notice's UI position, not reflect-on-greeting behavior) * fix(opencode): stop writing banner/reflect notices to stderr — opencode renders plugin stderr inside the TUI at the cursor (text wedged against the input bar); the trail moves to the plugin log * feat(opencode): TUI companion plugin — visible presence via api.ui.toast (opencode's TUI plugin API): banner toast on activation + reflect goal/preview toasts from the plugin-log trail; installer registers the second entry * fix(opencode): visible presence via the server client's tui.showToast (POST /tui/show-toast) — banner + reflect toasts from the server plugin; the separate TUI module approach removed (1.18.9's loader rejects tui-only entries in the shared plugin list); SDK deps bumped to 1.18.9 * fix(opencode): toasts never rendered — v1 client wants {body}, and boot toast raced TUI mount opencode injects the v1 SDK client whose showToast signature is {body: {title, message, variant, duration}} and which resolves with {data|error} instead of rejecting — the earlier flat-params call sent an empty body and the failure was invisible. Also the toast event is not durable: the seed banner on a warm bank fired <1s after plugin init, before the TUI subscribed, and was lost. Toasts now use the body shape, log a rejected result at debug, and defer until ~3s past init. Verified live in tmux: boot banner and reflect toast both render. * fix(coding-agents): reflect must report history, never issue directives The 0.8.6-blog incident: reflect fused two true but unrelated facts (the hermes-deprecation goal and the blog-section removals of c87e7ac19) into one confabulated narrative rendered in the imperative — 'You should explicitly remove the following sections' — a completed past action re-issued as a present directive, indistinguishable from a prompt injection to the receiving agent. Three changes: - buildReflectQuery wraps the session's first prompt with strict rendering rules: declarative past-tense attributed facts only, no instructions or recommendations, no stitching unrelated episodes into one narrative. - The <hindsight_memory> wrapper now states the block is a record of the past that never assigns tasks: imperative wording inside it is a description of work already done, to be ignored unless it informs the task as historical fact (and unrelated memories are still ignored outright). - The reflect_ok diag event records the injected synthesis verbatim (8k cap), so the next incident is one grep instead of harness-transcript spelunking. * refactor(coding-agents): read and seed knowledge pages through the knowledge-base API The plugin advertised knowledge pages but drove them off /mental-models, so the two halves of the feature never met: pages seeded via the bank template's mental_models key got a mental model and no knowledge_pages node, and /knowledge-base/search joins through that table — the five seeded pages were absent from the corpus of the tool billed to the agent as its FIRST STOP. The one page search could return (an initiative, created through the KB endpoint) came back as a kp-… node id, which the reader then fed to GET /mental-models/{id} and 404'd. Search found only what read could not open. Every page operation now speaks one id space: - listPages reads /knowledge-base/tree and flattens it to {items:[…]}, dropping folders and keeping the containing folder name. - getPage reads /knowledge-base/pages/{id} — the ids search and [[page:<id>]] links already hand back. - seedPages replaces the template's mental_models key: it creates the PAGES taxonomy through /knowledge-base/pages and re-syncs a drifted source_query via PATCH /knowledge-base/nodes/{id}, so a plugin upgrade that rewords a query lands on the live page instead of orphaning its synthesized content. Matched by name, since the endpoint mints its own id; a 409 from a concurrent deepen run is tolerated rather than failing the run. - createPage/updatePage/deletePage are deleted — mental-models CRUD with no callers outside its own tests. Verified against a live server on a scratch bank: five real kp- nodes, re-run reports 0 created / 5 unchanged, all five readable by their listed id, all five now returned by /knowledge-base/search, and a hand-drifted source_query restored onto the same node rather than a duplicate. * feat(coding-agents): autoReflect flag — opt out of injected reflect into tool-only mode autoReflect (default true, layerable per-harness/per-bank like every other field) keeps today's validated behavior: one reflect synthesis injected on the session's first prompt. Set false and nothing is injected; instead the knowledge preamble and every roster refresh carry an explicit trigger telling the agent to call hindsight_reflect itself whenever a new task/goal is set — the pull-based variant, ready to benchmark against the push default. * docs(blog): move the 0.9.0 launch post to its own PR The draft now lives on blog/0-9-0-launch so this PR merges independently of launch timing (hero image, publish date, and final voice pass pending there). * fix(deepen): dead-holder locks are stale immediately, not after 30 minutes The per-bank deepen lock only honored its TTL: a killed run (SIGKILL, crashed harness) left its bank locked for LOCK_STALE_MS, and every subsequent deepen exited 'another run holds the lock — nothing to do' against an empty bank. The lock already records the holder's pid — probe it (kill -0); if the holder is gone the lock is stale now. Found live: a killed benchmark ingestion left four banks locked and the retry campaign polled empty banks to its deadline. * feat(coding-agents): expand native harness support * fix(reflect): table-shaped decisions must be reproduced verbatim, not summarized Benchmark replay showed reflect compressing mapping/table policies into prose ('specific extensions map to specific types') and even asserting a lossy generalization that matched a known-wrong fix — while rule-shaped policies survive intact. The reflect query now demands complete verbatim enumeration of mappings/sets/tables including carve-outs. * fix(reflect): decisions outrank implementation-derived memory Under heavy retrieval noise, reflect surfaced the git-ingested BUGGY module source as 'the established implementation logic' while claiming no decision records existed — presenting the bug under investigation as authority. The rendering rules now state: report decisions and rationale, never the current implementation (the reader has the code); when decision memory and code-derived memory conflict, the decision wins; implementation-only matches are not policy. * feat(coding-agents): expand harness integrations * Expand coding-agent integrations and legacy compatibility * chore(coding-agents): fix the CI-only test failure and complete the release wiring The `test-coding-agents` job failed on every run while passing locally: the gitDiffTarget fixture committed into a temp repo without a git identity, which a developer machine supplies from its global config and a CI runner does not ("empty ident name not allowed"). The identity is now passed per-command, the way the harness E2E fixture already did it. Release wiring, which was incomplete in three places that each fail at a different point: - scripts/release-integration.sh had no entry, so the release refuses to start. - generate_changelog.py keeps its OWN integration list; the release script aborts and reverts at the changelog step when a name is missing there. - The docs build cross-checks released tags (`integrations/<name>/vX.Y.Z`) against the SLUGS in integrations.json. The directory was the only integration carrying a `hindsight-` prefix, so the tag would have been `integrations/hindsight-coding-agents/...` against a `coding-agents` slug — green release, then a failing docs build. The directory is renamed to `coding-agents` so directory, integration name, tag and docs slug all agree, matching every other integration. Also drops the claude-code-v2 / codex-v2 / gemini-v2 wrappers and the hindsight-memory-v2 marketplace entry. Claude Code is fully served by `hindsight-coding-agents install claude-code` — hooks, MCP and skill — so the wrappers were a second copy of the same core with its own version to keep in lockstep. The README rows that pointed at their dev-installers now name the supported installer command instead. * fix(coding-agents): make the installer actually re-point a moved package Both bugs were exposed by the directory rename, which invalidated the absolute paths every host config stores — the case `install` exists to repair. - Grok wrote its block only when one was absent, so every later `install` was a silent no-op and the dead paths survived; the only repair was editing config.toml by hand. It now replaces the block, sharing one regex with uninstall. - MARKER was the full package name, which identifies our entries for dedupe-on-reinstall and for uninstall. A repo checkout stopped containing it once the directory dropped its `hindsight-` prefix, so from a checkout re-installs would have accumulated duplicate hook entries and `uninstall` would have removed nothing. Narrowed to the substring both layouts share. Regression tests cover a moved package being repointed (not appended past), the marker matching npm and checkout paths, and a repeated checkout install leaving one entry per event. --------- Co-authored-by: Chris Latimer <chris.latimer@vectorize.io>	2026-07-31 22:15:21 +02:00
Sanderhoff-alt	bc604ab91b	fix(packaging): bundle licenses in Python distributions (#3067) ... Stage the canonical repository license in each isolated Python build context so wheels and source distributions include the MIT text. Declare SPDX license metadata and verify every release artifact before publishing to prevent repository firewalls from quarantining packages. Closes #3054	2026-07-30 16:55:15 +02:00
Scott Guymer	6500944c74	feat(copilot-cli): add GitHub Copilot CLI hooks integration (#2742) ... * feat(copilot-cli): add GitHub Copilot CLI hooks integration Add hindsight-integrations/copilot-cli/, giving GitHub Copilot CLI persistent long-term memory via Hindsight hooks (see docs.github.com/en/ copilot/how-tos/copilot-cli/customize-copilot/use-hooks). Modeled on the existing cursor-cli integration. Hooks: - sessionStart: recall using initialPrompt (or a cwd-derived fallback query), injects additionalContext - subagentStart: recall for every subagent Copilot CLI spawns (explore, task, research, code-review, rubber-duck, security-review, and custom agents, not the built-in general-purpose agent, which never fires this hook). Subagent payloads carry no per-invocation task text, so this always uses the fallback query. - agentStop: reads the transcript, retains to Hindsight on a configurable turn cadence, caches the transcript path for sessionEnd - sessionEnd: forces a final retain using the transcript path cached from the last agentStop, since sessionEnd's own payload has no transcript path field Install via pip install hindsight-copilot-cli, then hindsight-copilot-cli install (user scope, writes ~/.copilot/hooks/hindsight-copilot-cli.json) or --scope repo for a team-shared .github/hooks/ registration. Zero runtime dependencies, hook scripts are pure stdlib Python. Also wires up CI (test-copilot-cli-integration job), release-integration.sh and generate_changelog.py registration, and docs gallery/sidebar entry. Closes #1588 * fix(copilot-cli): regen skill mirror, drop unreleased changelog link - Run generate-docs-skill.sh to add the missing skill mirror for the new copilot-cli doc page (verify-generated-files was failing on the untracked references/sdks/integrations/copilot-cli.md). - Remove the [View Changelog] link, which pointed at /changelog/integrations/copilot-cli — a page the release script only creates on first release, so it was a broken link failing build-docs.	2026-07-28 09:52:59 -04:00
Nicolò Boschi	6d5157575c	fix(oracle): unblock Oracle CI — free runner disk space + fix the retain deadlock (#2948) ... * ci(oracle): free runner disk space before Oracle jobs The three Oracle jobs run the Oracle 23ai `free` service image, which together with the Python ML deps (torch) exhausts the runner's ~14 GB root disk. Two symptoms, one cause: - uv fails to extract a wheel with "No space left on device (os error 28)" (fast ~2 min failure), and - a near-full disk starves I/O badly enough to trip the 30-minute job timeout. test-python-client-oracle and test-typescript-client-oracle have been red on every open PR (#2941, #2942, #2943) from this, independent of the code under test. Reclaim ~20 GB of preinstalled tooling (the same jlumbroso action the Docker build job already uses) before the Oracle setup step. docker-images stays false here: unlike the Docker build job, the Oracle service container is already running by the time steps execute, so pruning images could disrupt it. The savings come from the tool cache, Android SDK, .NET, Haskell, large apt packages and swap. * ci(oracle): trim disk reclaim to the fast, high-yield options The first pass enabled every reclaim, which cost ~4 minutes of job time — counterproductive on jobs that are already fighting a 30-minute limit. android + dotnet + haskell + swap are a few rm -rf's worth ~16-21 GB, which is ample headroom for the Oracle image plus torch. Dropped: - large-packages: apt-get remove, costs minutes for little extra space; - tool-cache: deletes the preinstalled Python that actions/setup-python then re-downloads, making the job slower rather than faster. * fix(retain): flush entity stats after releasing the connection (Oracle hang) Retain hung forever on the Oracle backend: every retain test burned its 120s client timeout while the server sat idle, so test-python-client-oracle and test-typescript-client-oracle only ever reached ~5% of the suite before the 30-minute job limit. The server was not slow — it was deadlocked. flush_pending_stats() acquires its own connection, but it was being called while the enclosing acquire_with_retry(...) block still held one: async with acquire_with_retry(pool) as conn: # conn checked out async with conn.transaction(): # SAVEPOINT only ...write facts/entities... await entity_resolver.flush_pending_stats() # takes a 2nd connection oracledb does not autocommit and OracleConnection.transaction() is only a SAVEPOINT, so the write is committed by OracleBackend.acquire() when its block exits. Connection #2's `UPDATE entities ...` therefore waits on row locks held by the still-open connection #1, which cannot commit until the call returns — a circular wait. Oracle never reports ORA-00060 because session #1 is blocked in Python, not on the database, so it hangs indefinitely instead of erroring. Move the flush after the acquire block in all three call sites (streaming retain, delta retain, transfer importer), which is what its own docstring already required ("must be called AFTER the retain transaction commits") and which PostgreSQL satisfied only by accident via asyncpg autocommit. Guarded with an AST lint test rather than a behavioural one: the deadlock cannot be reproduced against PostgreSQL, which is what the suite runs on. * test(repair): retry the concurrent index drop on deadlock test_dry_run_creates_nothing still flaked in test-api shard 3. CONCURRENTLY avoids ACCESS EXCLUSIVE but still takes ShareUpdateExclusive, which conflicts with the ShareLock a fresh bank's plain CREATE INDEX holds — and that one cannot be made concurrent, since it runs inside the bank-create transaction. So _drop_bank_indexes can still be picked as the deadlock victim while another xdist worker seeds a bank: Process A waits for ShareUpdateExclusiveLock on memory_units; blocked by B. Process B waits for ShareLock on virtual transaction; blocked by A. The bank-create side already retries (#2943); give the drop the same treatment. The drop is idempotent, so retrying is safe.	2026-07-24 17:31:38 +02:00
Nicolò Boschi	a514d39624	fix(deps): require litellm>=1.93.0 for Python 3.14 support (#2950) ... litellm ships its own Rust extension (litellm-rust python-bridge -> litellm.rust_bridge._native, built via maturin/PyO3). Releases before 1.93.0 publish no cp314 wheel, so on Python 3.14 uv falls back to the sdist and the build fails: error: the configured Python interpreter version (3.14) is newer than PyO3's maximum supported version (3.13) 1.93.0 adds cp314 wheels and a PyO3 that builds on 3.14. Raising the floor fixes the failure at its source, so the interpreter no longer has to be constrained. That lets us drop the UV_PYTHON=3.13 workaround added in #2801: the _set_uvx_python_compat() helper and its call sites are removed from the claude-code, codex, cursor, and cursor-cli daemons, along with the tests that pinned that behaviour. Dropping the pin costs nothing — litellm publishes no macOS wheels at all, so macOS builds from the sdist on every version regardless, while Linux now gets a real cp314 wheel instead of a source build. Also strengthen the build-api-python-versions CI matrix. It previously ran only `uv build`, which just packages the source and passes even when the dependency set cannot install or import on the target interpreter -- it would not have caught this. It now installs into a fresh venv, byte-compiles, and runs an import smoke test on each version. Verified on CPython 3.14.4 with UV_PYTHON unset: litellm 1.93.0 installs, the Rust bridge builds, and hindsight_api plus the engine import cleanly. Refs #2783	2026-07-24 15:06:52 +02:00
Ben	b11e053323	feat(zcode): add Hindsight long-term memory integration for ZCode (#2549) ... * feat(zcode): add Hindsight long-term memory integration for ZCode Adds a hooks-based, no-MCP integration for ZCode (Z.ai's GLM desktop coding agent). ZCode embeds the Claude Code agent runtime and reads the standard Claude Code hook schema from its own config namespace (~/.zcode/cli/config.json), so `hindsight-zcode install` wires three process hooks — SessionStart, UserPromptSubmit (recall), and Stop (retain) — without touching the user's ~/.claude config and without an MCP server. Recall injects relevant memories as additionalContext before each prompt; retain assembles each turn from the prompt (captured at UserPromptSubmit) and the response (Stop payload) and stores it to Hindsight. Verified end-to-end in ZCode 3.2.2: hooks fire, retain persists to the cloud bank, and recall injects memory into the agent. Includes the pip package + installer, hook scripts, tests, CI job, release-integration wiring, changelog registration, docs page, and gallery entry. * feat(zcode): add self-serve marketplace + hooks-only plugin variant Publishes the ZCode integration as a hooks-only Claude Code plugin (hindsight-zcode) in the repo's plugin marketplace, so ZCode users can install it via 'zcode plugins add-marketplace vectorize-io/hindsight' without pip and without depending on Z.ai's marketplace. The plugin reuses the pip package's hook scripts via CLAUDE_PLUGIN_ROOT (no duplication) — settings.json resolves as a sibling of scripts/ in both the pip and plugin layouts. Adds a plugin manifest, plugin-format hooks.json (SessionStart/UserPromptSubmit/Stop — no SessionEnd), marketplace entry, validation tests, and docs. * fix(zcode): drop changelog link from docs page (page exists only after release) The /changelog/integrations/zcode page is generated at release time, so linking to it broke the Docusaurus build (build-docs + verify-generated-files). Most unreleased integration pages omit this link; follow that convention.	2026-07-20 10:58:24 -04:00
Sanderhoff-alt	dddd571a99	fix(ci): avoid rebuilding docs in verify-generated-files (#2739) ... Run the existing build-docs job for every PR so the production docs build remains an unconditional check. Generate OpenAPI directly in verify-generated-files to avoid rebuilding the Docusaurus site serially in that job.	2026-07-20 14:30:32 +02:00
Ben	9dee1c594b	refactor(zed): port setup CLI to Node, drop the Python dependency (#2599) ... * refactor(zed): port setup CLI to Node, drop the Python dependency The Zed integration is configuration-only — it writes the `context_servers` entry into Zed's settings.json and a recall/retain rule into AGENTS.md — and the MCP server it configures runs via `npx mcp-remote`, so Node.js was already a hard requirement. Requiring Python *as well* just to write two config files meant users needed two runtimes. Port the `hindsight-zed` CLI to a zero-dependency Node CLI so the integration needs only Node: - Node CLI under `src/` + `bin/hindsight-zed.js`, shipped via `package.json` (matches the existing TypeScript integrations; release-integration.yml already detects package.json for npm publishing). - Behavior-preserving: same commands (`init`/`status`/`uninstall`), flags, `--print-only`, env/file/flag config resolution, JSONC-safe settings edits, and fenced AGENTS.md rule block. - Tests ported to Node's built-in runner (`node --test`) — 21 tests. - CI (`test.yml`) updated to run `npm test` on Node 22 instead of pytest. - Removes the Python package (`hindsight_zed/`, `pyproject.toml`, `uv.lock`, Python `tests/`). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(zed): make the Node package publishable by the release workflow The release workflow classifies any integration with a package.json as `type=typescript` and unconditionally runs `npm ci` + `npm run build` in the integration dir. This zero-dependency, no-build JS package had neither, so `integrations/zed/v*` would fail at release time (invisible in test CI, which only runs `npm test`): - add a no-op `build` script so `npm run build` succeeds - commit package-lock.json so `npm ci` succeeds (it refuses to run without one, even with zero deps); lockfile has no node_modules entries, so check-integration-lockfiles.sh passes trivially - drop the stray settings.json (a local `init` scaffold accidentally committed) and gitignore it Verified locally: node --test (21/21), npm ci, npm run build, and npm publish --dry-run all pass; tarball ships only bin/src/README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEHyNxWfn1miSWRW6NUtEW * docs(zed): update setup to Node/npx (drop pip install) * refactor(zed): scope npm package as @vectorize-io/hindsight-zed Match the scoped-name convention of the other TS integrations (@vectorize-io/hindsight-ai-sdk, -chat, -openclaw). CLI/bin command stays 'hindsight-zed'; npx/global-install references updated to the scoped name. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: DK09876 <dk09876@Dikshants-MacBook-Pro.local>	2026-07-07 16:01:33 -04:00
DK09876	fcb2c958e7	feat(devin-desktop): rename Windsurf→Devin Desktop + fix(continue) thread-safe adapter (#2410) ... * feat(devin-desktop): rename windsurf integration to Devin Desktop Cognition rebranded Windsurf to Devin Desktop (June 2026); Cascade is EOL July 1. Rename the (unreleased) windsurf integration to devin-desktop before first publish: - Package hindsight-windsurf -> hindsight-devin-desktop (module hindsight_devin_desktop, CLI hindsight-devin-desktop, DevinDesktopConfig, bank default 'devin-desktop', HINDSIGHT_DEVIN_DESKTOP_BANK_ID) - Rule now writes to .devin/rules/hindsight.md (preferred path) instead of the legacy .windsurf/rules/; trigger: always_on unchanged - MCP config path stays ~/.codeium/windsurf/mcp_config.json (Devin Desktop's on-disk data dir, unchanged by the rebrand) - Official Devin logo; docs + integrations.json + README refreshed with the 'formerly Windsurf' framing - Registries updated: test.yml job, release-integration.sh, generate_changelog, integrations.json (strict JSON), docs page 26 unit tests + gated live-MCP E2E pass; ruff check+format clean; real-app smoke against local Hindsight verified (init writes both files; live recall returns seeded facts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(continue): resolve a fresh Hindsight client per request (thread-safe) The adapter runs on a ThreadingHTTPServer (one worker thread per request) but shared a single Hindsight client across all of them. The client's aiohttp session is bound to the thread/event-loop that first used it, so the first @hindsight recall worked and every one after threw 'Timeout context manager should be used inside a task' — Continue then showed an error context item and the model answered with no memory. Resolve the client per request (test-injected clients still used as-is), and close per-request clients in a finally so the fresh aiohttp session doesn't leak a connector each call. Bump to 0.1.1. Found via a real in-editor VS Code test. Adds a regression test asserting per-request client resolution across the threaded server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 17:27:39 -07:00
Nicolò Boschi	b6608076ff	fix(release): bump marketplace version on claude-code release (#2386) (#2398) ... The Claude Code plugin ships via the marketplace manifest, not a package registry. The integration release (release-integration.sh claude-code) already bumps plugin.json, but the marketplace manifest carried no version and was never bumped — so the published catalog never reflected new releases (e.g. #2066 on Windows). - add a "version" field to the root .claude-plugin/marketplace.json - release-integration.sh now bumps it in lockstep with the plugin version when releasing claude-code, and commits it - remove the redundant hindsight-integrations/.claude-plugin/marketplace.json: `claude plugin marketplace add vectorize-io/hindsight` only ever reads the root manifest (even with --sparse), so the second manifest was never consulted - drop the stale --sparse install hint from the release-integration workflow The claude-code release flow is otherwise unchanged — release it as before.	2026-06-25 13:31:00 +02:00
Ben	d0b77f5bee	feat(eve): add Eve agent-framework MCP connection helper (#2280) ... * feat(eve): add Eve agent-framework MCP connection helper Add @vectorize-io/hindsight-eve: a thin helper that wraps Eve's defineMcpClientConnection to wire an Eve agent into a Hindsight MCP server in one line, pre-filling the endpoint, model-facing description, and bearer auth with env-var defaults (HINDSIGHT_MCP_URL, HINDSIGHT_API_KEY, HINDSIGHT_MCP_BANK_ID).	2026-06-24 14:48:38 -04:00
DK09876	7194f98b19	feat(windsurf): add Windsurf (Codeium) integration via MCP (#2358) ... * feat(windsurf): add Windsurf (Codeium) integration via MCP Config-only CLI that wires the Hindsight MCP server into Windsurf's ~/.codeium/windsurf/mcp_config.json (mcpServers, remote serverUrl + auth header) and writes an always-on recall/retain rule to .windsurf/rules/hindsight.md (trigger: always_on). Cascade then has recall/retain/reflect and uses them automatically. - hindsight_windsurf: config, mcp_config (strict-JSON parse-or-print), rules (dedicated sentinel-marked file), cli (init/status/uninstall) - 25 unit tests + gated live-MCP-endpoint E2E - CI job, release + changelog registries, docs page, icon, README row Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(windsurf): apply ruff format to cli.py lint.sh runs 'ruff format'; collapse the --rules-path add_argument to one line so verify-generated-files passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(windsurf): use official Windsurf logo for the integration icon Replace the placeholder abstract mark with the official Windsurf logo (simple-icons, CC0), matching the real-brand-logo convention used by the other integration icons. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-24 09:58:55 -07:00
DK09876	91bf32842e	feat(github-copilot): add GitHub Copilot (VS Code) integration via MCP (#2299) ... Adds hindsight-copilot: long-term memory for GitHub Copilot in VS Code, using Copilot agent mode's native MCP support (HTTP servers) — no bridge. `hindsight-copilot init`: - merges a Hindsight HTTP MCP server into .vscode/mcp.json (servers.hindsight), JSON-safe (prints a snippet if the file is JSONC), and - writes a recall/retain rule into .github/copilot-instructions.md, which Copilot applies to every chat in the workspace. Resolves the ask in #1588. Mirrors the Zed/OpenHands MCP-config pattern. - hindsight_copilot package: config, mcp_config (.vscode/mcp.json writer), instructions (copilot-instructions.md rule), cli (init/status/uninstall) - 25 deterministic tests (mcp.json merge incl. preserving servers/inputs + JSONC fallback, instructions rule block) + gated requires_real_llm MCP handshake E2E - CI job, release registration (VALID_INTEGRATIONS + changelog generator), docs page, registry entry, icon (octicons), README row Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-23 16:39:12 -07:00
Evo	0d60f0c638	fix(release): build Linux CLI on ubuntu-22.04 (glibc 2.35) instead of glibc-2.39 runners (refs #2321) (#2330)	2026-06-23 10:05:09 +02:00
DK09876	aab7032071	feat(aider): add Aider integration (session-bracketing memory wrapper) (#2297) ... hindsight-aider wraps the aider CLI: recalls project memory before each session (injected via --read) and retains the transcript after. Bank per git repo.	2026-06-18 13:52:38 -07:00
DK09876	65862c4fef	feat(openhands): add OpenHands integration (native MCP config + recall/retain rule) (#2276) ... Long-term memory for OpenHands via native Streamable-HTTP MCP: hindsight-openhands init wires the Hindsight MCP server into config.toml + a recall/retain rule in AGENTS.md.	2026-06-17 12:46:20 -07:00
DK09876	539101af38	feat(zed): add Zed editor integration (MCP context server + recall/retain rule) (#2153) ... MCP-only Zed integration: hindsight-zed init wires the Hindsight MCP server into Zed's settings.json (via mcp-remote) plus a recall/retain rule in AGENTS.md. Validated end-to-end in real Zed.	2026-06-17 10:28:07 -07:00
Ben	1c9ba0e659	feat(composio): add Composio integration (Hindsight memory as custom tools) (#2180) ... * feat(composio): add Composio integration (Hindsight memory as custom tools) Exposes Hindsight retain/recall/reflect as Composio in-process custom tools via register_hindsight_tools(). The Hindsight bank for each call is the Composio session's user_id, so one registered tool set isolates memory per user automatically. Also ships memory_instructions() for pre-recall system-prompt injection (Composio doesn't auto-inject context). - hindsight_composio/: tools.py, config.py (dataclass + env fallback), errors.py. - tests/: 50 tests using a FakeComposio (mirrors the real tool decorator + SessionContext) + mocked Hindsight client — exercises the framework wiring. - CI: test-composio-integration job (uv build/sync/ruff/pytest) + path filter. - Gallery card + doc page + official Composio icon; release-integration.sh entry. * fix(composio): register in changelog generator + test memory_instructions - Add composio to generate_changelog.py INTEGRATIONS dict (release would otherwise fail at the changelog step; it was only in release-integration.sh). - Add TestMemoryInstructions covering formatting, max_results cap, empty/error fallback, tag passthrough, and missing-config error. * address review: Literal config types, typed generics, debug log, real-LLM E2E - Type budget as Literal[low|mid|high] and tags_match as Literal[any|all| any_strict|all_strict] across config + tools (matches autogen/continue) - Parameterize bare list -> list[Any] on register_hindsight_tools - _ensure_bank: logger.debug the swallowed create_bank failure so a real auth/network error is visible rather than only surfacing later on retain - Add requires_real_llm E2E bucket exercising retain/recall/reflect through the (input, ctx) tool call path against a live Hindsight server; exclude from PR CI via -m 'not requires_real_llm'	2026-06-16 15:56:53 -04:00
DK09876	c05ab9103f	feat(continue): add Continue.dev integration via HTTP context provider (#2213) ... Adds hindsight-continue: Hindsight memory for Continue.dev via its native http context provider (@hindsight recall) plus an optional MCP-server + rules setup. Includes the adapter package, tests against Continue's HTTP contract + a gated E2E, CI job, release registration, docs, and registry entry.	2026-06-15 14:38:39 -07:00
Ben	359b2bc762	feat(zapier): add Hindsight Zapier app (actions + REST Hook triggers) (#2119) ... * feat(zapier): add Hindsight Zapier app (actions + REST Hook triggers) A Zapier Platform CLI app that brings Hindsight memory into Zaps. Actions: - Retain Memory (create) -> POST /v1/default/banks/{bank}/memories - Recall Memories (search) -> POST .../memories/recall - Reflect (search) -> POST .../reflect Triggers (instant, via Hindsight's webhook API — subscribe POSTs /webhooks, unsubscribe DELETEs it): - Retain Completed, Consolidation Completed, Memory Defense Triggered Auth: API key as Bearer token, Cloud default with self-hosted override; the Bank field is a dynamic dropdown from GET /v1/default/banks. Built on zapier-platform-core 19; 'private': true so the npm release path can never publish it (Zapier publishing is manual via zapier push/promote, not release-integration.yml — and zapier is intentionally NOT in VALID_INTEGRATIONS). Adds test-zapier-integration CI job (npm install -> zapier validate -> npm test) and a repo README row. 15 mocha/nock unit tests; 'zapier validate' is structurally clean. Docs-site gallery card + doc page + icon are a follow-up (need the official Zapier brand asset; omitted here to keep build-docs green). * fix(zapier): make apiKey optional for no-auth self-hosted + prettier-clean - authentication.js: apiKey now optional (required: false). The middleware only adds the Bearer header when a key is present, so you can connect to a self-hosted instance running without auth by leaving it blank; Cloud still requires a working key (blank -> 401 fails the connection test). - README: document self-hosted / localhost usage, the optional key, and correct the CLI binary name to 'zapier-platform' (v19 renamed it from 'zapier'); show the .env approach so 'zapier invoke' needs no global install. - Run prettier across the integration (fixes pre-existing format drift that was failing verify-generated-files on this branch). zapier validate still structurally sound; 15 tests pass. * fix(zapier): correct reflect answer field + recall output shape (found via live test) Extensive live testing against Hindsight Cloud surfaced two response-shape bugs the mocked unit tests missed (they mocked the wrong shapes): - reflect: the synthesized answer is in the response's `text` field, not `answer`. searches/reflect read `data.answer` (undefined), so a Zap got no answer. Now reads `data.text` and surfaces it as `answer`. Test mock fixed to use the real `text` field so it actually guards this. - recall: results carry no numeric `score`, and the fact-type field is `type` (not `fact_type`). Corrected the sample + outputFields so the Zap editor only advertises fields that actually populate; test mock made realistic. Verified live end-to-end: auth, bank dropdown, retain (full + minimal), recall (real fact extraction), reflect (now returns the grounded answer), and the webhook subscribe/list/delete lifecycle. 15 unit tests pass; zapier validate clean. * docs(zapier): correct .env auth-field prefix to authData_ in README zapier invoke reads .env auth fields with the authData_ prefix (e.g. authData_apiKey, authData_apiUrl), not bare apiKey/apiUrl. Confirmed against a working local .env during live testing. * docs(zapier): add integrations gallery card + doc page - gallery entry in integrations.json (id zapier, official, category framework) - doc page docs-integrations/zapier.md (actions + REST Hook triggers, setup) - official Zapier logo at static/img/icons/zapier.png check-integrations passes (forward: entry → doc page); JSON valid; prettier-clean. * feat(zapier): verify webhook HMAC signatures + optional async retain (review notes 2 & 4) #2 — Webhook signature verification (was: relying only on Zapier's unguessable URL): - performSubscribe now generates a random 32-byte secret and registers it with the webhook; the secret is stored in subscribeData. - perform verifies the X-Hindsight-Signature: sha256=<hmac> header (HMAC-SHA256 of the raw body) and rejects mismatches. (Corrected the header name — the API sends X-Hindsight-Signature, not X-Webhook-Signature; body is delivered byte-for-byte via content=, so the recomputed HMAC matches.) #4 — Optional 'Process asynchronously' toggle on Retain (default false). Lets users with very large content avoid Zapier's action timeout; pairs with the Retain Completed trigger. 17 unit tests pass (added valid/invalid signature cases); zapier validate clean.	2026-06-15 16:44:04 -04:00
Nicolò Boschi	f0802b826b	chore(ci): enforce unused imports/vars + advisory dead-code scan (#2144) ... * chore(ci): enforce unused imports/vars + advisory dead-code scan Enable ruff F401 (unused imports) and F841 (unused variables) -- previously ignored as "too noisy" -- across hindsight-api-slim, hindsight-dev, and hindsight-embed, and clean up the resulting violations. These are now blocking: lint.sh auto-removes them and the verify-generated-files CI job fails on any leftover diff. Add an advisory dead-code scan for what the linter cannot see -- whole unused Python functions (vulture) and orphaned files/exports/dependencies in the control plane (knip): - scripts/hooks/check-unused.sh runs both locally - new non-blocking check-unused-code CI job surfaces findings on PRs - hindsight-control-plane/knip.json tunes out toolchain false positives vulture stays advisory because its function/argument heuristics false-positive on FastAPI/SQLAlchemy/Pydantic patterns; knip can be flipped to blocking once the control-plane dead code (PR #2135) lands. * chore(ci): make knip blocking on unused files/deps; remove dead deps #2135 deleted tooltip.tsx but left @radix-ui/react-tooltip in package.json, and react-chrono / three were never imported. Remove all three, and declare @radix-ui/react-visually-hidden (used in directive-detail-modal but unlisted). With the control-plane tree now clean, the check-unused-code job runs `knip --include files,dependencies,unlisted` as a BLOCKING step. vulture and knip's unused-exports check (the shadcn/ui surface is kept intentionally) stay advisory.	2026-06-12 11:06:31 +02:00
Ben	f5a6c300f1	feat(agent-framework): Hindsight memory for Microsoft Agent Framework (no MCP) (#1989) ... * feat(agent-framework): add Hindsight memory integration via context provider Persistent memory for Microsoft Agent Framework (the successor to Semantic Kernel) without MCP. HindsightProvider is a ContextProvider whose before_run recalls relevant memories and injects them into the agent's instructions, and whose after_run retains the conversation. Reuses the LlamaIndex integration's client/config pattern and the hindsight-client Python SDK. Targets the agent-framework-core 1.x before_run/after_run + SessionContext contract (verified against the installed package since the API has churned). 15 unit tests subclass the real ContextProvider so drift fails loudly, plus a gated e2e. Includes CI job, release + changelog + docs wiring, and an icon. * chore(agent-framework): refresh lock to agent-framework-core 1.8.1 (verified no API drift) * fix(agent-framework): drop unused per-op timeout constants TIMEOUT_RETAIN/TIMEOUT_RECALL/TIMEOUT_BANK were defined but never used: the hindsight-client SDK sets one timeout on the constructor and has no per-call timeout argument, so per-op values can't be wired in. Keep the single constructor-level TIMEOUT_DEFAULT and document why. Addresses review feedback.	2026-06-11 13:57:20 -04:00
Nicolò Boschi	4032b27912	feat(embed): local control center web app (#2132) ... A persistent, localhost-only control center web app bundled in hindsight-embed: LLM config wizard, raw .env editor (with effective-only view), daemon + control-plane Start/Restart/Stop with live API/UI health, editable per-profile API/UI ports + component versions, daemon + control-plane log tail, profile delete, deep-linking, token-gated /api/* (CSRF-safe), localhost everywhere. UI built with Preact + Tailwind (Vite), output committed to static/ and served by the embed's stdlib http.server (no Node at runtime, offline). Ports moved from metadata.json into each profile's .env (HINDSIGHT_API_PORT / HINDSIGHT_EMBED_CP_PORT). CI job verifies the bundle builds + is wired.	2026-06-11 17:09:35 +02:00
Ben	c06e85b64f	fix(obsidian): clear community-store review for 0.1.2 + asset attestations (#2107) ... Obsidian community-store automated review (v0.1.1) flagged three errors and a warning; this clears them and adds build-provenance attestations. Errors: - Manifest description must not include the word 'Obsidian' → reworded to '...cites the source notes. Your vault stays the single source of truth.' - no-static-styles-assignment (chat-view.ts): el.style.height = ... → el.setCssStyles({ height }) per the plugin guidelines. - no-unsupported-api (main.ts): Workspace.revealLeaf requires Obsidian v1.7.2 → bump minAppVersion 1.5.0 → 1.7.2 (matches the obsidian@^1.7.2 types we build against). versions.json 0.1.2 → 1.7.2. Warning: - builtin-modules dep → Node's built-in module.builtinModules in esbuild config; dependency removed. Recommendation (build-provenance attestations for main.js/styles.css): - Add actions/attest-build-provenance for the Obsidian assets in release-integration.yml (+ attestations: write). Assets release in the dedicated repo while the build runs here, so verify at owner scope: gh attestation verify main.js --owner vectorize-io. Bumps manifest to 0.1.2. Verified with the bot's own linter (eslint-plugin-obsidianmd): both code errors clear. build + tsc + 46 tests pass.	2026-06-10 10:58:13 -04:00
Ben	96910071c9	fix(ci): npm provenance 409 guard + add Obsidian MIT LICENSE (#2094) ... * fix(ci): treat npm provenance 409 (tlog) as already-published in release-integration When a release tag is re-pointed and the workflow re-runs, npm publish --provenance fails with TLOG_CREATE_ENTRY_ERROR / (409) 'equivalent entry already exists in the transparency log' because the identical artifact was already logged by the prior run. The package is already published, so this is benign — widen the already-published guard to swallow it (alongside the existing 'cannot publish over'). * chore(obsidian): add MIT LICENSE for community-store submission The Obsidian community-store review bot requires a LICENSE file at the plugin repo root. The dist repo (vectorize-io/hindsight-obsidian) is mirrored from this directory via the release workflow, so adding it here propagates on next release.	2026-06-09 16:26:40 -04:00
Ben	85402035ed	fix(ci): override git auth with OBSIDIAN_DIST_TOKEN for obsidian mirror push (#2093) ... * fix(ci): override git auth header with OBSIDIAN_DIST_TOKEN for mirror push The previous fix (unsetting the checkout extraheader) wasn't enough: the runner also authenticates github.com via a git credential helper, so the subtree push still ran as github-actions[bot] (403 on the dedicated repo). Override the Authorization header with the dist token via `git -c` — an explicit header beats both the checkout header and the helper, and propagates to subtree's internal push via GIT_CONFIG_PARAMETERS. * fix(ci): unset bot extraheader before overriding with dist token http.extraheader is multi-valued, so adding our -c header on top of the checkout's bot header sent two Authorization headers → GitHub 400. Unset the checkout header first so only the OBSIDIAN_DIST_TOKEN header is sent. * fix(ci): reset credential helper so only the dist-token header is sent The runner's git credential helper was injecting the bot Authorization on top of our extraheader → 'Duplicate header: Authorization' (400). Reset the helper chain with -c credential.helper= so only the OBSIDIAN_DIST_TOKEN header remains. * fix(ci): isolate git context for the obsidian mirror push + diagnostics Push kept sending a duplicate Authorization from a config scope local --unset didn't reach. Split locally and push from an isolated context (global/system config nulled, helper disabled, local extraheader unset) with the token in the push URL → a single Authorization. Also dump auth-config origins for diagnosis. * fix(ci): reset the inherited extraheader to push as OBSIDIAN_DIST_TOKEN Diagnostic showed the runner injects the bot token as an http.extraheader via an *included* config file (no credential helper), which --unset-all can't touch. Reset the extraheader list with an empty -c value (read last → clears it at request time) and auth via the push URL → a single dist-token Authorization.	2026-06-09 15:46:49 -04:00
Ben	37a20ec524	fix(ci): use OBSIDIAN_DIST_TOKEN for the obsidian mirror push (#2091) ... actions/checkout sets an http extraheader for the default GITHUB_TOKEN that overrode the token embedded in the subtree-push URL, so the push ran as github-actions[bot] (no access to the dedicated repo → 403). Unset that header before the push so OBSIDIAN_DIST_TOKEN is used.	2026-06-09 14:26:57 -04:00
Ben	a67a8f774d	fix(obsidian): mirror plugin to a dedicated repo instead of releasing in the monorepo (#2078) ... * fix(obsidian): stop creating a GitHub Release per plugin version Per-integration GitHub Releases pollute the repo's release list (meant for the core Hindsight product) and steal the 'Latest' badge — the obsidian v0.1.0 release displaced v0.8.0. BRAT / the community store also can't target a tag inside a multi-release monorepo (they read a repo's *latest* release), so the step never gave working BRAT distribution anyway. - Remove the 'Attach Obsidian release assets' step from release-integration.yml (replaced with a comment explaining why; npm publish is unchanged). - Point BRAT install instructions at the dedicated repo vectorize-io/hindsight-obsidian in the integration README and docs page. - Add a 'Distribution & maintainers' section documenting the two-repo setup so plugin updates are released to both (monorepo = source of truth + npm, dedicated repo = BRAT / community-store releases). * ci(obsidian): mirror plugin to dedicated repo via subtree on release Instead of manually maintaining two repos, the release workflow now mirrors hindsight-integrations/obsidian/ to the root of vectorize-io/hindsight-obsidian (git subtree push --prefix) and cuts the BRAT / community-store GitHub Release there — the monorepo stays the single source of truth. - Add the 'Mirror Obsidian plugin to its dedicated repo' step to release-integration.yml (unshallow → subtree push → idempotent release). Needs secret OBSIDIAN_DIST_TOKEN (contents:write on the dedicated repo). - Drop the now-unused 'contents: write' permission (no releases are created in this repo anymore). - Rewrite the README 'Distribution & maintainers' section: the mirror is automatic, the dedicated repo is generated, don't edit it directly.	2026-06-09 13:47:50 -04:00