mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
perf-profile-workers
206 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nicolò Boschi
|
06e9c7054e |
feat(mental-models): dry-run refresh and keep_trace for troubleshooting (#3119)
When a refresh produced an unexpected document, nothing said why. The mode
decision, resolved scope, snapshot window, retrieved-versus-used fact counts
and dropped delta operations only ever reached a log line — and cron- or
consolidation-driven refreshes run with nobody watching.
Two ways to see that reasoning, from opposite directions.
POST /mental-models/{id}/dry-run-refresh runs the production refresh
pipeline and reports what it would do, skipping exactly two writes: the
content (with its structured document and history entry) and the watermark
that moves last_refreshed_at. It takes no parameters, on purpose — a dry run
you can configure stops predicting the refresh it exists to predict. Because
nothing is persisted, a delta dry run reads exactly the window the next real
refresh will.
trigger.keep_trace records the same reasoning on every refresh of a model,
scheduled ones included, under reflect_response.trace. It is written even
when a refresh fails, which is when it matters most. The trace is shaped
like reflect's — the calls the agent made plus the refresh decision — and
holds nothing derivable from elsewhere: evidence stays in based_on, and the
resolved scope and window are reported by the dry run. Each tool call
records the window bound it was given, named `updated_at` for what the
predicate actually filters; null means the tool applies no time bound at
all, which is what explains results older than the window would suggest.
refresh_mental_model is split into a shared _execute_mental_model_refresh
that computes a result and writes nothing, plus a thin persistence step, so
the preview and the real refresh run the same body. Existing refresh
behaviour is unchanged.
In the control plane the dry run is an action on the mental model, and its
result opens in a dialog built from the History tab's own diff components.
History shows each version's own trace: the history snapshot now carries
`trace` alongside `based_on` so it survives being superseded.
Surfaced but deliberately not fixed here: when delta operations fail, the
fallback writes a candidate built from a delta-scoped recall over the whole
document, dropping content grounded in older memories (#3112).
|
||
|
Nicolò Boschi
|
1ce308f359 |
fix(api): report the real last-write time on banks and documents (#3109)
`/v1/default/banks` exposed only `last_document_at` (MAX of `documents.created_at`), so a bank whose long-lived document keeps receiving appends looked idle: ingestion time never moves, and `banks.updated_at` only tracks name/mission edits. UIs reading either as "last write" showed hours-old activity while memories were still landing. Add `last_write_at` to the bank list — the newest of "a document was (re-)retained" (`documents.updated_at`) and "a fact was stored" (`memory_units.created_at`) — and order the list by it. `last_document_at` keeps its ingestion-time meaning and is now documented as such. The control-plane bank selector shows and sorts by the new field. Same symptom on the documents list (reported in #2944): it was ordered by `created_at DESC`, burying an actively-appended document behind every document created after it. It is now ordered by `updated_at DESC`. Fixes #2944 |
||
|
Nicolò Boschi
|
218e6d34b1 |
feat(knowledge-base): client-managed knowledge pages, control-plane UI + hindsight fs CLI (#2455)
* feat(knowledge-base): self-curating knowledge base (OKF pages + folder missions)
Server-side knowledge base: a hierarchy of folders and pages over mental
models, projected to the Open Knowledge Format, with a mission-driven curator
that maintains pages automatically after each consolidation.
- knowledge_pages table (PG + Oracle): parent_id tree, kind folder/page,
mission, managed, last_curated_at; partial unique index on (folder, name)
for concurrency-safe dedup; added to BACKUP_TABLES.
- api/okf.py: OKF serializer (frontmatter + body, index/log, constellation graph).
- engine/knowledge_curator.py: folder curator (LLM op plan + safe apply); reads
new memories since last curation (delta, not recall); ops create/merge/delete
page + spawn sub-folder (bounded depth<=3, <=8). Runs as an async curate_folder
task on folder/mission create and after consolidation. Curator pages use an
observation-only delta trigger with exclude_mental_models.
- MemoryEngine: folder/page CRUD, tree, curate, async submit + worker handler.
- /v1/default/banks/{bank}/knowledge-base/* endpoints.
- Control plane: knowledge-base tree view + constellation toggle, missions,
OKF page panel + bundle export; proxies, client, sidebar, i18n.
- Tests: okf unit, knowledge-base HTTP, curator apply + dedup guard, hs_llm_core e2e.
- Regenerated OpenAPI + SDK clients + docs-skill.
* feat(hindsight-fs): mirror a bank's mental models as a live local folder
Add @vectorize-io/hindsight-fs, a CLI under hindsight-tools/ that mirrors a
Hindsight bank's mental models as real markdown files (YAML frontmatter + body)
in a local directory, refreshed from the API on an interval. Once mounted,
ordinary shell tools (ls, cat, grep, find, ...) work against current memory.
- Pull-based sync engine: full list each tick, write changed/new/tampered
files, skip unchanged (content-hashed), prune deleted models. Atomic writes;
a transient API error never wipes the mirror.
- One-way mirror enforced two ways: files are read-only (0444) so agent edits
fail with EACCES, plus a tamper-revert backstop that compares on-disk bytes
and overwrites drift on the next pass. --writable opts out.
- Commands: mount/start/stop/restart/sync/status/list/logs/unmount. Background
daemon via detached process + pidfile; per-mount config is remembered.
- status doubles as a healthcheck: --json report and a non-zero exit when the
mount is dead/failed/stale (--stale-after overrides the threshold).
- Tests: unit (sync engine, frontmatter, health) + e2e that spawns the real
CLI against a mock API and exercises real bash commands. 26 tests.
* refactor(hindsight-fs): mirror the knowledge-base tree, not mental models
Re-point hindsight-fs at the knowledge base so it projects a bank's folder/page
hierarchy as nested directories + .md files, instead of a flat list of mental
models.
- client: fetch GET /knowledge-base/tree + /export (two calls, any bank size)
and join by page id; replaces the paginated mental-models list.
- format: planMirror() walks the tree into folder dirs + page files at nested
paths (slug per segment, collision-safe); pages render the page's OKF doc.
- sync: create folder dirs, write pages at nested paths, prune removed pages and
emptied folders; state keyed by relative path + tracked dirs.
- config/cli: drop the mental-model `detail` flag; `list` prints folders+pages;
help/README updated. Tests rewritten for the tree/export model.
Verified live against a bank's knowledge base: the `people` folder mirrors to
people/anna.md + people/marco.md with OKF frontmatter.
* refactor(knowledge-base): drop server-side curation + folder missions
The knowledge base is now purely client-managed (CRUD over folders/pages); the
server no longer auto-curates. Removes the folder curator entirely and the
folder `mission` concept, and leads the sidebar with Knowledge Base.
- Remove engine/knowledge_curator.py, the curate_folder task (handler + dispatch
+ submit_async_curate_folder / _bank_folders), the post-consolidation curation
hook, and the folder-create / mission-update curation triggers.
- Remove folder `mission` and `last_curated_at` (columns + engine + API + UI);
keep `managed` as a client-set flag. Migration a5b6 now adds `managed` only;
the last_curated_at migration is dropped and the unique-index migration
repointed. Single alembic head preserved.
- API: KnowledgeNode/CreateFolderRequest/UpdateNodeRequest lose `mission`;
PATCH node handles name/parent_id only.
- Control plane: sidebar leads with Knowledge Base (before Memories); remove the
mission field, edit-mission dialog, and mission display from the KB view.
- Delete the curator tests; regenerate OpenAPI + SDK clients.
* feat(knowledge-base): default pages to living-document trigger + 4096 tokens
Client-created pages had no server curation applying a trigger, so they fell back
to the plain mental-model default (no refresh, full mode, all fact types). Make a
knowledge page a living document by default: when the client omits `trigger`, use
observation-only + delta + exclude_mental_models + refresh_after_consolidation;
when it omits `max_tokens`, default to 4096 (vs the mental-model 2048). Clients
can still override either.
* feat(control-plane): bank Home dashboard, Knowledge tabs, Notion editor + memory Euler graph
A large control-plane pass on the knowledge base UX:
- Home dashboard (home-view): memory constellation + read-only knowledge-page
TOC (reuses the Pages tree) + recent documents + the bank-profile "Memory store"
card and "Memories by ingested time" chart (extracted as reusable exports).
Fixed-height top row so the constellation fills and the side cards scroll.
- Sidebar: add Home (first); order Home → Memories → Knowledge.
- Knowledge view: Pages / Mental Models sub-tabs (Mental Models moved out of the
Memories view). Pages tab is an Obsidian-style workspace — file-tree sidebar +
inline editor with open-page tabs; the generation prompt is tucked behind a
"How this page is derived" expander; a "Backed by N memories" line opens the
backing model's based_on via the existing mental-model detail modal. First page
auto-opens; deep-link via ?page=.
- Knowledge graph: reframed as an Euler/Venn of the source memories — nodes are
based_on memories, one translucent circle per page (overlaps = shared memories),
plus the memory graph's own edges. New Constellation venn mode (nodeGroupsFn /
groupColorFn / groupLabelFn) drawing overlapping per-group circles + pill labels.
Backend: knowledge_page_memory_graph endpoint (pages' based_on → memory nodes).
- Pages default to the living-document trigger (observation-only, delta, exclude
mental models, auto-refresh) + 4096 max_tokens.
- Documents: metadata badges are expandable (show all keys, not just 3).
- Regenerated OpenAPI + docs-skill.
* feat(cli): port hindsight-fs into the Rust CLI as `hindsight fs`
Rewrite the standalone TypeScript hindsight-fs tool as a native subcommand
of hindsight-cli. Mirrors a bank's knowledge base (folders + pages) to a
local folder of markdown files, one-way (API -> disk) with read-only files
and drift-revert, plus a detached background refresh daemon.
- new src/commands/fs/ module (client, format, sync, state, daemon,
health, config, paths) with unit tests for the pure logic
- subcommands: mount/start/stop/restart/sync/status/list/logs/unmount
- deps: reqwest blocking + sha2 + libc
- remove the TS package and its npm workspace entry
* feat(knowledge-base): drop the pages Graph view + polish the Pages UX
Client:
- Remove the Tree/Graph toggle and the whole graph branch (the toggle row
was the dead band between the tabs and the content).
- Tree rows: full-width page name + compact status dot (was a pill that
crushed the name to a few chars); float the hover actions so they no
longer reserve width; widen the sidebar (w-64 -> w-72).
- Borderless workspace card; solid sticky editor-tab bar (was bg-muted/20,
so scrolled body text bled through); tighter sub-tab spacing.
- Drop getKnowledgeBaseGraph + the /api/knowledge-base/graph proxy route
and the now-unused i18n keys (viewTree/viewGraph/graphEmpty).
Server:
- Remove GET /knowledge-base/graph, KnowledgePageGraphResponse, the
knowledge_page_memory_graph engine method + its helper, and the orphaned
KnowledgeGraph/palette code in okf.py.
- Regenerate OpenAPI + SDK clients + docs-skill.
* feat(knowledge-base): add hybrid GET /knowledge-base/search (BM25 + vector)
Doc-level hybrid search over a bank's knowledge pages, fused with Reciprocal
Rank Fusion in a single query — no reranker, tuned for latency (~sub-100ms
query on top of the embed).
- engine.search_knowledge_pages: vector arm (mm.embedding ANN) + BM25 arm
(mm.search_vector, the generated tsvector over page name+content) via
websearch_to_tsquery('english'), RRF-fused (k=60) in SQL. BM25-only
fallback when the query embedding is unavailable. Folders excluded.
- GET /v1/default/banks/{bank}/knowledge-base/search?q=&limit= with
KnowledgePageSearchResult/Response models (id, name, mental_model_id,
snippet, score, updated_at).
- tests: ranks the relevant page first, excludes folders, respects limit,
requires q.
- Regenerate OpenAPI + Python/TS/Go clients + docs-skill (Rust client has
no KB ops; the CLI's fs port uses raw reqwest there).
* feat(control-plane): wire knowledge-page hybrid search into the Pages sidebar
A debounced search box at the top of the Knowledge sidebar queries the new
/knowledge-base/search endpoint (BM25 + vector, RRF-fused). A non-empty query
swaps the folder tree for a ranked result list (name + snippet); clicking a hit
opens the page in a tab. Clear (×) restores the tree.
- new /api/knowledge-base/search proxy route
- client.searchKnowledgePages(bankId, q, limit) in lib/api.ts
- search box + results list in knowledge-base-view.tsx (reuses openPage)
- i18n: searchPlaceholder / clearSearch / searchEmpty + api.errors.knowledgeBase.search across all locales
* feat(control-plane): widen the Knowledge tree to 1/3 + show page tags inline
- file tree pane w-72 -> w-1/3 (content gets the other 2/3)
- render each page's tags as small chips under its timestamp in the tree
* feat(control-plane): let new pages set tags in the create dialog
The create-page dialog gains a comma-separated Tags field (wired to the
existing createKnowledgePage tags param); a type:<x> tag sets the page's
OKF type. i18n added across all locales.
* perf(control-plane): stop the home dashboard blocking on a 22MB graph
The memory constellation fetched limit=1000 (≈67k edges, ~22MB) and the whole
dashboard awaited it before painting. Now the light panels (stats/pages/docs)
render immediately and the constellation loads on its own (with a spinner) at a
200-node cap (~3.7MB). The full graph stays in the Memories view.
* feat(control-plane): add a 'View all' button to the home memory card
The memory-constellation card header gets a 'View all →' link to the Memories
(data) view, matching the Knowledge pages / Recent documents cards.
* feat(knowledge-base): edit a page's source query, tags, and token budget
PATCH /knowledge-base/nodes/{id} now also updates a page's options on its
backing mental model — source_query, tags, max_tokens — each applied only when
present. Changing source_query schedules an async refresh so the page rebuilds
against the new question.
- engine.update_knowledge_page + UpdateNodeRequest fields + endpoint wiring
- control plane: an "Edit" button on the open page opens a dialog (name /
source query / tags); tags pre-fill from the raw tree tags so the type:<x>
tag isn't dropped on save. i18n across all locales.
- tests: update options persist; empty PATCH is 400.
- regenerated OpenAPI + clients + docs-skill.
* refactor(knowledge-base): squash page migrations into one + drop OKF naming
- Fold the three knowledge_pages migrations (table / managed column / unique
page-name index) into a single a9b8c7d6e5f4 migration.
- Rename api/okf.py -> api/page_markdown.py and replace the "OKF" / "Open
Knowledge Format" terminology throughout (server, control plane, CLI, i18n)
with plain "markdown" / "page" wording — pages just render to markdown.
- Add the knowledge-base endpoints to the CLI OpenAPI-coverage skip list (they
live in the control plane UI / are mirrored by `hindsight fs`, no CLI cmd).
- Regenerate OpenAPI + clients + docs-skill.
* test(knowledge-base): rename test_okf -> test_page_markdown, drop dead graph tests
Follow-up to the okf.py rename + Graph-view removal: the test module still
imported the old `okf` module (breaking collection for the whole api test
suite) and still tested the removed tag-based knowledge_graph() builder.
* fix(transfer): classify knowledge_pages in export-bank (skip for now)
test_export_bank_covers_schema requires every BACKUP_TABLES entry to be
classified by export-bank. knowledge_pages is skipped: carrying its self-
referential parent_id tree needs a parents-first restore order that the generic
per-row _restore_rows doesn't provide (follow-up). Mental models are carried, so
the target can recreate the tree.
|
||
|
Nicolò Boschi
|
08995e3013 |
Release v0.8.6
- Update version to 0.8.6 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Sync documentation to version-0.8 |
||
|
Nicolò Boschi
|
6a460d2c9a |
feat(reflect): add apply_all_directives to bypass directive tag scoping (#3031) (#3046)
* feat(reflect): add apply_all_directives to bypass directive tag scoping (#3031) Directives are tag-scoped like memories: a reflect with no tags loads only untagged directives, and tagged directives apply only when the request's tags match. This is deliberate (isolation_mode), but it means an operator's tag-organized directives silently never reach an untagged reflect — 45% of standing rules in the deployment reported in #3031. Add an opt-in `apply_all_directives` flag on the reflect request (default false, preserving current behavior). When true, every active directive is loaded regardless of tags, ignoring tag scope. Wired through the HTTP API, both MCP reflect variants, and the engine. Also correct the docs, which claimed directives are "always" enforced without mentioning tag scoping. Regenerated OpenAPI, clients (Go/Python/TS/Rust), and the docs skill mirror; updated the control-plane reflect proxy + api.ts types. * chore(cli): record apply_all_directives CLI-coverage exemption The reflect field is intentionally not exposed as a CLI flag (available via the REST API, SDKs, and control plane). Record the exemption so cli-coverage-check passes, matching the existing tag_groups entry. |
||
|
Nicolò Boschi
|
c41ad9bd75 |
feat(api): filter memory list by linked entity + entity timeline UI (#2945)
* feat(api): filter memory list by linked entity + entity timeline UI Add an `entity_id` query param to `GET /memories/list` — an exact reverse lookup over stored entity links (not text/semantic match), backed by the existing idx_unit_entities_entity_unit index. Because entity links reference live memory units only, combining `entity_id` with `state=invalidated` returns nothing. Wire it through the control-plane list route + clients, and use it in the entity detail panel to render an observation timeline (reuses the memories TimelineView) — click an entity, see its linked observations over time. Closes #2936. * fix(control-plane): entity timeline shows all linked memories, not just observations Verified against real data: observations are derived/consolidated summaries and carry no entity links — entity links live on the source world/experience facts, which are also the ones with occurred dates. Filtering the entity timeline to type=observation therefore always rendered an empty panel. Drop the type filter so the panel shows every memory linked to the entity (the actual dated timeline), and relabel the section "Timeline" with dedicated i18n keys. * chore(control-plane): drop now-unused observation i18n keys from entitiesView * style(reflect): wrap over-length _generate_structured_output call Ruff format wraps this >120-char call; committing the formatter output so the verify-generated-files CI check (which runs the formatter and diffs) is clean. |
||
|
Nicolò Boschi
|
31218127e0 |
fix(retain): make async retries idempotent via caller-supplied operation_id (#2937) (#2947)
* fix(retain): make async retries idempotent via caller-supplied operation_id An async retain whose HTTP acknowledgement is lost or times out leaves the caller unable to tell whether the operation was created; retrying enqueues a second parent operation and repeats extraction, embeddings, and provider spend. Add an optional caller-supplied operation_id (UUID) used directly as the parent async_operations primary key. Re-submitting with the same id returns the original operation and creates no new work; the existing primary key is the concurrency authority, so no new columns, constraints, or migration are needed. Reusing an id owned by a different bank or operation type returns HTTP 409. Omitting operation_id keeps the current create-each-time behavior. Fixes #2937 * docs(retain): explain why the idempotency read is not in the create txn * fix(retain): sync generated docs-skill + Rust clients for operation_id - Regenerate the two docs-skill artifacts derived from the retain doc / OpenAPI change (verify-generated-files). - Add operation_id: None to the Rust client test and CLI RetainRequest literals so both crates compile against the regenerated struct. |
||
|
Nicolò Boschi
|
6a0b85f108 |
feat(config): make store_document_text overridable per bank (#2940)
* feat(config): make store_document_text overridable per bank HINDSIGHT_API_STORE_DOCUMENT_TEXT was static/server-level. Make it hierarchical so a data-minimizing bank (e.g. GDPR-sensitive) can keep only derived facts while other banks on the same deployment retain the raw source. - Add store_document_text to _CONFIGURABLE_FIELDS (settable per bank via the config API's generic updates dict, like audit_log_enabled). - Thread the per-bank resolved value into the retain storage path (chunk_storage.store_chunks_batch + fact_storage.upsert_document_metadata / handle_document_tracking / _upsert_document_row) from the orchestrator's resolved config; falls back to the server-level config when unset so non-retain callers (import) are unchanged. - Make the three consistency guards per-bank too so a store-off bank behaves coherently: append-mode rejection, recall include_chunks force-off, and the reflect 'expand' tool exclusion. - Docs: mark the flag hierarchical. Covered by a per-bank override test (one bank off, one default-on) + a configurable-fields guard; existing global-flag tests set the ConfigResolver global snapshot (env alone no longer suffices for a hierarchical field, mirroring enable_audit_default). * feat: expose store_document_text (+ audit_log_enabled) in bank templates & UI - BankTemplateConfig gains store_document_text and audit_log_enabled so bank templates can preset them; regenerated bank-template-schema.json. - Control-plane bank config: new 'Document Storage' tri-state section (Inherit / On / Off), mirroring the audit toggle; translations added across all 10 locales (non-en use English placeholders pending translation). Backend template round-trip + messages parity/used-keys + tsc all green. * chore(ui): rename bank-config 'Document Storage' section to 'Privacy' * feat(ui): merge audit + document-text toggles into one 'Security & Privacy' section Combine the separate Audit Logging and Privacy config sections into a single Security & Privacy section with both tri-state toggles and one save (writes audit_log_enabled + store_document_text together). Drop the now-unused section-level message keys across all locales; add securityPrivacy* keys. * fix(retain): use _get_raw_config for store_document_text fallback store_document_text became bank-configurable, so get_config().store_document_text now raises ConfigFieldAccessError (the guard forcing per-bank resolution). The storage functions' None-fallback hit that guard, breaking every direct/delta caller that didn't pass the value (test_chunk_storage_upsert, test_delta_retain). Fall back to _get_raw_config() instead — the unguarded global layer the ConfigResolver and the /config defaults response already use. The retain path still passes the per-bank resolved value; only non-retain callers hit the fallback. * chore: regenerate openapi + clients + docs-skill for BankTemplateConfig fields Adding store_document_text/audit_log_enabled to BankTemplateConfig changed the OpenAPI schema; regenerate the spec, Go/Python/TS client models, and docs-skill copies, and apply lint formatting (verify-generated-files). * test: bump configurable-field count 41->42 for store_document_text |
||
|
Nicolò Boschi
|
705757f362 |
Release v0.8.5
- Update version to 0.8.5 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Sync documentation to version-0.8 |
||
|
chethanuk
|
dcef72480b |
feat(api): allow deleting terminal bank operations with control-plane support (#2777)
* feat(api): allow deleting terminal bank operations with control-plane support * refactor(api): rename terminal-operation delete route to /delete Maintainer review on #2777 asked for the hard-delete endpoint to live at /delete rather than /record. Renames the path segment end-to-end: dataplane route + log message, tests, OpenAPI spec (and its skills mirror), and the generated Python/TypeScript/Go clients. The route's operation_id is explicitly "delete_operation", so no generated symbol names change -- only the path string. Go and Python generated output was verified byte-identical against the pinned openapi-generator v7.10.0. |
||
|
Nicolò Boschi
|
0d2dbe756d |
feat(audit): make audit_log_enabled overridable per bank (#2827)
* feat(audit): make audit_log_enabled overridable per bank Auditing was all-or-nothing per deployment. This makes the existing audit_log_enabled switch hierarchical (env -> tenant -> bank) so a bank can opt in while the server default is off, or opt out while it is on, rather than introducing a second near-identically-named field. Making the flag per-bank forces three call sites to change: - AuditLogger: the enabled check can no longer be a synchronous pre-filter, since a bank may enable auditing the global value has off. Split into action_allowed() (bank-independent allowlist, still a cheap sync pre-filter) and should_log() (awaits the per-bank resolution). Resolution failure falls back to the deployment default rather than failing closed, so a transient DB blip cannot silently create an audit gap for a bank that is meant to be audited. - Retention sweep: previously gated on audit_log_enabled, which is now per-bank while the sweep is a global cross-tenant job with no bank in scope. A bank opting in under a default-off deployment would have had its rows accumulate forever. Retention now keys off the (still server-level) retention window alone. - _audit_memory_defense: was sync and reached log_fire_and_forget directly, bypassing the per-bank decision entirely. Made async so the memory_defense action honours the bank's setting like every other path. The actions allowlist and retention window stay server-level: both are global sweeps with no bank scope. The /version audit_log flag keeps reporting the deployment default and now says so. Adds the Audit Logging toggle to the bank Configuration tab. The hindsight-docs skill regen also picks up pre-existing drift from #2694 (retain.md), which the pre-commit generator syncs unconditionally. * fix(control-plane): make the audit toggle tri-state A Switch cannot express "inherit the server default". It rendered the resolved value, so a bank inheriting `true` looked identical to one explicitly set to `true`, and touching it always wrote an explicit boolean with no way back to inherit. Replaced with a Select: Server Default / Enabled / Disabled. The slice now reads the bank's `overrides` rather than the resolved config, since the resolved value cannot distinguish inherited from explicitly-set. Choosing "Server Default" sends null, the tombstone the config resolver already treats as "clear this override". The option label shows which way the server default currently points, read from the existing /version features flag. Uses INHERIT_SENTINEL rather than "" for the inherit option: Radix rejects an empty SelectItem value at runtime. * chore(clients): regenerate for audit_log description change The audit_log field description in openapi.json changed; regenerate the Go/Python/TypeScript clients that embed it (they were skipped earlier because the generator needs Docker). Verify-generated-files was failing on the drift. * fix(audit): resolve gating config internally, bypassing permission filter _resolve_bank_audit_enabled used get_bank_config, the API-facing resolver that runs the tenant permission filter (get_allowed_config_fields). A deployment that makes audit_log_enabled read-only for a user — exactly the intended way to lock the field via an extension — would have that field stripped from the resolved config, so gating silently reverted to the deployment default and ignored the bank's stored override. Switch to resolve_full_config (the internal, unfiltered resolver every other internal config consumer uses). Gating is a system decision and must see the bank's true value regardless of who is asking. Adds a regression test with a restrictive tenant extension: the API read strips the field, but gating still audits the opted-in bank. Also: document the fail-open opt-out edge in should_log's comment, and refresh a stale "static, server-level switch" comment in the memory defense test. |
||
|
superafun
|
c1fadc008a |
feat: add tags filtering to list_memories / list_memory_units (#2848)
* feat: add tags filtering to list_memories / list_memory_units Add `tags` and `tags_match` parameters to `list_memory_units`, MCP `list_memories` tool, and HTTP `GET /memories/list` endpoint, bringing the browse side's tag filtering capability in line with the write side (`retain`) and semantic search side (`recall`). The implementation reuses the existing `build_tags_where_clause` function from `hindsight_api/engine/search/tags.py`, supporting all five matching modes: any, all, any_strict, all_strict, exact. Closes #2842 Related: #792 * review fixes: robust prefix strip, exact global scope, tests, regen clients - Use str.removeprefix("AND ") instead of str.lstrip("AND ") when appending the tags clause in list_memory_units (lstrip strips a char set, not a prefix — matches the existing idiom used elsewhere in the file). - Handle tags_match="exact" with no tags: select the untagged/global scope, mirroring recall and the sibling list path. - Type the MCP list_memories tools' tags_match as TagsMatch; document all five matching modes in the engine/HTTP/MCP docstrings. - Add integration tests covering all five modes + exact-empty global scope and the no-filter baseline (tests/test_tags_visibility.py). - Regenerate OpenAPI spec, docs-skill reference, and Python/TS/Go clients. --------- Co-authored-by: Nicolò Boschi <boschi1997@gmail.com> |
||
|
Voscko
|
86ff344c93 |
fix(worker): bound terminal operation history (#2708)
Add configurable TTL (default 30 days, 0=keep-forever) for terminal async_operations rows. Expired completed/failed/cancelled rows are pruned in bounded batches (1000/cycle) by a background task that never touches pending/processing work. Batch children are protected until their parent is pruned; cancelled-child cleanup atomically cancels a pending parent first. PG uses FOR UPDATE SKIP LOCKED; both PG and Oracle re-check eligibility under the row lock before deleting. Includes indexes, docs, and regenerated SDKs. 184 retention/worker/operation-status tests pass locally. All CI green. Fixes #2705 |
||
|
ishanmalik
|
73d3231bbd |
fix(api): accept verbatim extraction mode in bank manifests (#2576)
Co-authored-by: ishanmalik <16657815+ishanmalik@users.noreply.github.com> |
||
|
Sanderhoff-alt
|
5cc1482a72 |
fix(memory): return metadata from memory browse endpoints (#2583)
The list and get memory-unit paths selected tags and timing fields but skipped the memory_units metadata column, so metadata retained on facts was invisible outside recall. Select and serialize metadata for live and invalidated memory units, add a curation regression test for both paths, and update docs plus OpenAPI examples. |
||
|
Nicolò Boschi
|
92f433c904 |
Release v0.8.4
- Update version to 0.8.4 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Sync documentation to version-0.8 |
||
|
Nicolò Boschi
|
d68f618969 |
feat(stats): distributed bank_stats cache + ?refresh param + stats perf suite (#2495)
* feat(stats): distributed (table-backed) bank_stats cache on PostgreSQL get_bank_stats aggregates over memory_links/unit_entities — a multi-second scan on large banks. It was cached per-process (in-memory), so every API worker recomputed once per TTL and the first caller after expiry stalled. Add a bank_stats_cache table and a DistributedBankStatsCache that shares one worker's computation across all workers. Same get_or_load/invalidate contract as the in-memory cache, so the hot path is a single PK SELECT on a hit; only a miss runs the existing _compute_bank_stats loader and UPSERTs the row (ON CONFLICT, no lock — concurrent misses recompute, last write wins). All DB touches are best-effort: an unreachable/missing cache table degrades to computing uncached rather than failing the endpoint. PostgreSQL only; Oracle keeps the in-memory cache (selected by dialect at construction). * feat(stats): add ?refresh query param to force fresh /stats (default off) Adds force_refresh to get_bank_stats (and both cache backends): when set, the cached value is bypassed and recomputed, and the fresh result refreshes the cache for subsequent callers. Exposed on GET /stats as ?refresh=true (default false). Regenerated OpenAPI spec + clients. * test(perf): add stats benchmark suite + huge prod-sim scale New 'stats' perf suite measures get_bank_stats: uncached aggregation latency (node/link counts + entity rollup) vs cached, run with the result cache disabled so the headline numbers are the real per-poll cost. Adds a 'huge' prod-simulation scale that bulk-loads ~500k units / ~17.8M physical memory_links via COPY (entity links derived from unit_entities, not stored). * test(stats): exclude bank_stats_cache from backup guard + HTTP refresh test - bank_stats_cache is a derived TTL cache (no FK to banks, repopulates on demand), so exclude it from test_backup_tables_covers_entire_schema rather than back up stale cache rows — a restore starts it cold. - Add a ?refresh=true assertion to the /stats HTTP integration test. * fix(cli): pass refresh arg to get_agent_stats after ?refresh param The new /stats ?refresh query param adds a positional arg to the progenitor- generated get_agent_stats; the CLI reads the cached value, so pass None. |
||
|
Nicolò Boschi
|
758f346d30 |
feat(recall): structured per-stage scores and two-level min_scores filtering (#2422)
Replace the recall result's single `score` with a `scores` object exposing the scores from each pipeline stage, and replace the `min_score` request param with `min_scores`, a per-stage filter that operates at two levels. Response — each result carries `scores`: - final : the value results are ranked by - reranker : cross-encoder normalized relevance (null for passthrough rerankers) - semantic : raw vector cosine similarity (null if not surfaced semantically) - text : raw keyword/BM25 score (null if not surfaced by keyword search) Per-arm semantic/text scores are aggregated across retrieval arms during RRF / interleave fusion (ArmScores on MergedCandidate), since fusion otherwise keeps only the first-seen arm's score per doc. Request — `min_scores` floors (inclusive, AND-ed, opt-in; default no filtering): - semantic / text : retrieval-level cutoffs pushed into the SQL arms, overriding the global similarity / BM25 minimums for the request (prune before fusion) - reranker / final: post-query filters on the scored results There is deliberately no default threshold: the cross-encoder's absolute scores are reliable for ordering but not calibrated across queries (a clearly-relevant match can score ~0.001 on one query and ~1.0 on another), so a fixed cutoff would silently drop good results. Also surfaces proof_norm in the search trace and reworks the control-plane trace view to render scores at full precision (no rounding) and show the per-stage `scores` breakdown; relabels the trace's "CE" column to "reranker score". Threaded through engine, HTTP, MCP (both recall tools), and the control-plane proxy; OpenAPI spec, Python/TS/Go/Rust clients, and the docs-skill mirror regenerated; docs updated. |
||
|
Nicolò Boschi
|
91e095afa9 |
fix(control-plane): show pending uploaded documents from server operations (#2420)
Render in-flight and failed file uploads in the Documents view by deriving them from the server's file_convert_retain operations — no client-side store or client-generated document ids. - surface document_id + original_filename on the operations list endpoint (already stored in the operation's result_metadata) - documents-view derives pending/failed rows from those operations, deduped against the real document list by document_id, and polls while in-flight - bridge the brief window where an operation reports completed before the document becomes visible in listDocuments, so the row never flickers Supersedes #2346 (client-side sessionStorage approach). Closes #2314. |
||
|
Nicolò Boschi
|
dae18b1faf |
feat(mental-models): cron-scheduled mental model refresh (#2377)
Adds a third, independent way to refresh a mental model — on a cron schedule — alongside the existing auto (refresh_after_consolidation) and manual paths, driven by the background MaintenanceLoop ticker. API/engine: - trigger.refresh_cron (UTC 5-field cron, croniter-validated); mutually exclusive with refresh_after_consolidation. - PG-only discovery routine public.mental_models_with_cron() (migration f4d1c2b3a5e6); cron due-ness evaluated in Python, refresh only when stale. - HINDSIGHT_API_MENTAL_MODEL_REFRESH_TICK_SECONDS check cadence. - One timing line logged per maintenance sweep. Control plane: - Single "Refresh trigger" choice (Manual / On new memories / On a schedule) with per-option sub-labels; cron input shown only when scheduled. - Live cron schedule preview (human-readable + next/upcoming runs, UTC+local). - "Next refresh" shown next to "last refreshed" in list, dashboard, and dialog. - Fixed an app-wide off-by-one in formatRelativeTime. Regenerated OpenAPI + clients + bank-template schema; i18n across all locales. |
||
|
Chris Bartholomew
|
b4874672fa |
feat(tokens): propagate cached + thoughts tokens through return contexts (#2356)
* feat(tokens): propagate cached + thoughts tokens through return contexts
The Gemini 2.5+ family (and any future provider that combines prompt caching
with reasoning tokens) reports four distinct token counts on every response:
- prompt_token_count (total input)
- candidates_token_count (visible output)
- cached_content_token_count (subset of input served from prompt cache)
- thoughts_token_count (reasoning tokens, billed at output rate)
The provider already records the last two on the Prometheus
``hindsight.llm.tokens.{cached_input,thoughts}`` counters, but the values
stop at the metrics layer — every return context (TokenUsage,
LLMToolCallResult, TokenUsageSummary, RetainResult) only exposes the
top-level input/output split. As a result:
* a downstream metering extension can't attribute prompt-cache hit-rate
per operation (only globally via Prometheus aggregates), and
* reasoning-token spend is invisible to ``output_tokens`` because the
provider keeps it out of candidates_token_count. A workload that
"looks cheap" by visible output can be silently expensive if the
model is doing long reasoning chains.
This change threads the two fields through end-to-end:
- ``TokenUsage`` gains ``thoughts_tokens`` (cached_tokens already
existed); ``__add__`` sums it so multi-iteration agentic-loop
aggregation works.
- ``LLMToolCallResult`` gains ``cached_tokens`` + ``thoughts_tokens``.
- ``TokenUsageSummary`` (returned by ``run_reflect_agent``) gains
both fields and ``run_reflect_agent`` accumulates them at every
call site (main tool loop + structured-output extraction + 4
edge-case completion branches).
- ``_generate_structured_output`` now returns a 5-tuple
``(output, in, out, cached, thoughts)``; the 6 unpack sites in the
reflect agent are updated together.
- ``RetainResult`` gains optional ``llm_cached_input_tokens`` and
``llm_thoughts_tokens`` fields; ``memory_engine`` populates them
from the aggregated ``TokenUsage``. Defaults stay ``None`` for
engines that don't surface the data so existing metering extensions
are unaffected.
- The Gemini provider — which was already reading the four token
counts from the SDK response — now returns ``thoughts_tokens`` on
both the ``call`` and ``call_with_tools`` paths, and the existing
``cached_input_tokens`` value reaches ``LLMToolCallResult``.
Backward compatibility: every new field defaults to 0 (or None for the
RetainResult dataclass), so any caller built before this change keeps
working. Provider impls that don't surface these counts simply propagate
zeros — the structured Prometheus counters were already optional in
``record_llm_call``.
Adds focused tests (``test_token_usage_cached_thoughts.py``, 6 cases)
pinning the propagation through every return type and the aggregation
behavior. Existing reflect-agent + Gemini provider tests (87 cases) pass
unchanged.
This is a pure plumbing change — no metrics are renamed, no behavior is
gated, no flags are added.
* chore: regenerate clients + openapi spec for thoughts_tokens field
Picks up the new TokenUsage.thoughts_tokens field added in the parent
commit. Generated by:
./scripts/generate-openapi.sh
./scripts/generate-clients.sh
Plus ``ruff format`` over the two reflect/ source files to match the
project's enforced formatting style.
No hand edits in any generated file.
* chore: regenerate skills/hindsight-docs/references/openapi.json
* fix(reflect): return StructuredOutputResult instead of widened tuple
_generate_structured_output's return contract had drifted: the success
and no-fields branches returned a 5-tuple while the except branch still
returned a 3-tuple. All six call sites unpack five values, so any
structured-output failure would crash reflect with a ValueError instead
of degrading gracefully.
Replace the multi-item tuple return with a typed StructuredOutputResult
(per project rule: no multi-item tuple returns), making the arity
mismatch impossible and the failure path safe. Add a regression test.
---------
Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
|
||
|
Evo
|
4db8a12362 |
fix(http): reject negative limit/offset on list endpoints with 422 instead of raw Postgres 500 (#2357)
* fix(http): reject negative limit/offset on list endpoints with 422 instead of 500 Several user-facing GET list endpoints declared limit/offset without ge constraints, so a negative value flowed straight into Postgres LIMIT/OFFSET (emitted with no max(0, ...) clamp), which raises 'LIMIT/OFFSET must not be negative'. The generic `except Exception -> HTTPException(500, str(e))` then turned a client input error into a 500 that also leaked the raw Postgres error string. Add Query(ge=...) constraints (limit ge=0, offset ge=0) on the affected endpoints (graph, memories/list, documents, tags, entities, entities/graph), matching the ge constraints already enforced on the sibling list endpoints (document-chunks, directives, async-ops, audit) so FastAPI returns a clean 422 at the boundary. ge=0 rejects only negatives and preserves limit=0 (a valid empty page), so there is no behavior change for any previously-valid request. * chore: regenerate OpenAPI spec and clients for ge=0 pagination constraints Adds minimum:0 to limit/offset params across openapi.json, docs-skill spec, Go openapi.yaml, and Python clients; lint reformats the new test. --------- Co-authored-by: Nicolò Boschi <boschi1997@gmail.com> |
||
|
Nicolò Boschi
|
5f0b715517 |
feat(recall): add prefer_observations to dedupe raw facts superseded by observations (#2311)
Recalling `observation` alongside `world`/`experience` can return the same information twice — once as a raw fact and once folded into an observation consolidated from it. The opt-in `prefer_observations` flag drops any raw fact that a returned observation lists in its `source_memory_ids`, so the observation supersedes it. Dedup is by provenance (exact id membership), not semantics, and runs before recall truncation so freed slots backfill — keeping the result count at the requested budget. Disabled by default (opt-in). Internal callers — notably consolidation, which needs the raw facts it folds into observations — leave it off. Exposed on the full client surface: the maintained Python (`recall`/`arecall`) and TypeScript (`recall`) wrappers, the Rust CLI (`--prefer-observations`), the regenerated OpenAPI + low-level Python/TS/Go/Rust SDKs, the control-plane proxy + types, and the generated docs skill. Includes docs and deterministic provenance-based tests (engine + both wrappers). |
||
|
Nicolò Boschi
|
0ba613c3ce |
fix(recall): allow exact filtering of untagged/global observations (#2295) (#2364)
* fix(recall): allow exact filtering of untagged/global observations (#2295) An empty tag set with tags_match="exact" now selects only untagged (global-scope) observations — the scope that observation_scopes="shared" consolidation writes to. Previously empty/absent tags meant "no filter" in every mode, so there was no way to recall only global observations when mixing shared and tagged scopes. - tags.py: in exact mode, empty/absent tags emit an untagged-only clause (tags IS NULL OR tags = '{}') with no bind param, across the flat SQL builders, Python post-filter, and compound tag-group leaves. All other modes keep treating empty/absent tags as "no filtering". - link_expansion_retrieval.py: always run filter_results_by_tags so the exact-empty/global scope is applied (it's a no-op otherwise). - http.py + regenerated clients/docs: document the exact-empty scope. - Tests: SQL builders (flat + compound, param-offset preserved), Python post-filter, and a recall API test asserting only untagged memories return for tags=[] + tags_match="exact". * chore(docs-skill): regenerate references for untagged exact-scope recall Regenerated skills/hindsight-docs/references via generate-docs-skill.sh so the docs-skill mirror matches the updated recall/observations docs (and the canonical configuration table). Unblocks verify-generated-files. |
||
|
Nicolò Boschi
|
e1014cc790 |
Release v0.8.3
- Update version to 0.8.3 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Sync documentation to version-0.8 |
||
|
Sanderhoff-alt
|
5ee53c512f |
feat(api): add optional MarkItDown OCR support (#2145)
MarkItDown advertises image extensions, but without OCR config it can fail screenshots or scanned images with low-level no-content errors. Add server-level MarkItDown OCR config that is off by default and independent from HINDSIGHT_API_LLM_*. When OCR is enabled, the OCR API key, base URL, and model are required explicitly. Wire those settings into MarkItDown's llm_client support with a built-in OCR prompt. Image uploads now fail fast with actionable errors when OCR is disabled or required settings are missing. Docs and front-end copy explain that image OCR depends on server config and requires an OpenAI-compatible OCR/vision endpoint. Closes #927 |
||
|
Nicolò Boschi
|
d382b340f0 |
feat(api): dry-run fact extraction endpoint (preview, no persistence) (#2205)
Add POST /v1/default/banks/{bank_id}/memories/dry-run-extract — a
read-only tool that previews what the retain step would extract from
text WITHOUT changing the bank: extraction only, no entity resolution,
links, embeddings, or persistence. The "dry-run-extract" path makes the
non-mutating nature explicit.
Returns a dedicated DryRunExtractionResult: the candidate facts plus the
aggregated LLM token usage. Each fact (ExtractedFact) is a subset of the
memory-unit shape — only what a fresh extraction produces: text,
fact_type, occurred_start/end, entities[] (raw, unresolved names).
Every prompt-affecting setting is overridable per call (retain_mission,
extraction_mode, custom_instructions, chunk_size, entity_labels,
entities_allow_free_form, llm_output_language) plus the narrator
(agent_name), so a candidate config can be A/B'd against the bank's
current one. The reference date field is named `timestamp` to match the
retain item payload. The engine authenticates the tenant before reading
any bank-scoped config.
Gated by a static server-level flag HINDSIGHT_API_ENABLE_DRY_RUN_EXTRACT
(default true). Since extraction makes a real LLM call, set it to false
to remove the endpoint (returns 404) on cost/abuse-sensitive deployments.
Control plane: a "Dry-run extraction" dialog opened from the Memory Bank
actions menu (text input + raw JSON output, side-by-side).
Regenerated OpenAPI spec + Python/TypeScript/Go client SDKs.
|
||
|
Sanderhoff-alt
|
59d5319b84 |
feat(retain): make structured chunk size configurable (#2139)
Add retain_structured_chunk_size as an explicit retain chunking knob for structured inputs. When unset, structured inputs follow the effective retain_chunk_size instead of the hidden 1.5x overflow factor. Thread the setting through retain extraction, append/prepend chunking, bank config resolution, templates, MCP docs, maintained clients, generated OpenAPI artifacts, the control-plane retain strategy UI, and the Rust CLI set-config command. Validate retain_chunk_size and retain_structured_chunk_size as positive integers while allowing either value to be smaller. Keep the existing retain_max_completion_tokens check scoped to retain_chunk_size. Preserve upstream validation details for client errors through the control-plane proxy so UI alerts and toasts can show concrete configuration errors without exposing server-side failure details. Update chunking, config, hierarchical config, template, MCP, client payload, control-plane serialization, SDK-response, API-client, and retain UI validation tests for the new behavior. |
||
|
Nicolò Boschi
|
0a046f97ae |
feat(consolidation): add "shared" observation_scopes keyword (#2202)
* feat(consolidation): add "shared" observation_scopes keyword Add a "shared" value for observation_scopes that resolves to a single global, untagged scope ([[]]). Memories consolidate into one observation regardless of their tags, while the tags stay on the source facts for recall filtering. This is the supported way to deduplicate observations across volatile per-call provenance tags (e.g. per-session ids): with combined/per_tag, a unique session tag puts every retain in its own scope, so near-identical facts never dedup and accumulate one observation per session. "shared" keeps recall and the dedup probe on the same (empty) scope, fixing consolidation quality rather than only the duplicate count. - consolidator: _resolve_obs_tags_list -> [[]], _resolve_write_scopes -> [frozenset()] - API/engine type literals + OpenAPI + regenerated Python/TS/Go/Rust clients - CP client type kept in sync - docs: retain.mdx 'shared' section (+ shared vs [[]] vs [] caveat), observations.mdx dedup pointer; regenerated hindsight-docs skill mirror - tests: unit scope-resolution + e2e parallel-consolidation scope correctness * chore(opencode): apply prettier formatting to plugin.test.ts Pre-existing lint drift unrelated to this PR — CI's verify-generated-files job reformats all integrations (LINT_ALL_INTEGRATIONS) and flagged this file. Folding the one-line reflow in here to get the gate green. |
||
|
Nicolò Boschi
|
6f59a09479 |
Release v0.8.2
- Update version to 0.8.2 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Sync documentation to version-0.8 |
||
|
Ben
|
a63a0c0e70 |
docs(api): list all 3 supported webhook event types in CreateWebhookRequest (#2155)
The event_types field description said 'Currently supported: consolidation.completed', but the API actually emits and accepts all three events: - retain.completed (memory_engine.py) - consolidation.completed (memory_engine.py) - memory_defense.triggered (retain/orchestrator.py) Update the description accordingly and propagate through the regenerated OpenAPI spec + the embedded copies in the go/python/typescript clients. Description-only change; no behavior change. |
||
|
Nicolò Boschi
|
b08f43496a |
feat(observations): enumerate + filter + visualize observation scopes (#2149)
* feat(observations): enumerate + filter observations by scope Add an exact (set-equality) tag match mode, a list_observation_scopes engine method + GET /observations/scopes endpoint, and a scope filter in the control-plane Observations tab (list + graph views). A scope is the exact tag set an observation was consolidated under; the empty set is the global/untagged scope. Regenerated OpenAPI + clients + docs skill. * fix(i18n): add missing memoryDetailPanel curation* keys invalidate-memory-dialog.tsx references memoryDetailPanel.curationInvalidateTitle/ Explain/ReasonPlaceholder/Cancel/Invalidate, but these keys were never added to any locale (the parity test passed because all 10 locales lacked them equally), so the invalidate dialog logged IntlError: MISSING_MESSAGE and rendered raw key names. Add all five strings across the 10 locales. Pre-existing gap, unrelated to scopes. * fix(observations): keep scope-filter trigger single-line for long/multi tags The scope dropdown trigger relied on SelectValue, which clones the selected item's wrapping pill layout; a multi-tag or long-tag scope (e.g. [session:2, user:nicolo]) wrapped to two lines and overflowed the fixed-height control. Render a compact, single-line, truncating summary in the trigger instead, keeping the full pills only in the open dropdown list. * feat(documents): capture observation_scopes in retain_params, show in detail dialog observation_scopes passed at retain time was only persisted per source fact (memory_units), never on the document, so the document detail dialog couldn't show which scoping was requested. Capture it into documents.retain_params in _build_retain_params (alongside context/event_date/metadata) and surface it as a top-level field on the get_document response. The control-plane document detail dialog now shows an 'Observation scopes' row (mode badge or scope chips). New-documents-only by design: existing docs have no captured value and show nothing. Note: this also clarifies that all_combinations on 2 tags correctly creates 3 scopes — the transient '2' is async consolidation still in flight. * feat(observations): live consolidation refresh + scope clusters on the constellation Two UX improvements to the observations view: 1. Live refresh while consolidating. The 'In Sync' badge previously read a one-shot, up-to-60s-cached stat, so it could show green while observations were still materializing (each scope is a separate consolidation pass). The view now polls every 4s while pending_consolidation > 0, silently refreshing the observations, scope list, and badge in place until consolidation settles. 2. Group-by-scope clustering on the Constellation. A new 'Group by scope' toggle lays observations out around per-scope centroids (instead of the id-hash ring), colors each scope distinctly, and wraps each scope's nodes in a translucent, labeled convex-hull blob — so overlapping tag scopes read as visual clusters. Adds clusterKeyFn/clusterColorFn/clusterLabelFn props to Constellation and an inline monotone-chain convex hull; suppresses the heat legend while clustering. * fix(observations): cap scope dropdown height to the viewport With many scopes the scope filter dropdown grew past the bottom of the screen. Cap its height at min(60vh, --radix-select-content-available-height) so it fits the space below the trigger and scrolls for the rest, instead of overflowing. * feat(observations): make the scope filter a searchable combobox Replace the plain Select with a Popover + Command (cmdk) combobox so scopes can be searched by typing — matching the tag filter's search UX — which matters once a bank has many scopes. Uses a substring filter over each scope's tags (not cmdk's fuzzy default, which over-matches scattered letters). Keeps the compact, single-line, height-capped trigger; selection still applies exact-scope filtering. * chore(cli): mark list_observation_scopes UI-only in coverage manifest The new scope-enumeration endpoint powers the control-plane scope filter/clusters and isn't a useful end-user CLI command, so add it to the [skip] list (matches the other UI-only endpoints) to satisfy check-cli-coverage. * fix(tests): import TokenUsage from response_models #2135 removed the TokenUsage re-export from llm_wrapper, but test_load_large_batch and test_retain still imported it from there, breaking test collection across the API test jobs. Import it from response_models (where it's defined), matching every other test. |
||
|
Nicolò Boschi
|
b385393b6d |
feat(consolidation): per-scope observation limits (#2140)
Add an `observation_scope_limits` config field that overrides the bank-wide
`max_observations_per_scope` on a per-scope basis. Each rule maps a scope
pattern (a list of fnmatch tag-globs) to a limit; a consolidation scope
matches under *exact cover* — every tag matched by a glob and every glob
matched by a tag — so `["shared"]` caps the `{shared}` scope without affecting
`{run_1, shared}`, and `["run_*", "shared"]` caps the combined scope only.
The first matching rule wins; scopes matching no rule fall back to
`max_observations_per_scope`.
- config: new `HINDSIGHT_API_OBSERVATION_SCOPE_LIMITS` (JSON), hierarchical
(per-tenant/bank overridable)
- consolidator: resolve the cap per scope at slot computation; wildcards live
only in the resolution layer so the SQL count stays exact and indexed
- exposed on `BankTemplateConfig`; regenerated OpenAPI + clients
- unit tests for rule parsing, exact-cover matching, and resolution
|
||
|
Sanderhoff-alt
|
c96106cc01 |
chore: remove dead code and stale config (#2135)
Remove unreferenced backend helpers, stale UI/docs components, and unused imports across the API, control plane, clients, and integrations. Drop obsolete consolidated-observation helpers and unused scoring code, clean orphaned React/docs components, and remove stale Radix dependencies. Align release scripts, Helm docs, lockfiles, generated clients, and current API examples with the package and endpoint surface still in use. |
||
|
Nicolò Boschi
|
de22b606e7 |
feat(memory): reversible curation — edit/invalidate/revert memory units (#1976)
Edit (text/context/dates/fact_type/entities), invalidate (move to a separate
invalidated_memory_units archive, reversible), and revert raw memory units via
PATCH /memories/{id}. Tracks user edits with edited_at. Control-plane UI, docs
(Memories API page), and multi-language examples included. RFC #1951.
|
||
|
Nicolò Boschi
|
0d55a9b78d |
feat(api): per-bank LLM connectivity probe (#2034)
Adds POST /v1/default/banks/{bank_id}/health/llm so operators can verify the LLMs a
bank uses for retain / consolidation / reflect actually connect — instead of
consolidation silently stalling when the LLM is unconfigured or unreachable.
- Deliberate (non-polled) probe: one minimal real call per unique LLM config —
operations sharing a configuration are probed once and the result fanned out.
- Status only per operation: connected / not_configured / auth_failed (rejected —
usually a wrong or expired API key, the most common failure) / unreachable / timeout.
Never returns the provider, model, endpoint, API key, or raw provider error (the
detailed error is logged server-side; the auth category is derived from a 401/403 or
known auth markers in the error, and leaks nothing).
- Off by default (it makes a real provider call); enable with
HINDSIGHT_API_ENABLE_BANK_LLM_HEALTH=true. Exposed as features.bank_llm_health on
/version so the UI hides the action when disabled.
- Engine returns typed dataclasses; the handler holds no SQL and auth is enforced
in the engine. The probe reuses the bank's per-operation LLM clients.
- Control plane: a "Health" item in the bank Actions menu opens an "LLM connectivity"
dialog that probes on open (with a re-test button) and shows per-operation status,
including a clear "Invalid API key" label for auth failures.
- Regenerated OpenAPI + Python/TS/Go clients; i18n across all 10 locales; tests in
tests/test_bank_health.py.
A broader per-bank GET /health endpoint (#747) was explored but dropped as redundant
with the existing bank stats; only the connectivity probe is net-new.
|
||
|
Nicolò Boschi
|
4dc149a1ac |
Release v0.8.1
- Update version to 0.8.1 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Sync documentation to version-0.8 |
||
|
Nicolò Boschi
|
1296e9fc12 |
feat(api): config flag to skip storing raw document text (#2061) (#2062)
* feat(api): add HINDSIGHT_API_STORE_DOCUMENT_TEXT flag to skip raw text storage When set to false, the retain pipeline runs unchanged (chunking, fact extraction, embedding, entity linking) but drops the raw source text: documents.original_text is stored as NULL and chunks.chunk_text as empty. content_hash is still computed from the real text so delta-retain dedup is unaffected, and recall is unaffected because it reads from memory_units. Closes #2061 * feat(api): reject append + drop source-text reads in privacy mode Follow-up to the HINDSIGHT_API_STORE_DOCUMENT_TEXT flag, covering the features that read raw document/chunk text back: - retain update_mode='append' is now rejected when text storage is disabled (it rebuilds the document from the stored original_text, which is NULL, and would silently drop prior content). - reflect no longer offers the 'expand' tool (get chunk/document source), gated via get_reflect_tools(include_expand=...); the hallucination guards no longer hardcode 'expand' as always-allowed. - reflect's recall step no longer attaches empty source chunks. Other read sites (get-document/list-chunks/get-chunk endpoints + MCP, export/import, public recall include_chunks) already degrade gracefully to empty/None and are documented. * fix(api): get-document 200 with null text + 400 on append in privacy mode Caught while testing the flag live against a running server: - DocumentResponse.original_text was a non-optional str, so the GET document endpoint raised ResponseValidationError -> HTTP 500 when the text is NULL. Made it str | None. - The retain handler mapped all exceptions (including the append-rejection and duplicate-document_id ValueErrors) to HTTP 500. Map ValueError to 400, matching the convention used by the other endpoints. Adds an HTTP-level regression test (the engine-level test passed because get_document returns a dict, bypassing response-model validation). * feat(ui): warn when document text storage is disabled Surface the store_document_text flag so the control plane can warn users that raw source text isn't persisted: - /version feature flags now include store_document_text (regenerated OpenAPI spec + SDK clients; also picks up the earlier DocumentResponse original_text optional change). - features-context exposes it (defaults true, so the warning only shows when the server explicitly reports privacy mode). - Document detail dialog: the Content tab shows a small notice instead of an empty body when text isn't stored. - Add Document dialog: a small notice that raw text won't be kept. - i18n strings added across all locales. Adds an API test asserting /version reports the flag. * refactor: drop "privacy mode" wording; reposition document-text warnings - Remove the "privacy mode" phrasing I had introduced from comments, docstrings, test names, docs, and UI labels. The flag is described by what it does (skip storing raw document text) instead. - Add Document dialog: move the warning to just above the action buttons. - Document dialog: also show the warning on the Chunks tab. * fix(cli): handle optional document original_text original_text is now Option<String> in the generated client (it can be null when document text storage is disabled), so the CLI can't print it with {} directly. Show "(not stored)" when absent. |
||
|
Nicolò Boschi
|
8cadecb3a1 |
Release v0.8.0
- Update version to 0.8.0 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Create documentation version-0.8 |
||
|
Nicolò Boschi
|
c94935bfa2 |
feat(operations): durable progress snapshot for consolidation and batch retain (#2013)
* feat(operations): durable progress snapshot for consolidation and batch retain Long-running consolidation could look identical whether healthy or stuck: updated_at was only touched on claim/complete, with no mid-run progress, so operators couldn't tell a slow job from a frozen one without DB access (#1840). Add a best-effort heartbeat that writes a coarse {stage, processed, total, detail} snapshot into async_operations.result_metadata (top-level jsonb merge so sibling keys survive) and bumps updated_at, at phase/batch boundaries: - consolidation: scanning -> processing_batch (per round, with observation counters) -> refreshing_mental_models - batch retain: processing_sub_batch per sub-batch (split loop + small-batch path) Each call mirrors the same stage into the existing set_stage() so live worker logs and the durable row tell one story. Surface it as a typed `progress` field on the operation list/status API (OperationProgress model); null when no snapshot was recorded. Regenerate OpenAPI spec + Python/TS/Rust/Go clients. Scope is visibility only: no staleness classification or auto-kill. Tests: helper merge-without-clobber + updated_at bump, API surfacing on get/list, null-when-absent, and real-run wiring for consolidation (processed advances to total) and batch retain. * feat(control-plane): show operation progress snapshot in operations view Surface the new `progress` field (stage + processed/total + per-phase counters) that the dataplane writes for running consolidation/batch-retain operations. - Type `progress` through api.ts (listOperations + getOperationStatus) via a shared OperationProgress interface. - bank-operations-view: render a compact stage + processed/total bar under the status badge on processing rows, and a full progress block (with detail counters) in the operation details dialog. Refreshes via the existing poll. - Add the `field.progress` label to all locale message files. UI half of #1840; pairs with the dataplane progress snapshot. * fix(operations): make retain progress reach total on completion; hide on terminal ops A finished single-sub-batch retain was frozen at a pre-run "processing_sub_batch 0/1" snapshot: it was written *before* the sub-batch ran and never updated, so a completed operation looked stuck. The control-plane details dialog also rendered that leftover heartbeat regardless of status, so a completed op showed an in-progress bar. - Write the retain progress snapshot *after* each sub-batch commits (processed=i for the split loop, 1/1 for the small-batch path), so the last snapshot reaches total/total and reflects completion instead of a stale pre-run count. - Control plane: only render the progress section while status is "processing"; for terminal operations the status badge + completed_at are the source of truth. Update the retain progress test to assert the snapshot reaches total/total and the durable row reflects completion. * fix(operations): per-LLM-batch consolidation progress + live heartbeat in UI Consolidation progress was written only at the outer DB-fetch round boundary, but a whole batch of memories is processed inside a single round's LLM dispatch — so the snapshot sat at "scanning 0/N" for the entire (often minutes-long) LLM phase and only jumped at the very end, looking stuck even while healthy. - Write the snapshot per LLM batch using the cumulative processed count that the per-batch log already tracks, with cumulative observation counters in detail. processed now climbs 8/42, 16/42, … as batches commit. Drop the now-redundant round-boundary write. - Control plane: show a live "last heartbeat · Ns ago" line under the progress bar, ticking every second (only while an operation is processing) so a frozen heartbeat on an active job is visible at a glance. Add heartbeat/lastHeartbeat labels to all locales. * fix(operations): clearer consolidation stage, compact progress row, faster poll Address operator-feedback on the progress UI: - Collapse consolidation's "scanning" + "processing_batch" into one self-explanatory "consolidating" stage that advances 0/N -> N/N, instead of an opaque scan->process hop nobody could interpret. - Control plane: render the in-row progress as a single compact line (bar + count + heartbeat age) so the status column no longer stacks three rows; the full breakdown (stage, counters, labelled heartbeat) stays in the details dialog. - Poll the operations list every 2s while something is processing (was a flat 5s) so the bar and heartbeat feel live, backing off to 5s when everything is terminal. * feat(operations): chunk-level retain progress; cap consolidation total; drop detail badges - Retain now reports "storing N/total chunks" from the streaming pipeline as each consumer batch commits (threaded via a progress_callback so the engine stays decoupled and operation_id/total_chunks are already in scope). Replaces the coarse per-sub-batch tick — a long document now shows chunks committing live. - Consolidation: treat total as an estimate that grows with processed (max(total_count, processed)) so the bar never reads >100% (e.g. 58/51) when memories are retained mid-run. - Control plane: drop the per-counter detail badges from the progress dialog (noisy); the bar + stage + heartbeat carry the signal. * feat(control-plane): inline progress + heartbeat on the status badge row Put the compact progress (bar + count + heartbeat age) on the same line as the status badge instead of stacking a second row under it, so a processing row reads "⟳ processing ▓▓░ 8/42 · 5s" on one line. * feat(operations): Updated column, fixed-width status, snappy completion flash Operator-feedback polish on the operations table: - Add an "Updated" column (relative time, absolute in tooltip). Required surfacing updated_at on the operations *list* endpoint (it was only on the detail endpoint); regenerated OpenAPI + clients. - Give the status column a fixed width so the row no longer shifts left when the inline progress appears/disappears as an operation starts or finishes. - Flash a row briefly (emerald on completed, red on failed/cancelled) when it transitions to a terminal state, with a 700ms color transition, so a completion landing on a poll reads as a deliberate change instead of a silent badge swap. - Refresh the relative-time clock on every poll so the Updated column stays accurate while idle (not just while the per-second heartbeat ticker runs). * feat(control-plane): label and fix the Actions column width The Actions column had no header and no fixed width, so it grew when a pending/failed row's Cancel/Retry button appeared — shifting the whole table. Give it an "Actions" label (added to all locales) and a fixed 110px width on header and cell so the layout stays put regardless of which rows show an action button. * feat(operations): update consolidation total by re-counting instead of clamping Replace the max(total, processed) clamp (which pinned the bar at 100% once processed caught the start-of-job estimate) with a real re-count: once processed passes the initial estimate, report total = processed + still-pending. Guarded so the extra COUNT only runs after the estimate is exhausted (≈the final batch normally, or repeatedly only if memories keep arriving mid-run) — no per-batch query in the common case. Also explain it in the UI: the consolidation progress section notes that the total is an estimate from job start and can grow if new memories arrive while it runs. * fix(control-plane): label file_convert_retain as "Convert File" |
||
|
Nicolò Boschi
|
82800ba864 |
fix(ci): repair zeroentropy embedding tests and regenerate drifted clients (#2009)
* fix(openapi): keep binary upload fields as format:binary; regen spec+clients The #1982 dep bump (FastAPI 0.136 / Pydantic 2.12) serializes binary upload fields as OpenAPI-3.1 {"type":"string","contentMediaType":"application/ octet-stream"}. openapi-generator v7.10.0 (generate-clients.sh) does NOT treat contentMediaType as a file upload, so it regenerated the Files `files` and document-transfer `file` params as plain strings — silently breaking multipart upload in the Go/Python/TypeScript clients ([]*os.File -> []string, StrictBytes -> StrictStr, Blob|File -> string). generate_openapi.py now post-processes the exported schema to restore the prior `format: binary` representation (still valid under openapi 3.1.0, and what the generator understands) for application/octet-stream string fields, scoped to binary uploads only. Regenerated the spec and clients: the upload signatures are back to the file-upload form (identical to main); the only remaining delta vs main is ValidationError dropping its `url` field, a real Pydantic 2.12 change (error metadata, harmless). * test(embeddings): give zeroentropy routing mocks a dimension attribute PR #1670 added post-encode dimension validation to generate_embeddings_batch — it now reads embeddings_backend.dimension, which the EmbeddingsBackend Protocol already requires. The pre-existing QueryAwareEmbeddings/ DocumentAwareEmbeddings routing mocks (#1770) omit it, so the two routing tests started failing with AttributeError on main. The mocks return single-element vectors, so declare dimension = 1 to satisfy the Protocol and let validation pass. Pure test fix; no behavior change. * test(openapi): lock _restore_binary_format binary-upload rewrite Regression guard for the file-upload break: asserts octet-stream string fields are rewritten to format:binary (incl. nested/array-item schemas) and that other content media types are left untouched. |
||
|
Nicolò Boschi
|
2834192800 |
feat(control-plane): "not enabled" splash for disabled audit logs & LLM requests (+ bank name fix) (#1950)
* feat(control-plane): show "not enabled" splash for disabled audit logs & LLM requests Add a reusable FeatureNotEnabled component (centered icon + title + description) and use it for the Audit Logs and LLM Requests tabs, plus refactor the existing Observations splash to reuse it. Tabs gain an "Off" badge when the feature is disabled. To let the UI detect server-side gating, expose audit_log and llm_trace in the /version features object (sourced from config.audit_log_enabled / config.llm_trace_enabled), wire them through the features context and the control-plane SDK type, and add i18n keys across all 10 locales. * fix(retain): default bank name to bank_id in ensure_bank_exists ensure_bank_exists inserted banks without a name (NULL), unlike the other creation path (get_or_create_bank_profile, which defaults name to bank_id). Since #1940 wired PATCH /config to ensure_bank_exists, a config PATCH on a never-retained bank (and any retain-only bank) produced a NULL name, which then 500'd the deprecated GET /profile endpoint (name is typed as a required str). Default name to bank_id at insert so every creation path is consistent. Extends the #1940 regression test to assert the auto-created bank's profile returns 200 with name == bank_id. * test(api): assert audit_log and llm_trace flags in /version response * chore: regenerate openapi spec and client SDKs for new feature flags |
||
|
Nicolò Boschi
|
1d6d73bce4 |
feat(transfer): export/import documents between banks without re-running the LLM (#1909)
* feat(transfer): export/import documents between banks without re-running the LLM
Export a bank's already-extracted facts (text, entity canonical names, causal
links, chunks) to a ZIP archive, and import them into another bank by replaying
the deterministic half of the retain pipeline — re-embedding locally with the
target bank's model and re-resolving entities. No LLM fact extraction runs on
import. Consolidated observations are excluded (regenerated by consolidation in
the target bank).
Two use cases: testing a different embedding model, and moving data between
banks/instances without LLM cost.
- engine/transfer/: schema, export, importer (LLM-free replay)
- MemoryEngine.export_documents_async / import_documents_async
- Admin CLI: export-documents / import-documents
- HTTP API: GET/POST /v1/default/banks/{bank_id}/document-transfer
- Gated by HINDSIGHT_API_ENABLE_DOCUMENT_EXPORT_API / _IMPORT_API
(default on), surfaced via /version features for the control plane
- Control plane Documents page: Export All / Import (zip upload) +
per-document Export, hidden when the backend disables the feature
- Tests, docs, regenerated OpenAPI spec and client SDKs
* fix(transfer): trigger consolidation, graph maintenance & webhooks on import
Imported documents were second-class citizens: unlike a normal retain, an
import fired no retain.completed webhooks and never enqueued consolidation
or graph maintenance, so imported facts never produced observations.
Thread an outbox callback factory through import_documents -> _import_one_document
so each imported document fires its retain.completed webhook transactionally
inside its own insert. After the import completes, submit async consolidation
(when observations + auto-consolidation are enabled) and graph maintenance,
mirroring the post-retain side effects.
* refactor(transfer): share post-insert maintenance helper between retain and import
The consolidation + graph-maintenance triggers added for import duplicated the
retain post-processing block verbatim. Extract it into
_submit_post_insert_maintenance and call it from both the retain pipeline and
the import pipeline, so the two paths stay in lockstep.
* feat(transfer): fire on_retain_complete per imported document
Import now fires the post-retain extension hook (usage tracking / metrics /
notifications) once per imported document, mirroring retain — so imported
facts are first-class for extensions. Token counts are zero and
processed_content_tokens is 0 (import runs no LLM extraction), so cost-metering
extensions correctly bill an import as free.
The importer returns per-document outcomes (ImportedDocument) so the engine can
build the RetainResult; these are not serialized into the operation's
result_metadata (the worker still writes counts only).
Tests: assert the hook fires once per document with zero tokens, and that
import queues a retain.completed webhook delivery per document.
|
||
|
Nicolò Boschi
|
b1f6bbb8b4 |
feat(api): per-bank LLM request tracing via OTel GenAI recorder (#1922)
* feat(api): per-bank LLM request tracing via OTel GenAI recorder
Record every LLM call (success and failure) into a new `llm_requests`
table, per bank, when HINDSIGHT_API_LLM_TRACE_ENABLED=true (disabled by
default). Capture is wired into the OpenTelemetry GenAI record_llm_call
path: the DB tracer is registered as a span recorder alongside the OTLP
exporter, so providers' existing success calls flow through it and the
LLM wrapper forwards failures.
Each row stores input messages, model output, token usage
(input/output/cached/total from the provider response), finish reason,
provider/model/scope, timing, and caller metadata.
- GET /v1/default/banks/{bank}/llm-requests (+ /stats) read API
- Control-plane "LLM Requests" tab: list, filters, detail dialog, and a
Calls/Tokens chart with Total/Breakdown and Cumulative toggles
- Reusable JsonViewer component (word-wrap + copy), applied to audit logs
- TokenUsage.cached_tokens; cached-token extraction for
openai-compatible, gemini, anthropic
- Migrations for the table + token columns; backup/restore coverage
- Tests, docs, regenerated OpenAPI + SDK clients
* test(llm-trace): regression test for delta re-retain document_id binding
* feat(llm-trace): map produced/consumed memory_ids to retain & consolidation traces
Retain traces now carry metadata.memory_ids (the facts created); consolidation
traces carry metadata.source_memory_ids (memories consumed) and metadata.memory_ids
(observations created/updated). Accumulated at the DB-write sites onto the
operation-level trace context and flushed onto every row of the trace via
LLMTraceRecorder.attach_memory_ids (awaits in-flight fire-and-forget writes first
so the UPDATE never races ahead of the rows). Surfaced in the trace dialog as
'Memories created' / 'Source memories' chips.
* perf+feat(llm-trace): fire-and-forget mapping + bidirectional memory↔trace
Performance:
- attach_memory_ids is now fire-and-forget — it snapshots ids synchronously and
patches the trace on a background task, off the retain/consolidation critical
path. The pending-write flush is scoped to the operation's own trace_id
(bucketed pending set) so it never waits on unrelated operations.
Memory ↔ trace navigation:
- New memory_id filter on the llm-requests listing, matching metadata.memory_ids
(produced) OR metadata.source_memory_ids (consumed), so a memory resolves both
the run that created it and the consolidation runs that used it as a source.
- Memory detail panel shows 'Created by' and 'Used by' sections opening the
trace dialog. Regenerated OpenAPI spec + SDK clients.
* ui(llm-trace): rename 'Used by' to 'Consolidated by' on memory trace panel
* chore(clients): regenerate SDK clients after merge (llm_requests endpoints)
* ci(cli-coverage): mark llm_requests tracing endpoints UI-only
* fix(control-plane): drop invalid 'as const' on ternary (prod build typecheck)
* fix(llm-trace): guard trace_context() access for mock/substitute providers
run_consolidation_job and retain read the operation trace context off the
configured provider, but tests substitute a bare MockLLM without a
trace_context() method, which AttributeError'd and crashed all consolidation.
Add trace_context_of() to read it defensively (None when unsupported), so
tracing degrades gracefully and never breaks the operation.
|
||
|
Nicolò Boschi
|
ffd7f94572 |
Release v0.7.2
- Update version to 0.7.2 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Sync documentation to version-0.7 |
||
|
Nicolò Boschi
|
779e3140c8 |
Release v0.7.1
- Update version to 0.7.1 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Sync documentation to version-0.7 |
||
|
Sanderhoff-alt
|
2de19e578b |
fix(api): anchor recall recency to query timestamp (#1788)
Use recall question_date/query_timestamp as the reference time for combined scoring instead of always using server utcnow(). This keeps historical replay and offline evaluations from penalizing memories that were recent at query time. Normalize naive query timestamps to UTC before scoring, update API/client/OpenAPI/docs/MCP descriptions, and add recall-level coverage proving combined scoring receives the query-time anchor. |
||
|
Nicolò Boschi
|
ded52e8de6 |
Release v0.7.0
- Update version to 0.7.0 in all components - Regenerate OpenAPI spec and client SDKs - Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed - Python client: hindsight-clients/python - TypeScript client: hindsight-clients/typescript - hindsight-all npm wrapper: hindsight-all-npm - Rust CLI: hindsight-cli - Control Plane: hindsight-control-plane - Helm chart - Create documentation version-0.7 - Fix broken link to consolidate endpoint in configuration docs |
||
|
Nicolò Boschi
|
ac3ab2b54c |
feat(api): add targeted consolidation by observation scopes (#1746)
* feat(api): add targeted consolidation by observation scopes (#1625) Add `observation_scopes` parameter to the consolidate endpoint to run consolidation only on memories matching specific tag scopes, and add `enable_auto_consolidation` config flag to disable automatic post-retain consolidation. * docs: add targeted consolidation and auto-consolidation config docs Update observations docs with targeted consolidation section, trigger consolidation endpoint reference, and auto-consolidation disable flag. Regenerate OpenAPI spec and client SDKs. * docs: add enable_auto_consolidation to banks API docs |
||
|
Nicolò Boschi
|
7bd11bedf6 |
feat(api): add clear endpoint for mental model content (#1706)
* feat(api): add clear endpoint for mental model content (#1706) Add POST /mental-models/{id}/clear that resets content to empty so the next refresh performs a full re-synthesis regardless of trigger mode. Useful for periodic compaction of delta-mode models that accumulate drift over many incremental refreshes. * docs: add SDK code examples for clear_mental_model Add clear_mental_model to Python and TypeScript wrapper clients, and add code snippets (Python, Node.js, CLI, Go) to the mental models docs page using the same CodeSnippet pattern as other operations. * ci: add clear_mental_model to CLI coverage skip list * fix: update MCP tool count assertion for clear_mental_model |