4 Commits

Author	SHA1	Message	Date
Nicolò Boschi	99a319513f	fix(knowledge-pages): move a page's scope onto its mental model, and stop trigger patches resetting it (#3687) (#3755) ... * docs(knowledge-pages): say that a page's tags are a filter, and let the dialog widen it (#3687) A page's `tags` are the scope it is synthesized from, not labels on it, and a tagged page matches with `all_strict` by default: a memory must carry EVERY tag and untagged memories are excluded outright. Nothing said so. The create-page API documented `tags` as "tags that scope which memories the page is built from" with no match mode, and `type:<x>` is documented as setting the page's rendered *type* — so a page created with `["type:runbook", "homelab", "infrastructure"]` silently required all three on every memory, matched nothing, and generated as "I don't have information about this" while a plain recall for the same query returned 81 results scoring 1.087. The server default stays. What changes is that you can now see it and move it: - The control-plane create/edit dialog states the rule under the tags field and, once there is a tag to widen, offers "Also build from untagged memories" -> `trigger.tags_match: "all"`. The edit dialog prefills from the page's stored mode and only sends `tags_match` when the checkbox actually moved, so a page on `any`/`any_strict`/`exact` survives a rename untouched. - `CreatePageRequest.tags`, `UpdateNodeRequest.tags` and `MentalModelTrigger.tags_match` carry the rule in the OpenAPI spec, so it reaches Swagger UI and the generated SDKs; both hand-written wrappers (TS `HindsightClient`, Python `Hindsight`) get it in parity. - A "Tags Are a Filter" section in the API docs works the reported page through, with the three ways to scope one and the `PATCH {"tags": []}` repair. Also corrects a stale claim in the same three places: a supplied `trigger` has merged over the page defaults since #3506, but the docs and both wrapper docstrings still said it REPLACES them and told you to repeat the fields you wanted to keep. Under that text the fix above reads as destructive. * fix(knowledge-pages): move a page's scope onto its mental model, and stop trigger patches resetting it (#3687) Two changes, one subject: where a knowledge page's retrieval scope is edited, and whether editing it destroys the rest of the page's configuration. 1. The page dialogs no longer take tags. `tags` on a page are the scope it is synthesized FROM, and a tagged page matches with `all_strict`: every tag required, untagged memories excluded. The create dialog offered a bare text input for them next to a hint that a `type:<x>` tag "sets the page's type" — so tags typed there to describe a topic silently became a hard filter that matched nothing, and the page generated as "I don't have information about this". A partial copy of the scope controls was the problem, so the copy is gone rather than extended: the page dialogs now take a name and a source query, and the page links to its backing mental model, which already owns the whole scope (tags, tags_match, tag_groups, fact types, schedule). Pages are created reading the whole bank; narrowing one is now a deliberate step taken where the match mode is visible next to the tags it governs. The edit dialog no longer sends `tags` at all. That is load-bearing: the PATCH applies only the keys present, so a dialog that no longer shows the field must not send it, or every rename would clear the page's scope. 2. `PATCH /mental-models/{id}` patches its trigger instead of replacing it. Which matters much more now that it is where pages send people. The route dumped the whole request model and the engine wrote that dict wholesale, so setting one field stamped `MentalModelTrigger`'s own defaults over every field left unset: a page edited there lost `mode: delta`, its observation-only `fact_types`, and `exclude_mental_models`, quietly becoming a from-scratch rebuild that also reflected over its sibling pages. #3506 fixed exactly this, but only on the two page routes. The MCP `update_mental_model` tool drives the same endpoint with a one-key dict (`{"refresh_after_consolidation": ...}`), so it was wiping triggers too. `_merge_page_trigger` is renamed `_merge_trigger` and is now the shared merge for any mental model; a full trigger still replaces, which keeps bank-template import declarative. Verified against a live API: a page created through the UI, opened via its new "Advanced options" link and saved from the mental-model editor, keeps `mode: delta`, `fact_types: [observation]`, `exclude_mental_models` and `refresh_after_consolidation`. Four of the five new regression tests fail without the engine change; the fifth is the guard that a complete trigger still replaces. * fix(knowledge-pages): open the backing mental model in place, and name it in the link (#3687) The link opened a page's scope editor by navigating to the Mental Models tab. Changing a page's scope is part of working on the page, so sending the reader to another tab — losing the page they were reading, and their place in the tree — was the wrong trade for a shareable URL. It now opens the same detail modal in place, on its configuration, and hands off to the very same `UpdateMentalModelDialog` the Mental Models tab uses (exported for it, not copied). Saving re-pulls the tree and the open page, since tags and trigger drive the chips and the freshness line. The label says "Mental model options" rather than "Advanced options": what opens is that model, and the page's scope living on a mental model is exactly the thing a reader needs told. The ?mentalModel= deep link added for the navigation is removed with it — nothing links there now, and a query param nothing produces is a trap for the next reader. * refactor(knowledge-pages): name the options i18n key for what it opens (#3687) The key was still `advancedOptions` after the label became "Mental model options", and the refresh callback asserted `currentBank!` where guarding is free.	2026-08-24 14:06:12 +02:00
Nicolò Boschi	a5d00660b5	fix(knowledge-base): answer page staleness per scope, not from the bank watermark (#3291) (#3589) ... The knowledge tree, the mental-model list and the control-plane stats card derived staleness from one bank-wide write watermark: any write flagged every page that had not read it. Only an in-scope write can move a page's own watermark, so a page whose scope went quiet stayed flagged indefinitely while the refresh gate — which does check each page's own scope — correctly refused to refresh it. The reporter measured 24 of 29 pages flagged for hours against 0 of 29 by the exact check. The approximation existed because the exact check cost a scan of the bank's memories per page: memory_units had no index involving updated_at, so `WHERE bank_id = $1 AND updated_at > $2` could not stop early on the common negative answer. Add (bank_id, updated_at DESC) and that cost goes from 10.7ms (tagged page, fresh) and 29.8ms (untagged model) to 0.046ms and 0.021ms on a 400k-row table. With the index the exact answer is affordable, so those surfaces now ask it, batched: any_memory_updated_since_batch answers a whole bank in one query per scope shape (30 pages in 0.2-0.5ms). Reflect's search_mental_models, which ran the scoped check per model inside every reflect call, uses the same batch. The watermark shortcut is kept for callers holding a live watermark, but the polling surfaces no longer pass the 60s-cached one — it can be older than a model's own last read and would prove a freshness that is not there. _bank_write_watermark had no callers left and is removed. Staleness still asks only what has been *written*; deletions leave no write behind and remain invisible, which the API and docs now state outright.	2026-08-19 14:15:02 +02:00
Nicolò Boschi	0ca0e87a08	fix(control-plane): report mental-model freshness from the bank write watermark (#3156) ... * fix(control-plane): report mental-model freshness from the bank write watermark The mental-models card compared each model's last_refreshed_at against the bank's last_consolidated_at, so any consolidation after a refresh — nearly always — reported every model as stale, and a bank that had never consolidated reported the opposite. Computing the real per-model answer on a list is the expensive fix: compute_mental_model_is_stale has no index to use (there is none on memory_units.updated_at), so it scans the bank's memories in full, per model — 10ms per model at 100k memories, 101ms at 500k, on a view that polls every 5s. Report a bank-wide watermark instead. MAX(updated_at) rides along on the aggregate _compute_bank_stats already runs and is served from the same cached payload as last_memory_write_at. A model refreshed at or after it is up to date, exactly; older only means something was written, possibly outside its tags, so the card says "may need refresh" rather than asserting stale. The exact answer stays on the single mental-model read, behind the dialog. The knowledge-base tree ran that same scan once per page and polls every 12s — already a full scan per page per tick in production. It now shares the watermark: one cached lookup for the whole tree. Fixes #3139 * perf(reflect): skip the per-model staleness scan below the bank watermark search_mental_models computed staleness with the exact scoped query for every model it returned — up to 5 full scans of the bank's memories per tool call, serially, on a held connection, and the agent can call the tool several times per reflect. get_bank_freshness already computes the bank's write watermark in the same scan it runs once per reflect, and was discarding it. Thread it through: a model refreshed at or after the newest write in the bank cannot be stale whatever its scope, so it skips the query entirely. Everything above the watermark still gets the exact tag-aware answer — the agent only trusts a model without a verifying recall() when is_stale is False, so guessing conservatively here would buy LLM turns to save a query. * chore(docs-skill): re-sync the generated reference copies generate-docs-skill.sh output drifted from the docs pages that landed on main (retain narrator guidance, configuration, coding-agents). Regenerated so verify-generated-files has nothing to report.	2026-08-04 16:53:19 +02:00
Nicolò Boschi	4d22a882f5	docs(knowledge-pages): document Knowledge Pages and Mental Models, and manage them from the CLI (#3151) ... * docs(knowledge-pages): document Knowledge Pages and Mental Models Knowledge Pages shipped in #2455 with no documentation at all — no architecture page, no API page, no mention in the sidebar. Mental models had an API page but nothing explaining what they are or why they are fast. Add both, as top-level entries under Architecture and API. - Architecture: how pages are mental models with a simplified, document-shaped configuration; the folder hierarchy; the `hindsight fs` filesystem projection; page-level search; and why a projected view over reconciled memory is not the same thing as a folder of raw files. - Architecture: mental models as standing answers built in the background, so an application reads the current version instead of paying for synthesis on the request path. - API: the full knowledge-base endpoint surface, the page defaults and what each one buys, staleness gating, what a refresh reads, and how delta mode edits a structured document instead of regenerating prose. The mental-model trigger table gains the seven settings it was missing. - FAQ: mental model vs knowledge page. Also corrects the neighbouring answer, which described mental models as built automatically during retain — that is observations. The API examples use the maintained clients like every other API page, so this adds the knowledge-base surface to the Python and TypeScript wrappers (kept at parity, with request-mapping tests on both sides) and runnable Python/Node/Go examples. * feat(cli): manage knowledge pages from the CLI The knowledge base was reachable from every client except the CLI, where the eight endpoints were listed as deliberate coverage skips ("managed in the control plane UI"). That left `hindsight fs` able to mirror pages read-only but nothing able to create, edit, search, or delete them — and it meant the API docs could not show a CLI tab alongside Python/Node/Go. Adds `hindsight knowledge-base` with tree, create-folder, create-page, get-page, search, update, delete, and export, removing the skips so cli-coverage-check enforces the surface from here on. `create-page` sends no trigger unless --mode or --fact-types is passed, so the server's page defaults stand; when either is given the whole trigger has to be restated, because a supplied trigger replaces the defaults rather than merging with them. Also adds the CLI tab to the Knowledge Pages API page and a Knowledge Base section to the CLI reference.	2026-08-03 14:53:57 +02:00