mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Nicolò Boschi
9e6d9e76cc
* feat(api,control-plane): a prompt tester for retain, and prompt preview for every operation Closes the "render prompts without calling an LLM" half of #3774. A bank's missions only mean something once you can see the prompt they land in, and today that means tracing Python constants and format calls. `POST /banks/{id}/prompts/preview` returns the messages retain, consolidation or reflect would send — in send order, no LLM call, no writes. The operation is the whole request: everything comes from the bank, and the runtime data an operation would be given is a fixed placeholder. A message arrives as `blocks`. The active ones concatenate back to the exact text sent — enforced by a test against what the extraction path itself builds. Each is identified by machine values only (`field`, a `section` slug, or the `heading` the prompt text carries); the response ships no display copy, so names and explanations live in the UI that localises them. An inactive block has no text and marks a setting switched off at the point it would land, so an unset mission is still visible where it would go. Both messages always come back: retain and consolidation keep their system prompt bank-agnostic so one provider-side cache serves every bank, and carry the mission in the user message instead. Only reflect puts its mission in the system prompt. **Two bugs found on the way, both pre-existing in dry-run extraction:** - Neither dry-run nor the preview applied retain strategies. Both resolved config directly instead of through `_resolve_retain_config`, so they ignored the bank's `retain_default_strategy` too — silently extracting and previewing under settings a real retain would never use. Both now resolve the way retain does and take an optional `strategy`. - The preview read the bank's config without running `validate_bank_read`, so a tenant extension denying `GET_BANK_CONFIG` was bypassed by an endpoint that renders that config as prompt text. Both paths now share `_authorize_bank_config_read`, which also carries the bank-existence check. **Control plane.** The dry-run dialog is gone; its work moved into the prompt tester on the bank Configuration tab, because changing a mission and seeing what it extracts is one loop that was split across two dialogs. Blocks re-render for free as settings change; a sample-text box and a Run button spend the LLM call on demand. A strategy picker renders any of the bank's named strategies. Editing a block saves that setting to the bank; `editable` comes from the config layer's own allowlist, so server-level fields say so rather than offering an edit that would collect a 400. `PromptBlockModel.kind` is required with no default: progenitor rejects a default on an inline enum with TypeError(InvalidValue), which breaks the Rust client. * chore(cli): skip preview_prompt in the OpenAPI coverage manifest