vectorize-io__hindsight/hindsight-docs/docs/developer/api/memory-banks.mdx

---
sidebar_position: 6
---

# Memory Banks

Memory banks are isolated containers that store all memory-related data for a specific context or use case.

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeSnippet from '@site/src/components/CodeSnippet';

{/* Import raw source files */}
import memoryBanksPy from '!!raw-loader!@site/examples/api/memory-banks.py';
import memoryBanksMjs from '!!raw-loader!@site/examples/api/memory-banks.mjs';
import memoryBanksSh from '!!raw-loader!@site/examples/api/memory-banks.sh';
import memoryBanksGo from '!!raw-loader!@site/examples/api/memory-banks.go';
import directivesPy from '!!raw-loader!@site/examples/api/directives.py';
import directivesMjs from '!!raw-loader!@site/examples/api/directives.mjs';
import directivesSh from '!!raw-loader!@site/examples/api/directives.sh';
import directivesGo from '!!raw-loader!@site/examples/api/directives.go';

## What is a Memory Bank?

A memory bank is a complete, isolated storage unit containing:

- **Memories** — Facts and information retained from conversations
- **Documents** — Files and content indexed for retrieval
- **Entities** — People, places, concepts extracted from memories
- **Relationships** — Connections between entities in the knowledge graph
- **Directives** — Hard rules the agent must follow during reflect operations

Banks are completely isolated from each other — memories stored in one bank are not visible to another.

You don't need to pre-create a bank. Hindsight will automatically create it with default settings when you first use it.

Only *writes* create a bank — retaining, updating its profile, changing its config. **Reads of a
bank that does not exist return `404`**, so a typo'd, renamed or deleted `bank_id` is reported
rather than answered with empty results. That matters if you are monitoring a bank: `GET .../stats`
on a missing bank fails loudly instead of returning zeroed counters that look like a healthy,
empty bank.

:::tip Prerequisites
Make sure you've completed the [Quick Start](./quickstart) to install the client and start the server.
:::

## Creating a Memory Bank

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={memoryBanksPy} section="create-bank" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={memoryBanksMjs} section="create-bank" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={memoryBanksSh} section="create-bank" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={memoryBanksGo} section="create-bank" language="go" />
</TabItem>
</Tabs>

## Bank Configuration

Each memory bank can be configured independently per operation. Configuration can be set via the [bank config API](#updating-configuration), the [Control Plane UI](/), or [server-wide environment variables](/developer/configuration).

### retain_mission {#retain-configuration}

A plain-language description of what this bank should pay attention to during extraction. The mission is injected into the extraction prompt alongside the built-in rules — it steers focus without replacing the extraction logic.

```
e.g. Always include technical decisions, API design choices, and architectural trade-offs.
     Ignore meeting logistics, greetings, and social exchanges.
```

Works alongside any extraction mode. Leave blank for general-purpose extraction.

### retain_extraction_mode

Controls how aggressively facts are extracted:

| Mode | Description |
|------|-------------|
| `concise` *(default)* | Selective — only facts worth remembering long-term |
| `verbose` | Captures more detail per fact; slower and uses more tokens |
| `custom` | Write your own extraction rules via `retain_custom_instructions` |
| `verbatim` | Stores each chunk's original text as one memory; the LLM extracts metadata such as entities and dates |
| `chunks` | Stores each chunk as one memory without an LLM call; only caller-provided entities are available |

### retain_custom_instructions

Only active when `retain_extraction_mode` is `custom`. Replaces the built-in extraction rules entirely with your own instructions.

### retain_chunk_size

Maximum number of characters per chunk when splitting content for fact extraction. Larger chunks mean fewer LLM calls but may reduce extraction quality on long inputs; smaller chunks improve granularity at the cost of more calls.

Default: `3000`

### retain_structured_chunk_size

Maximum number of characters for a single JSONL line or conversation turn to keep whole when it exceeds `retain_chunk_size`. When unset, the limit is exactly `retain_chunk_size`; set a larger value for structured logs or chat transcripts where splitting a single record would lose useful context.

Default: unset, which uses `retain_chunk_size`

See [Retain configuration](/developer/configuration#retain) for environment variable names and defaults.

### entity_labels {#entity-labels}

Defines a controlled vocabulary of `key:value` classification labels extracted at retain time and stored as entities. Because labels become entities, they automatically link memories in the knowledge graph (two memories with `pedagogy:scaffolding` are linked), improve semantic and BM25 retrieval, and optionally filter memories via the standard `tags`/`tags_match` API when `tag: true` is set on a group.

Each entry in `entity_labels` is a **label group** — one classification dimension:

```json
{
  "entity_labels": [
    {
      "key": "engagement",
      "description": "Student engagement level during the session",
      "type": "value",
      "optional": true,
      "values": [
        { "value": "active",  "description": "Student is actively participating" },
        { "value": "passive", "description": "Student is listening but not participating" }
      ]
    },
    {
      "key": "pedagogy",
      "description": "Teaching strategies used",
      "type": "multi-values",
      "values": [
        { "value": "scaffolding",          "description": "Breaking complex tasks into smaller steps" },
        { "value": "direct_instruction",   "description": "Explicit explanation by the teacher" },
        { "value": "socratic_questioning", "description": "Guiding through questions rather than answers" }
      ]
    }
  ]
}
```

| Field | Default | Description |
|-------|---------|-------------|
| `key` | — | Label group identifier. Becomes the prefix in `key:value` entities (or `key:field:value` for `"map"`). |
| `description` | `""` | Shown to the LLM to guide label assignment. |
| `type` | `"value"` | `"value"` → pick one enum value; `"multi-values"` → pick multiple; `"text"` → free-form string; `"multi-text"` → any number of free-form strings; `"map"` → structured group with named fields. |
| `values` | `[]` | Allowed values for `"value"` and `"multi-values"` types. Ignored for `"text"`, `"multi-text"` and `"map"`. |
| `fields` | `{}` | Field definitions for `"map"` types. Each field is itself typed (`"text"`, `"multi-text"`, `"value"`, `"multi-values"`, or nested `"map"`). Ignored for non-map types. |
| `optional` | `true` | When `true` the LLM may skip the label if not applicable. When `false` the LLM must always assign a value. Has no effect on `"multi-values"` or `"multi-text"` groups (always optional). |
| `tag` | `false` | When `true`, extracted `key:value` labels are also written as tags on the memory unit, enabling filtering via `tags`/`tags_match` in recall/reflect. |

**Enum groups** (`type: "value"` or `type: "multi-values"`): the LLM picks from the predefined `values` list; anything outside the list is silently dropped. Vocabulary stays stable and graph links stay tight. Use `"multi-values"` when a fact can belong to several values at once.

**Free-text groups** (`type: "text"`): the LLM writes any string. Use the `description` field to provide examples and guidance. Graph clustering is less reliable than with enum groups because the model may phrase the same concept differently across sessions.

```json
{
  "key": "topic",
  "description": "Specific subject being discussed. Examples: algebra, quadratic equations, geometry.",
  "type": "text",
  "optional": true,
  "values": []
}
```

**Open-vocabulary groups** (`type: "multi-text"`): like `"text"`, but the LLM writes a *list* of strings instead of one — so a single fact can carry several values that nobody enumerated in the bank config. Use it when the interesting values only exist in the content: the names a thing is known by (canonical name plus abbreviations, acronyms and alternative spellings), ticket references a fact cites, product codes or SKUs. Each value becomes its own `key:value` entity, and with `tag: true` each is written as a tag, so a bank can derive a classification from content and then filter on it at recall without the caller supplying the vocabulary.

```json
{
  "key": "name",
  "description": "Every name the subject of this fact is known by — canonical name plus abbreviations, acronyms, short forms and alternative spellings.",
  "type": "multi-text",
  "tag": true
}
```

Retaining *"We deploy services to a Kubernetes cluster on EKS — the team usually just says k8s, or kube"* yields the entities and tags `name:kubernetes`, `name:k8s` and `name:kube`, so `{"tags": ["name:k8s"], "tags_match": "any_strict"}` finds the fact. A `"text"` group would keep only one of the three.

**Map groups** (`type: "map"`): defines a structured entity type with named fields. Each field is itself typed (`"text"`, `"multi-text"`, `"value"`, `"multi-values"`, or nested `"map"`) so you can describe rich entities like a person with name, role, and organization. Each extracted field is stored as a flat `key:field:value` entity string (e.g. `person:name:Alice`), reusing the existing entity storage with no schema changes — so map fields participate in the knowledge graph and retrieval the same way single-value labels do.

```json
{
  "key": "person",
  "description": "A person mentioned in the text",
  "type": "map",
  "fields": {
    "name":         { "type": "text", "description": "Full name of the person" },
    "role":         { "type": "text", "description": "Job title or role" },
    "organization": { "type": "text", "description": "Company or organization" }
  }
}
```

**How label entities resolve.** Regular entities resolve fuzzily so close name variants merge ("Alice" / "Alice Chen"). Label entities are different: their canonical names are user-defined, so two similar-looking values (`use:use-001` / `use:use-002`) must stay distinct. They therefore resolve by **exact match only** and are stored with `entity_kind = "label"`, which keeps them out of fuzzy name matching entirely — a free-text label group accumulating thousands of similar values doesn't slow down resolution of the bank's regular entities. The classification is fixed when the entity is first stored; removing a label group later doesn't reclassify its existing entities.

### entities_allow_free_form

By default, entity labels are extracted **alongside** regular named entities (people, places, concepts). Set to `false` to disable free-form extraction so only label entities are stored:

```json
{
  "entity_labels": [...],
  "entities_allow_free_form": false
}
```

### enable_observations {#observations-configuration}

Toggles observation consolidation on or off. When `false`, no consolidation runs for this bank — neither automatic nor manual. Defaults to `true` when the observations feature is enabled on the server.

### enable_auto_consolidation

Controls whether consolidation runs automatically after retain, delete, and update operations. When `false`, consolidation only runs when explicitly triggered via the [consolidate endpoint](/developer/observations#trigger-consolidation). Defaults to `true`.

This is useful when you want full control over consolidation timing — for example, batching many retains before consolidating, or running [targeted consolidation](/developer/observations#targeted-consolidation) for specific scopes only.

### observations_mission

Defines what this bank should synthesise into durable observations. Replaces the built-in consolidation rules entirely — leave blank to use the server default.

```
e.g. Observations are stable facts about people and projects.
     Always include preferences, skills, and recurring patterns.
     Ignore one-off events and ephemeral state.
```

### consolidation_llm_batch_size

Number of facts sent to the LLM in a single consolidation call. Higher values reduce LLM calls and improve throughput at the cost of larger prompts. Set to `1` to disable batching. Leave unset to use the server default (`8`).

### consolidation_source_facts_max_tokens

Total token budget for source facts included with observations in the consolidation prompt. Source facts give the LLM evidence to compare new facts against existing observations. `-1` = unlimited. Leave unset to use the server default (`-1`).

### consolidation_source_facts_max_tokens_per_observation

Per-observation token cap for source facts in the consolidation prompt. Each observation independently gets at most this many tokens of source facts, preventing a single observation with many source facts from consuming the entire budget. `-1` = unlimited. Leave unset to use the server default (`256`).

See [Observations configuration](/developer/configuration#observations) for environment variable names and defaults.

### reflect_mission

A first-person narrative that provides identity and framing context for `reflect`. The agent uses this to ground its reasoning and apply a consistent perspective.

```
e.g. You are a senior engineering assistant.
     Always ground answers in documented decisions and rationale.
     Ignore speculation. Be direct and precise.
```

### disposition_skepticism

How skeptical vs trusting the bank is when evaluating claims during `reflect`. Scale 1–5.

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={memoryBanksPy} section="bank-with-disposition" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={memoryBanksMjs} section="bank-with-disposition" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={memoryBanksSh} section="bank-with-disposition" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={memoryBanksGo} section="bank-with-disposition" language="go" />
</TabItem>
</Tabs>

| Value | Behaviour |
|-------|-----------|
| `1` | Trusting — accepts information at face value |
| `3` *(default)* | Balanced |
| `5` | Skeptical — questions and doubts claims |

### disposition_literalism

How literally to interpret information during `reflect`. Scale 1–5.

| Value | Behaviour |
|-------|-----------|
| `1` | Flexible — reads between the lines, considers context |
| `3` *(default)* | Balanced |
| `5` | Literal — takes things exactly as stated |

### disposition_empathy

How much to weight emotional context when reasoning during `reflect`. Scale 1–5.

| Value | Behaviour |
|-------|-----------|
| `1` | Detached — focuses on facts and logic |
| `3` *(default)* | Balanced |
| `5` | Empathetic — considers emotional context |

:::info
Disposition traits and `reflect_mission` only affect the `reflect` operation. `retain_mission` and `observations_mission` are separate per-operation settings.
:::

### mcp_enabled_tools

An allowlist of MCP tool names that are enabled for this bank. When set, only the listed tools can be invoked; any tool not in the list returns an error (tools still appear in the MCP tools list for protocol compatibility). Set to `null` (or omit) to allow all tools.

```json
["recall", "reflect"]
```

Available tool names: `retain`, `recall`, `reflect`, `list_banks`, `create_bank`, `list_mental_models`, `get_mental_model`, `create_mental_model`, `update_mental_model`, `delete_mental_model`, `refresh_mental_model`, `list_directives`, `create_directive`, `delete_directive`, `list_memories`, `get_memory`, `list_documents`, `get_document`, `delete_document`, `list_operations`, `get_operation`, `cancel_operation`, `list_tags`, `get_bank`, `get_bank_stats`, `update_bank`, `delete_bank`, `clear_memories`, `get_knowledge_base_tree`, `search_knowledge_base`, `get_knowledge_page`, `create_knowledge_folder`, `create_knowledge_page`, `update_knowledge_node`, `delete_knowledge_node`.

### llm_gemini_safety_settings

Controls content filtering thresholds for Gemini and VertexAI providers. Accepts a list of safety setting objects in the [Google AI safety settings format](https://ai.google.dev/api/generate-content#v1beta.SafetySetting). When `null` (default), Gemini's built-in safety defaults are used.

```json
[
  {"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"},
  {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"}
]
```

Only applies when `HINDSIGHT_API_LLM_PROVIDER` is `gemini` or `vertexai`.

### recall_budget_function {#recall-budget-configuration}

Selects how the [`recall` request's `budget` parameter](./recall) (`low` / `mid` / `high`) maps to the internal `thinking_budget` integer used by every retrieval method (semantic, BM25, graph, temporal). Two functions are supported:

| Function | Behaviour |
|----------|-----------|
| `fixed` *(default)* | `thinking_budget = recall_budget_fixed_<level>` — independent of `max_tokens`. Preserves legacy behavior. |
| `adaptive` | `thinking_budget = round(max_tokens * recall_budget_adaptive_<level>)`, clamped to `[recall_budget_min, recall_budget_max]`. Retrieval breadth scales with the requested output size. |

```json
{
  "recall_budget_function": "adaptive",
  "recall_budget_adaptive_low": 0.05,
  "recall_budget_adaptive_mid": 0.1,
  "recall_budget_adaptive_high": 0.3,
  "recall_budget_min": 30,
  "recall_budget_max": 1500
}
```

### recall_budget_fixed_low / recall_budget_fixed_mid / recall_budget_fixed_high

When `recall_budget_function` is `fixed` (the default), these positive integers are used directly as the per-method retrieval limit for each `budget` level. Defaults: `100` / `300` / `1000` — exactly matching the legacy hardcoded mapping.

### recall_budget_adaptive_low / recall_budget_adaptive_mid / recall_budget_adaptive_high

When `recall_budget_function` is `adaptive`, these positive ratios multiply the request's `max_tokens` to derive the per-method retrieval limit. Defaults: `0.025` / `0.075` / `0.25` — chosen to roughly match the fixed defaults at `max_tokens = 4096`.

### recall_budget_min / recall_budget_max

Floor and ceiling applied to the result of the adaptive function (after the ratio multiplication). Both must be positive integers and `min ≤ max`. Defaults: `20` / `2000`.

See [Recall budget mapping](/developer/configuration#recall-budget-mapping) for environment variable names and full defaults.

### memory_defense {#memory_defense}

Per-bank Memory Defense policy. Defaults to absent (Memory Defense disabled on this bank).

| Field | Type | Default | Description |
|---|---|---|---|
| `enabled` | bool | `false` | Master switch. |
| `default_action` | `allow`\|`redact`\|`quarantine`\|`block` | `allow` | Fallback action when no rule matches. |
| `protected_tag_namespaces` | `list[str]` | `[]` | Writes with tags in these namespaces (`ns:*`) are subject to the `protected_key` detector. |
| `immutable_tag_namespaces` | `list[str]` | `[]` | Writes to these namespaces are blocked. |
| `rules` | `list[Rule]` | `[]` | Detector-to-action mappings (see below). |
| `detector_overrides` | `dict` | `{}` | Per-detector tuning (e.g. `size_anomaly.max_size`). |

`Rule` shape:

| Field | Required | Description |
|---|---|---|
| `on` | yes | Detector name (`prompt_injection`, `sensitive_data`, `protected_key`, `immutable_key`, `size_anomaly`) or `*` for any. |
| `action` | yes | One of `allow`, `redact`, `quarantine`, `block`. |
| `min_severity` | no | Minimum severity (`low`, `medium`, `high`, `critical`) for the rule to fire. Defaults to `low`. |

Invalid policies are rejected on PATCH with HTTP 422.

See the [Memory Defense guide](../memory-defense/index.md) for usage examples.

---

## Previewing Prompts

A mission only means something once you can see the prompt it lands in. `POST /v1/default/banks/{bank_id}/prompts/preview` renders the exact messages `retain`, `consolidation` or `reflect` would send for a bank — no LLM call, no reads, nothing stored.

The operation is the whole request. Everything that shapes the prompt is read from the bank — its resolved config, profile and directives — and the runtime data an operation would be given is a fixed bracketed placeholder:

```bash
curl -X POST "$HINDSIGHT_URL/v1/default/banks/my-bank/prompts/preview" \
  -H "Content-Type: application/json" \
  -d '{"operation": "retain"}'
```

For `retain`, add `"strategy": "<name>"` to render under one of the bank's named retain strategies. Omit it and the bank's `retain_default_strategy` applies — exactly as it does for a retain that names none — so what you see is what retain would send. The response echoes the strategy that applied in `strategy`, and lists the bank's strategy names in `strategies` so a picker needs no second call.

There is deliberately nothing to override. A preview answers "what does this bank send"; letting a caller pass its own mission or sample text only moves that question somewhere the bank cannot answer it. To try a candidate value, save it and look again — the response says which settings are editable.

The response carries the messages in send order, each broken into the blocks it is built from:

```json
{
  "messages": [
    {
      "role": "system",
      "blocks": [
        {
          "text": "Extract SIGNIFICANT facts from text...",
          "source": "builtin",
          "field": "retain_extraction_mode",
          "section": "",
          "heading": "Selectivity",
          "active": true,
          "value": "concise",
          "kind": "choice",
          "choices": ["concise", "verbose", "verbatim", "chunks", "custom"],
          "editable": true
        },
        {
          "text": "",
          "source": "config",
          "field": "retain_custom_instructions",
          "section": "",
          "heading": "",
          "active": false,
          "kind": "text",
          "editable": true
        }
      ]
    },
    { "role": "user", "blocks": [] }
  ],
  "response_schema": { "type": "object", "properties": { "facts": {} } }
}
```

- **`messages`** — the request, system first, exactly as the model receives it.
- **`blocks`** — the **active** blocks of a message concatenate back to its text exactly: nothing dropped, duplicated or reordered. `source` says what produced each: `config` for a setting you can change, `builtin` for Hindsight's own wording. The runtime data an operation is given is not a block — it is a hole in the text, marked inline with `«…»`.
- **Inactive blocks** (`active: false`) have no text. They mark a setting that is switched off, at the point where it *would* land — so a mission you have not written yet is still visible where it would go.
- **Identifying a block** — whichever of these applies, in order: `field` is the config field behind it; `section` is a slug for a part no single field owns (`bank_identity`, `disposition`, `directives`); `heading` is the section heading the prompt text itself carries there. All three are machine values. The response carries **no display copy** — what a block is called, and what turning a switched-off one on would do, is for the client to say in the language it is running in.
- **`editable`** — false for server-level fields such as `llm_output_language`, which shape the prompt but cannot be overridden per bank.

**Both messages are returned on purpose.** For `retain` and `consolidation` the mission travels in the *user* message, not the system prompt: keeping the system prompt identical for every bank lets one provider-side prompt cache serve them all, which is a large cost saving on high-volume ingestion. Only `reflect` puts its mission in the system prompt.

### When there is no prompt

`chunks` extraction mode stores each chunk verbatim and never calls an LLM. There is no prompt to show, so `messages` is empty and `skipped_reason` explains why:

```json
{
  "operation": "retain",
  "messages": [],
  "skipped_reason": "Chunks mode stores each chunk verbatim as its own memory and never calls an LLM, ..."
}
```

In the Control Plane, each Mission field on the bank **Configuration** tab has a **Preview prompt** button, and each block's setting can be edited there and saved to the bank.

:::tip
[Dry-run extraction](/developer/api/memories) is the paid counterpart: it spends a real LLM call to show what the same configuration actually *extracts*. It resolves its config the same way, strategy included, so the two agree. The Control Plane pairs them in one prompt tester.
:::

## Updating Configuration

Bank configuration fields (retain mission, extraction mode, observations mission, etc.) are managed via a **separate config API**, not the `create_bank` call. This lets you change operational settings independently from the bank's identity and disposition.

### Setting Configuration Overrides

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={memoryBanksPy} section="update-bank-config" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={memoryBanksMjs} section="update-bank-config" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={memoryBanksSh} section="update-bank-config" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={memoryBanksGo} section="update-bank-config" language="go" />
</TabItem>
</Tabs>

You can update any subset of fields — only the keys you provide are changed.

### Reading the Current Configuration

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={memoryBanksPy} section="get-bank-config" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={memoryBanksMjs} section="get-bank-config" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={memoryBanksSh} section="get-bank-config" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={memoryBanksGo} section="get-bank-config" language="go" />
</TabItem>
</Tabs>

The response distinguishes:
- **`config`** — the fully resolved configuration (server defaults merged with bank overrides)
- **`overrides`** — only the fields explicitly overridden for this bank

### Resetting to Defaults

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={memoryBanksPy} section="reset-bank-config" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={memoryBanksMjs} section="reset-bank-config" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={memoryBanksSh} section="reset-bank-config" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={memoryBanksGo} section="reset-bank-config" language="go" />
</TabItem>
</Tabs>

This removes all bank-level overrides. The bank reverts to server-wide defaults (set via environment variables).

You can also update configuration directly from the [Control Plane UI](/) — navigate to a bank and open the **Configuration** tab.

---

## Directives

Directives are hard rules that the agent must follow during [reflect](./reflect) operations. Unlike disposition traits which influence *how* the agent reasons, directives are explicit instructions that are enforced whenever they are in scope (see [Directive Scope and Tags](#directive-scope-and-tags)).

:::info
Directives only affect the `reflect` operation. They are injected into prompts and the agent is required to comply with them in all responses.
:::

### When to Use Directives

Use directives for rules that must never be violated:

- **Language/style constraints**: "Always respond in formal English"
- **Privacy rules**: "Never share personal data with third parties"
- **Domain constraints**: "Prefer conservative investment recommendations"
- **Behavioral guardrails**: "Always cite sources when making claims"

### Directive Scope and Tags

Directives can carry `tags`, and those tags scope **when** a directive is applied during `reflect` — mirroring how tags scope memories:

- **Untagged directives always apply**, on every `reflect`.
- **Tagged directives apply only when the `reflect` request includes matching tags** (using the request's `tags_match` mode). A `reflect` call with no tags applies only the untagged directives.

To apply **every** active directive regardless of tags, set `apply_all_directives: true` on the `reflect` request. This ignores tag scope for directives (untagged and tagged alike are enforced) and is useful when an operator keeps tagged directives for organization but wants all of them enforced on an untagged reflection.

### Creating Directives

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={directivesPy} section="create-directive" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={directivesMjs} section="create-directive" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={directivesSh} section="create-directive" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={directivesGo} section="create-directive" language="go" />
</TabItem>
</Tabs>

### Listing Directives

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={directivesPy} section="list-directives" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={directivesMjs} section="list-directives" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={directivesSh} section="list-directives" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={directivesGo} section="list-directives" language="go" />
</TabItem>
</Tabs>

### Updating Directives

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={directivesPy} section="update-directive" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={directivesMjs} section="update-directive" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={directivesSh} section="update-directive" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={directivesGo} section="update-directive" language="go" />
</TabItem>
</Tabs>

### Deleting Directives

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={directivesPy} section="delete-directive" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={directivesMjs} section="delete-directive" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={directivesSh} section="delete-directive" language="bash" />
</TabItem>
<TabItem value="go" label="Go">
<CodeSnippet code={directivesGo} section="delete-directive" language="go" />
</TabItem>
</Tabs>

### Directives vs Disposition

| Aspect | Directives | Disposition |
|--------|------------|-------------|
| **Nature** | Hard rules, must be followed | Soft influence on reasoning style |
| **Enforcement** | Strict — responses are rejected if violated | Flexible — shapes interpretation |
| **Use case** | Compliance, guardrails, constraints | Personality, character, tone |
| **Example** | "Never recommend specific stocks" | High skepticism: questions claims |

---

## Document export & import

Move documents — and the facts already extracted from them — between banks **without re-running the LLM**. Useful for testing a different embedding model, or copying data between banks/instances without paying for re-extraction. The archive carries documents, raw chunks, and extracted facts (entities by canonical name, causal links) — but **no embeddings or database ids**. On import, facts are re-embedded with the *target* bank's model and entities/links are recomputed against it, so imported documents are integrated with whatever already exists there.

### Export documents

`POST /v1/default/banks/{bank_id}/document-transfer/export` — runs as a **background operation** (a whole-bank export loads every unit and compresses a large archive, which on a big bank could exhaust memory and pin a connection). It returns `202` with an `operation_id`; poll the bank's operations endpoint, then download the archive from the `download_url` in `result_metadata`.

```bash
# 1. Submit the export (whole bank; add ?document_id=… to scope it)
curl -X POST -H "Authorization: Bearer $API_KEY" \
  "$HINDSIGHT_URL/v1/default/banks/my-bank/document-transfer/export"
# -> {"operation_id": "…", "status": "pending"}

# 2. Poll until completed
curl -H "Authorization: Bearer $API_KEY" \
  "$HINDSIGHT_URL/v1/default/banks/my-bank/operations/$OPERATION_ID"
# -> {"status":"completed","result_metadata":{
#      "download_url":"/v1/default/files/download/banks/my-bank/exports/…/transfer.zip",
#      "storage_key":"banks/my-bank/exports/…/transfer.zip","byte_size":12345,"filename":"my-bank-documents.zip"}}

# 3. Download the archive
curl -H "Authorization: Bearer $API_KEY" \
  "$HINDSIGHT_URL$DOWNLOAD_URL" -o my-bank.zip
```

| Query param | Description |
|-------------|-------------|
| `document_id` | Repeatable. Export only these documents; omit for the whole bank. |
| `include_observations` | Also export consolidated observations (default `false`). Only valid for a **whole-bank** export — combining it with `document_id` returns `400`. |

:::note The synchronous `GET …/document-transfer` was removed
It loaded the entire bank into memory and held a database connection for the full request, which could take down the shared API on large banks. It now returns `410` pointing here. Use the async flow above. The download route (`GET /v1/default/files/download/{key}`) authorizes the caller against the bank the archive belongs to.
:::

The archive lives as long as its export **operation record** — indefinitely by default, or until the operation is pruned when `HINDSIGHT_API_OPERATION_RETENTION_DAYS` is set (the archive is deleted in step with the row). Deleting the operation removes the archive immediately.

### Import documents

`POST /v1/default/banks/{bank_id}/document-transfer` — multipart upload (`file` = the ZIP). Runs as a **background operation** (re-embedding + entity resolution can take a while), so it returns `202` with an `operation_id`; poll the bank's operations endpoint for status and the result counts in `result_metadata`.

```bash
curl -H "Authorization: Bearer $API_KEY" -F "file=@my-bank.zip" \
  "$HINDSIGHT_URL/v1/default/banks/other-bank/document-transfer?on_conflict=replace"
# -> {"operation_id": "…", "status": "pending"}

curl -H "Authorization: Bearer $API_KEY" \
  "$HINDSIGHT_URL/v1/default/banks/other-bank/operations/$OPERATION_ID"
# -> {"status":"completed","result_metadata":{"documents_imported":3,"facts_imported":42,"observations_imported":5,...}}
```

`on_conflict` controls what happens when a document id already exists in the target bank:

| Mode | Behavior |
|------|----------|
| `skip` (default) | Leave the existing document untouched. |
| `replace` | Delete the existing document's data and re-import. |
| `new-id` | Import a copy under a freshly generated id. |

### Observations

Consolidated observations are excluded by default — the target bank regenerates them from the imported facts during consolidation. Pass `include_observations=true` to carry them instead: they're restored with no LLM, their source references remapped to the imported facts (which are marked consolidated so the target won't re-consolidate them).

Because an observation can be derived from facts spanning several documents, `include_observations` is only supported on a **whole-bank export** (omit `document_id`); combining it with a document subset returns `400`.

:::warning Imported observations are inserted as-is — no merge
They are not merged or deduplicated against observations already in the target bank (consolidation merges related observations; import does not). Prefer importing observations into a fresh/empty bank, or omit `include_observations` and let the target consolidate the imported facts itself.
:::

### Enabling / disabling

Both endpoints are gated by server-level flags (default `true`). A disabled endpoint returns `404`, and `/version` reports the state under `features.document_export_api` / `features.document_import_api` (the control plane hides the buttons accordingly).

| Variable | Gates |
|----------|-------|
| `HINDSIGHT_API_ENABLE_DOCUMENT_EXPORT_API` | `POST …/document-transfer/export` and `GET …/files/download/{key}` |
| `HINDSIGHT_API_ENABLE_DOCUMENT_IMPORT_API` | `POST …/document-transfer` |

## Migrating a bank to a new instance

To move a bank to an instance configured with a different **embedding model**, **vector extension**, or **text-search backend** — which can't be changed in place on a populated bank — export the whole bank and import it into the new instance, where every embedding and index is re-derived from the stored text with **no LLM re-extraction**. This carries documents, facts, observations, bank config, mental models, directives, and webhooks (never embeddings).

Use the `hindsight-admin export-bank` / `import-bank` commands and follow the blue-green runbook in **[Admin CLI → Migrating a bank to a new instance](../admin-cli.md#migrating-a-bank-to-a-new-instance)**.