* fix(compose): use HINDSIGHT_API_LLM_API_KEY env var
* docs(compose): point run examples at HINDSIGHT_API_LLM_API_KEY
The compose files no longer read OPENAI_API_KEY, so every doc telling
users to export it was left describing a variable nothing reads.
- custom-models/README.md + compose header: export the correct var
- timescale/README.md quick start, prereq and env-var table
- timescale/.env.example: compose's project directory is the compose
file's own directory, so this file IS auto-loaded - naming the wrong
var here silently dropped the key on the documented happy path
---------
Co-authored-by: Ish Fuseini <me@ishfuesini.com>
Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
The PGroonga example referenced groonga/pgroonga:latest-debian-pg17, which has no registry manifest, so the documented Compose stack could not build.\n\nPin the current PGroonga 4.0.8 PostgreSQL 17 image and install pgvector 0.8.6 from the PGDG repository already configured by that base image. This removes the source clone and build toolchain while keeping the example on PostgreSQL 17 for parity with neighboring recipes.\n\nContext:\n- Fixes #3311.\n- Verified on arm64 with a disposable PostgreSQL instance.\n- CREATE EXTENSION vector and pgroonga both succeed.\n- Mixed Korean/English PGroonga search and pgvector distance queries pass.\n- PostgreSQL 18 deployment work remains a separate operational concern.
* feat(bm25): make native language configurable + opt-in pgroonga backend
Adds two new env-level config knobs and a new opt-in BM25 backend so users
can serve non-English banks (especially CJK) out of the box.
- HINDSIGHT_API_BM25_LANGUAGE drives the PostgreSQL text search dictionary
used by the native tsvector backend (default: english). Validated as a
PG identifier so it can be safely embedded in to_tsvector('<lang>', ...).
- HINDSIGHT_API_RETAIN_OUTPUT_LANGUAGE forces the fact extractor to emit
facts in the specified language regardless of source content's language.
Independent from bm25_language so users can mix indexing/extraction
languages deliberately.
- New 'pgroonga' option for HINDSIGHT_API_TEXT_SEARCH_EXTENSION. Uses
TokenBigram + NormalizerNFKC150 — single polyglot index handles English,
CJK, etc. simultaneously. Ships with a docker-compose recipe.
To support a per-deployment language, the GENERATED ALWAYS expression on
memory_units.search_vector (and reflections.search_vector) is dropped via
new alembic migration p4q5r6s7t8u9. The application now populates these
columns at INSERT time using the configured bm25_language.
* docs(bm25): rename env var to scope it to native; move multilingual content to dedicated page
- Rename HINDSIGHT_API_BM25_LANGUAGE → HINDSIGHT_API_TEXT_SEARCH_EXTENSION_NATIVE_LANGUAGE.
The setting only applies to the "native" backend (vchord/pg_textsearch/pgroonga
use their own tokenizers), so the env var name now reflects that scope. Field
renamed to text_search_extension_native_language.
- Trim configuration.md back to a brief env-var table + link. The expanded
multilingual / CJK / pgroonga content moves to the dedicated multilingual.md
page, alongside the existing LLM / embedding / reranker multilingual guidance.
* feat(llm-output-language): rename and broaden to cover retain + consolidation + reflect
Renames HINDSIGHT_API_RETAIN_OUTPUT_LANGUAGE → HINDSIGHT_API_LLM_OUTPUT_LANGUAGE
(field llm_output_language) and applies the same "respond exclusively in {lang}"
directive across every LLM-generated artifact:
- retain (fact extraction) — already wired, just renamed.
- consolidation (observations / mental models) — appended to the batch
consolidation prompt via a new llm_output_language parameter.
- reflect (response synthesis) — appended to the final-system prompt via a
new parameter threaded through run_reflect_agent and memory_engine.
The shared directive lives in engine/prompt_utils.output_language_directive
so all three pipelines build the same instruction from a single source.
* docs(multilingual): drop the backfill-after-language-change section
Ish Fuseini