mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Nicolò Boschi
828a6e55d3
* refactor(retain): format pgvector literals without orjson orjson was a single-purpose dependency: one function (`embedding_to_pgvector`), on the retain write path only, rendering an embedding as the '[0.1,...]' literal asyncpg binds to `vector`. It played no part in HTTP serialization — there is no ORJSONResponse in the API, and recall builds its query literal with plain str(). Measurement decided it. On `perf-test --suite retain --scale small` (200 items, 384d, mock LLM + pg0) the orjson calls totalled 2.9ms of a ~1.9s retain, and paired throughput runs could not tell the two implementations apart from noise (orjson 75-124 items/s, replacement 81-115). What replaces it is one C-level %-format call per vector against a per-dimension cached "[%.9g,...]" template: ~1.8x faster than the repr() generator that was already the non-finite fallback, ~5x slower than orjson. No pure-Python formatter closes that gap — orjson formats floats with SIMD Ryu in Rust. Beating it means not emitting text at all, which the benchmark now measures as `binary_pgvector` (~2 orders of magnitude faster, 2.8x fewer wire bytes) but which needs an asyncpg codec for the `vector` type and would still leave Oracle on the text path. Values are narrowed to float32 before formatting, exactly once. Applying %.9g straight to a float64 rounds twice — to nine digits, then to float32 — and the two roundings pick different neighbouring float32s for ~0.8% of values. Narrowing first makes the single remaining rounding the one PostgreSQL would have done, so stored bytes are unchanged. The new regression test samples 4,096 elements, where that rate misses by ~31. The literal is now fixed-width rather than shortest-form (0.100000001 where orjson wrote 0.1) — same float32, ~12% more SQL text. The orjson version floor stays in pyproject as a transitive-only pin (langsmith still pulls it), matching how pyjwt is handled. * build: drop orjson from the runtime dependency closure The previous commit removed the last import; this removes the package from what production resolves. orjson's only other path into the environment is langchain-text-splitters -> langchain-core -> langsmith -> orjson, and langchain has been test-only since #3756 — retain's chunker is its own streaming RecursiveCharacterTextSplitter, and langchain survives purely as the reference implementation test_chunking_matches_legacy.py diffs against (with a guard test asserting the reference is still really langchain). So the `orjson>=3.11.6` floor follows langsmith into the test extra and the dev group, exactly as the langchain-core and langsmith floors already did when langchain left the runtime. Deleting the floor outright would have dropped the unbounded-recursion DoS bound for the version langsmith pulls. uv export --no-dev -> orjson absent uv export -> orjson 3.11.7 (via langsmith, test only) uv.lock is unchanged. * test: vendor the chunking oracle and drop langchain, langsmith and orjson orjson's last path into the environment was the test extra: langchain-text-splitters -> langchain-core -> langsmith -> orjson. langchain was there for one reason — the differential chunking tests need the splitter retain used before #3756 as their oracle, and that oracle has to be genuinely the old implementation or the tests pass vacuously. So the oracle moves in-tree. tests/chunking_reference.py transcribes langchain 1.1.2's RecursiveCharacterTextSplitter (MIT), specialised to the one configuration retain ever used: chunk_overlap=0, keep_separator default "start", length_function=len, non-regex separators. The branches that configuration cannot reach are left out rather than transcribed untested. The transcription was verified against the real splitter before the dependency was removed: 20,079 comparisons — every case and size the chunking tests use, plus 20,000 seeded fuzz documents built from the fragment shapes that break splitters (empty strings, whitespace runs, CRLF, CJK, oversized tokens) — with zero mismatches. tests/test_chunking_reference_matches_langchain.py keeps that check runnable: uv run --with langchain-text-splitters pytest \ tests/test_chunking_reference_matches_langchain.py It skips by default, since langchain is deliberately absent. test_the_legacy_reference_really_is_the_old_implementation loses its live langchain call, so its anchor becomes a table of five expected splits — one per separator tier, including the per-character fallback — produced BY langchain and pasted in. A reference that quietly became the new implementation has to reproduce all five to slip through. uv export --no-dev -> no orjson, no langchain uv export -> no orjson, no langchain uv.lock loses 392 lines.