Files
Dragan Spiridonov a9f7082f15 fix(embeddings): harden ADR-097 endpoint client against devil's-advocate audit
Closes 11 findings from a post-merge audit of the external embedder endpoint
implementation. The "no silent hash fallback" correctness promise was being
contradicted by two callers; the connect timeout was actually an idle timeout;
the identity probe could be bypassed in several paths; and identity drift
across runs was undetectable. All hardened.

Critical/High fixes:
- sqlite-persistence backfill no longer falls back to hash embeddings when the
  endpoint is configured; re-raises to protect the HNSW index from non-comparable
  vectors (was poisoning silently on every endpoint blip).
- qe-reasoning-bank.embed() propagates the throw under isUsingEndpoint() instead
  of silently substituting a hash vector.
- EmbedderEndpointClient.embed() now gates on probe() — lazy probe on first use
  with in-flight dedup so 20+ concurrent embeds share one probe.
- Circuit breaker recovery (window expires) clears cached identity so the next
  embed re-probes — endpoint restarts often coincide with model swaps.
- True TCP-connect timeout enforced at the socket layer (was req.setTimeout,
  which is an idle-after-write timeout — SYN blackholes hung ~75s instead of 5s).
- Endpoint identity persisted to memory.db kv_store (_system namespace) — cross-
  run fingerprint drift now surfaces as a loud warning on next boot.

Medium fixes:
- encoding_format: 'float' pinned in request; base64 responses fail loud.
- TLS knobs (ca, cert, key, rejectUnauthorized, servername) passthrough for
  self-hosted https endpoints without NODE_TLS_REJECT_UNAUTHORIZED=0.
- URL userinfo stripped + warned at parse; redacted from all error messages
  and logs.
- Embedding cache key namespaced by mode (endpoint:<fingerprint>:<text> vs
  inproc:<text>) — cross-mode collisions structurally impossible.

Tests:
- 33 unit tests pass (was 23): adds probe-gate, breaker re-probe, connect
  timeout, DNS failure, concurrent embed dedup, base64 fail-loud, URL
  redaction, encoding_format wire format.
- 6 live integration tests against real llama-server (all-MiniLM-L6-v2.Q8_0.gguf):
  probe, single embed, batch, lazy probe gate, dim mismatch, 50-input batch.
  Replaces the prior "9ms in-process fake" with honest numbers (cold 15ms,
  warm 2ms against localhost llama.cpp).
- New tests/integration/embedder-endpoint-llamacpp.test.ts is gated on
  AQE_LLAMA_EMBED_URL so it skips cleanly in CI without the binary.

ADR-097 updated with the per-caller audit table (12 writers verified
non-poisoning) and "drop-in for 6 providers" claim softened to "verified
against llama-server; OpenAI shape compat expected but unverified" per
provider.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 07:52:23 +00:00
..
2026-04-05 18:51:49 +02:00