Closes 11 findings from a post-merge audit of the external embedder endpoint
implementation. The "no silent hash fallback" correctness promise was being
contradicted by two callers; the connect timeout was actually an idle timeout;
the identity probe could be bypassed in several paths; and identity drift
across runs was undetectable. All hardened.
Critical/High fixes:
- sqlite-persistence backfill no longer falls back to hash embeddings when the
endpoint is configured; re-raises to protect the HNSW index from non-comparable
vectors (was poisoning silently on every endpoint blip).
- qe-reasoning-bank.embed() propagates the throw under isUsingEndpoint() instead
of silently substituting a hash vector.
- EmbedderEndpointClient.embed() now gates on probe() — lazy probe on first use
with in-flight dedup so 20+ concurrent embeds share one probe.
- Circuit breaker recovery (window expires) clears cached identity so the next
embed re-probes — endpoint restarts often coincide with model swaps.
- True TCP-connect timeout enforced at the socket layer (was req.setTimeout,
which is an idle-after-write timeout — SYN blackholes hung ~75s instead of 5s).
- Endpoint identity persisted to memory.db kv_store (_system namespace) — cross-
run fingerprint drift now surfaces as a loud warning on next boot.
Medium fixes:
- encoding_format: 'float' pinned in request; base64 responses fail loud.
- TLS knobs (ca, cert, key, rejectUnauthorized, servername) passthrough for
self-hosted https endpoints without NODE_TLS_REJECT_UNAUTHORIZED=0.
- URL userinfo stripped + warned at parse; redacted from all error messages
and logs.
- Embedding cache key namespaced by mode (endpoint:<fingerprint>:<text> vs
inproc:<text>) — cross-mode collisions structurally impossible.
Tests:
- 33 unit tests pass (was 23): adds probe-gate, breaker re-probe, connect
timeout, DNS failure, concurrent embed dedup, base64 fail-loud, URL
redaction, encoding_format wire format.
- 6 live integration tests against real llama-server (all-MiniLM-L6-v2.Q8_0.gguf):
probe, single embed, batch, lazy probe gate, dim mismatch, 50-input batch.
Replaces the prior "9ms in-process fake" with honest numbers (cold 15ms,
warm 2ms against localhost llama.cpp).
- New tests/integration/embedder-endpoint-llamacpp.test.ts is gated on
AQE_LLAMA_EMBED_URL so it skips cleanly in CI without the binary.
ADR-097 updated with the per-caller audit table (12 writers verified
non-poisoning) and "drop-in for 6 providers" claim softened to "verified
against llama-server; OpenAI shape compat expected but unverified" per
provider.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add opt-in OpenAI-compatible /v1/embeddings client to eliminate duplicate
in-process model loads in ruflo/ruvector co-deployments and the per-hook
cold-load cost in fresh agentic-qe CLI processes.
When EmbeddingConfig.endpoint (or AQE_EMBEDDER_ENDPOINT) is set, the
@huggingface/transformers import is bypassed entirely and feature-extraction
routes through the remote service. Endpoint unset = today's behavior.
Wire format is OpenAI's /v1/embeddings so any existing embedder server (TEI,
vLLM, llama.cpp, Ollama, LocalAI, LM Studio, OpenAI) works as a drop-in,
not just bespoke ruvector servers. Transport is HTTP or HTTP-over-Unix-socket.
Errors hard-fail with a circuit breaker (3 failures / 60s window) — no
silent hash-embedding fallback, which would poison the HNSW index by mixing
non-comparable vectors. A startup probe asserts dim === 384 and computes a
stable identity fingerprint that catches model/quantization drift at the
boundary.
- src/learning/embedder-endpoint-client.ts: OpenAI client with keep-alive
agent, 5s/30s timeouts, Bearer auth from env, circuit breaker, probe
- src/learning/real-embeddings.ts: endpoint branch placed strictly before
the dynamic transformers import so the cold-load fix holds at runtime;
new isUsingEndpoint() / getEndpointIdentity() observability helpers
- 23 new unit tests (real http.createServer fixtures, no http mocking)
- End-to-end smoke: 9ms for import + init + probe + embed, transformers
never appears in the loaded-module list
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dragan Spiridonov