mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
agentics-teams-session
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dragan Spiridonov
|
da638f9079 |
feat(kernel): migrate NativeHnswBackend from @ruvector/router to hnswlib-node (#399)
@ruvector/router 0.1.28 was found to have four serious bugs that
together broke production code-intelligence semantic search and
forced the v3.9.5 hotfix to disable native HNSW entirely:
1. HNSW search returned essentially random results — recall@10 was
0% on the project's FNV fixture and 10% on textbook unit-Gaussian
random vectors. The graph walk could not even find self-vectors
when querying with a stored vector. Reproduced deterministically
on linux-arm64; matches the linux-x64 GitHub Actions failures
reported in #399.
2. The VectorDb constructor unconditionally wrote a `vectors.db`
redb file to the current working directory, polluting users'
project roots with multi-MB files outside .agentic-qe/, in a
non-SQLite format that no AQE migration or backup tooling knew
about. Direct violation of the unified memory architecture.
3. The redb file held a process-wide exclusive lock — only ONE
VectorDb instance could exist per process. Subsequent constructors
threw "Database error: Database already open. Cannot acquire lock."
4. NAPI dispose did not synchronously release the redb lock (NAPI GC
is not synchronous). The lock outlived dispose() and caused the
v3.9.5 futex deadlock when the indexer tried to recreate after
reset.
NativeHnswBackend is rewritten to wrap hnswlib-node@^3.0.0 — the
canonical Node.js binding for Yury Malkov's reference C++ Hnswlib
implementation, used by Pinecone, Weaviate, Qdrant, LangChain, and
ChromaDB. Empirical verification on the same fixture
(`scripts/diagnose-issue-399*.mjs`):
- hnswlib-node: recall@10 = 100%, top-1 = id 42 ✓
- @ruvector/router: recall@10 = 10%, top-1 = id 999 (random)
Insert is also faster (~324ms vs ~430ms for 1k vectors) and search
is comparable (0.35ms vs 0.21ms with correct vs wrong results).
No new dependency was added — `hnswlib-node` was already a direct
dependency in `package.json` from the legacy code path in
`src/integrations/embeddings/index/HNSWIndex.ts`.
Implementation notes:
- Drops the local `vectorStore` mirror that the previous backend
needed to re-score wrong native results — hnswlib-node returns
correct distances directly.
- Drops the `bruteForceSearch` private fallback — there is no
fallback path because hnswlib-node always returns correct results.
- `clear()` now recreates the underlying HierarchicalNSW from
scratch instead of just markDelete-ing every label. The
tombstone-only approach leaks state across long test runs that
share the singleton HnswAdapter registry, producing duplicate
labels from `searchKnn` after many cycles.
- `search()` overshoots k by 2x and dedups by id as a defensive
measure against any future hnswlib-node duplicate-label edge
case, then truncates back to k.
- `add()` doubles maxElements via `resizeIndex()` when the live id
count fills the current capacity, so callers don't need to know
the final index size at construction time.
- `useNativeHNSW` default flipped back to `true`. Users with large
codebases get sublinear HNSW search by default again.
- All four `@ruvector/router` failure modes have explicit regression
guards in the unit and integration test suites.
The two #399-skipped tests in `phase1-integration.test.ts`
(line 222 and 1185) are un-skipped and now pass green against the
new backend. The three flag-default tests that asserted
`useNativeHNSW: false` (broken by v3.9.5's hotfix) are updated to
assert `true` again.
Stale comments referencing `@ruvector/router`'s lock behavior are
updated in `src/kernel/unified-memory.ts`, `src/init/phases/06-code-intelligence.ts`,
`src/init/phases/10-workers.ts`, and `src/cli/index.ts`. The CLI
status output (`aqe ruvector status`) and the `useNativeHNSW` flag
description are updated to mention `hnswlib-node` instead of
`@ruvector/router`.
Refs #399, ADR-090.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
3a1d9636b3 |
fix(kernel): disable native HNSW by default — deadlocks on certain inputs (v3.9.5)
ROOT CAUSE FOUND for the ruview / ruview_live.py hang.
Investigation
-------------
v3.9.4's per-file log gave the user the exact stalling file:
examples/ruview_live.py from the public RuView project. I downloaded
the file (28 KB, 776 lines, 95 entities — normal-shaped Python),
reproduced the hang locally against v3.9.4, then dumped kernel
state of the hung process via /proc/<pid>/wchan and strace.
Findings:
Main thread: futex(0x3c6ded50, FUTEX_WAIT_PRIVATE, 1, NULL)
(NULL timeout = wait forever)
17 total threads:
- MainThread (blocked in futex)
- V8Worker × 4 (blocked in futex)
- DelayedTaskScheduler (epoll_wait — normal libuv)
- SignalInspector (blocked in futex)
- tokio-runtime-w × 6 (blocked in futex)
- libuv-worker × 4 (blocked in futex)
Native modules loaded:
- ruvector-router.linux-arm64-gnu.node
- ruvector-gnn.linux-arm64-gnu.node
- better_sqlite3.node
The deadlock is reproducible: every run against this file produces
the same futex_wait_queue state. After 7 successful VectorDb.insert()
calls (7 rows visible in the vectors table), the 8th never returns.
This is a futex deadlock inside the @ruvector/router native NAPI
module, triggered by VectorDb.insert() against certain vector content
shapes. The Rust tokio runtime threads inside the module hold a lock
the main thread is waiting for, and never release it. The setTimeout-
based watchdog v3.9.3 added cannot interrupt this — setTimeout
callbacks queue between event loop iterations and the event loop is
frozen inside a kernel futex syscall. That's a Node.js architectural
limitation, not a bug in our watchdog code.
The fix
-------
One-line change: useNativeHNSW feature flag default flipped from
true to false in src/integrations/ruvector/feature-flags.ts. All
HNSW operations now route through ProgressiveHnswBackend instead of
NativeHnswBackend. The native module is still loaded and still
available to anyone who explicitly opts in via
setRuVectorFeatureFlags({ useNativeHNSW: true }) — the default path
just no longer touches the deadlock-prone code.
Why this isn't a regression
---------------------------
ProgressiveHnswBackend was already updated in v3.9.3 commit
|
||
|
Dragan Spiridonov
|
a6202c39a3 |
chore(release): v3.9.1 (#395)
* feat: implement ADR-062 (StrongDM all 3 tiers) and ADR-087 (RuVector 14/14 capabilities)
ADR-062 StrongDM Software Factory — all 6 actions across 3 tiers:
- Tier 1: Loop detection wired to MCP protocol-server (blocks on strike 3+),
token dashboard (pre-existing)
- Tier 2: Holdout testing (10% FNV-1a selection, filtered in test-execution),
gate ratcheting (monotonic 5-pass/2%/95% cap, persisted to MemoryBackend)
- Tier 3: Progressive context (TF-IDF prediction, reduces availableFiles on spawn),
meta-learning cycle (4 insight types, stored + published as domain events)
ADR-087 RuVector Advanced Capabilities — milestones 1-5:
- EWC++ recordOutcome + backgroundConsolidate wired to 3 domain coordinators
- R1: HDC fingerprinting (10K-bit hypervectors, wired to PatternStore.store())
- R3: Delta event sourcing (sparse encoding, wired to PatternStore.store())
- R13: Cognitive routing (predictive compression, wired to mailbox.send/receive)
- R14: Hyperbolic HNSW (Poincare ball, auto-indexed in PatternStore.store())
Also updates ADR statuses: 059, 060, 061 → Implemented (were already done),
062 → Implemented, 086 → In Progress, 087 → Implemented.
223 new tests across 11 test files, all passing. Zero regressions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: implement RVF cluster (ADR-065/066/067/071) with native benchmarks
Implements the RVF persistence cluster for persistent HNSW vector storage,
agent memory branching, and HNSW unification foundation.
Key changes:
- RvfPatternStore: IPatternStore backed by @ruvector/rvf-node native HNSW
(1.2ms cold-start, 0.19ms search p50, verified with real native binary)
- AgentMemoryBranch: per-agent COW isolation via derive(), ingest-log merge,
wired into Kernel.initialize() + DefaultAgentCoordinator
- HnswLegacyBridge: adapts old IHNSWIndex to unified IHnswIndexProvider,
wired into PatternStore when useUnifiedHnsw flag enabled
- Factory routing: createPatternStore() returns RvfPatternStore when flag
enabled + native available + data directory exists
- Dream engine: accepts external RVF adapter (no duplicate file handles),
RVCOWBranchManager enhanced with recall validation check
- @ruvector/gnn bumped 0.1.19→0.1.25, 3 new feature flags added
- IPatternStore interface extracted for polymorphic pattern store usage
- 50 new tests (16 RvfPatternStore + 17 AgentMemoryBranch + 12 bridge + 5 benchmarks)
- 1188 tests pass, 0 regressions
ADR status: 066 Implemented, 067 Implemented, 065 Implemented,
069/071 In Progress (honest assessment after brutal-honesty review)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(init): resolve skills/agents paths from package root, not relative to bundle depth
esbuild code-splitting puts installer code into dist/cli/chunks/ (3 levels
deep) instead of dist/init/ (2 levels deep), so the old ../../ traversals
landed at dist/ instead of the package root — causing "Source directory not
found" errors for all global installs.
Replace brittle relative-path resolution in all 4 installers with a shared
findPackageRoot() that walks up to the nearest package.json with
name=agentic-qe. Prioritise assets/ paths (canonical npm source) over
.claude/ paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: complete RVF cluster Phases 2-3 (ADR-067/069/071/072) with devil's advocate fixes
Phase 2A (ADR-067): Agent memory branching enabled — useAgentMemoryBranching=true,
BRANCH_DISCARD witness action, recordIngest callback exposed to agents via config.
Phase 2B (ADR-069): Dream RVF COW — useRvfFork driven by isRVFPatternStoreEnabled(),
child adapter stored + temp files cleaned up on merge/discard, 13 tests.
Phase 2C (ADR-071): HNSW unification — useUnifiedHnsw=true, HNSWEmbeddingIndex routes
through HnswAdapter, UnifiedMemory vectorIndex uses UnifiedHnswIndex, HnswLegacyBridge
delegates to adapter string-ID APIs (no duplicate ID maps), addEmbeddingsBatch routed
correctly, saveIndex/loadIndex safe under unified mode.
Phase 3 (ADR-072): RVF primary migration scaffolding — RvfMigrationAdapter (5-stage
routing with write latency tracking), RvfConsistencyValidator (rolling divergence +
compaction), RvfStageGate (go/no-go with witness chain), rvfMigrationStage=1 default,
wired into shared-rvf-dual-writer at stage >= 2, 32 tests.
Cross-cutting fixes from devil's advocate review:
- Shared RVF adapter singleton eliminates dual file handles (kernel + dream engine)
- RvfPatternStore reports honest nativeAvailable=false + rvfInitError on failure
- ESM-safe dynamic import() replaces require() in dream-engine and branch-manager
- quality_assess defaults to src/ instead of cwd to prevent full-project scan hang
- discoverSourceFiles skips .agentic-qe/.claude/.cache dirs, adds maxFiles=5000 cap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: complete RVF cluster plan — Phase 2C decommission, Phase 3 integration, fix 7 pre-existing test failures
Phase 2C (ADR-071) — HNSW decommission:
- Remove RuvectorFlatIndex fallback from unified-memory.ts; always use UnifiedHnswIndex
- Hardcode isUnifiedHnswActive()=true in HNSWEmbeddingIndex (hnswlib-node legacy path is dead code)
- Create HnswShadowValidator for brute-force vs HnswAdapter divergence validation
- Benchmark alias (rvf-pattern-store.bench.ts) for plan compliance
Phase 3 (ADR-072) — RVF Primary Migration integration:
- RvfMigrationCoordinator: wires adapter + validator + gate with real SQLite/RVF handles
- MCP tools: migration_status, migration_check, migration_promote (persistence category)
- Kernel boot wiring: coordinator initializes when rvfMigrationStage >= 2
- RvfMigrationAdapter.setStage(): propagate promotions to routing logic
- Promote default rvfMigrationStage from 1 (hybrid) to 2 (dual-write-sqlite-primary)
- Integration test: 13 tests covering full lifecycle (stage 1→2→3→4), write/read routing,
consistency checks, gate enforcement, fallback, metrics
Test fixes (7 pre-existing failures → 1 remaining):
- code-intelligence-metrics-wiring: spy console.info not console.log (logger routes to console.info)
- mcp-integration: disable ADR-062 loop detection (repeated identical calls are by design)
- dream-cycle-tool (2 tests): RvfPatternStore empty-query branch returns all patterns from SQLite
- coherence-integration (4 tests): RvfPatternStore vector search fallback computes brute-force
cosine over SQLite embeddings when RVF adapter unavailable; getStats() falls back to SQLite count;
auto-attach SQLitePatternStore in initialize() when not explicitly wired
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(release): bump version to v3.9.1
- Version bump in package.json and skills-manifest.json
- Fix "Dynamic require of X is not supported" in ESM bundles by adding
createRequire banner to both CLI and MCP build scripts — every chunk
now has a working require() for Node.js builtins
- Add CHANGELOG.md entry and release notes for v3.9.1
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): regenerate package-lock.json for npm ci compatibility
The lock file was out of sync with package.json — new transitive
dependencies added on main were missing. All 15 CI checks failed
at `npm ci` with "Missing from lock file" errors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): pin Node to 24.13.0 and restore working lock file
Node 24.14.1 ships npm 11.11.0 which enforces stricter lock file
validation than 11.6.x (what 24.13.0 ships). The existing lock file
passes under 11.6.x but fails under 11.11.0 with "Missing from lock
file" for transitive dependencies of private-registry packages.
Pin all 5 workflow files to node-version: '24.13.0' and restore the
lock file from commit
|
||
|
Dragan Spiridonov
|
fdd32a42ce |
fix(tests): add vi.useFakeTimers() to 11 files using real setTimeout (#374 P2)
11 test files used real setTimeout without vi.useFakeTimers(), adding wall-clock time to sequential CI execution. Converted to fake timers with vi.advanceTimersByTime() for deterministic, instant execution. Files fixed: - error-paths/ (5 files): circuit breaker, workflow timeout, consensus timeout, task timeout, concurrent init — ~33 setTimeout patterns - kernel/ (3 files): TTL expiration tests that waited 1100ms real time - mcp/metrics (1 file): 7 timing-measurement tests - cli/progress (1 file): 3 ETA estimator tests - memory/crdt (1 file): convergence time tracking Estimated CI time savings: ~10s per sequential run. 11 files intentionally skipped (real I/O, production timer logic): websocket-transport, mcp-server, adr-040, swarm-skill-validator, plan-executor, parallel-eval-runner, failover, behavior-tree, etc. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
fc1e86ad2d |
fix(security): resolve P0/P1 issues from v3.8.3 QE swarm analysis
P0 fixes: - Fix command injection (CWE-78) in test-verifier.ts: replace exec() with execFile() + ALLOWED_TEST_COMMANDS allowlist - Unify SQL allowlists: add 5 missing tables to sql-safety.ts, replace local duplicate in unified-memory.ts with canonical validateTableName() - Add validateTableName() to all 7 SQL interpolation sites in ruvector/brain-shared.ts P1 fixes: - Remove 70 process.exit() calls from learning.ts (45) and hooks.ts (25), replacing with return/throw for proper cleanup handler execution - Move @faker-js/faker from dependencies to devDependencies (~8MB savings) - Enable bundle minification: CLI 9.8→6.9MB (-30%), MCP 12→7.2MB (-40%) - Fix no-op verification scripts to perform real checks with non-zero exit codes on failure QE reports: - Add 12 comprehensive QE analysis reports for v3.8.3 covering code quality, security, performance, tests, SFDIPOT, dependencies, API contracts, architecture/DDD, accessibility, and brutal honesty audit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
c911aa88a8 |
feat: enable all 15 RuVector feature flags by default (opt-out instead of opt-in)
All new RuVector capabilities (native HNSW, neural routing, coherence gates, transfer learning, etc.) now ship enabled so users get immediate value on upgrade. Users can disable individual features via CLI or setRuVectorFeatureFlags() for opt-out. Update 11 test files to match new defaults. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
6b249c034f |
feat: RuVector integration phases 1-4 — native HNSW, neural routing, safety gates, differentiation
Implements 23 tasks across 4 phases per docs/implementation/ruvector-integration-plan.md: Phase 1 — Foundation (make what we have faster): - Native HNSW backend via @ruvector/router (150x search speedup, ADR-081) - Metadata filtering layer with composable AND/OR/NOT/FIELD expressions - Temporal tensor compression (Int8Array quantization, 4x memory reduction) - Deterministic golden-ratio dithering for cross-platform reproducibility - 15 feature flags (all default false) with CLI profiles Phase 2 — Intelligence (make agents smarter): - Neural TinyDancer router with REINFORCE policy gradient (ADR-082) - SONA three-loop enhancement (MicroLoRA 0.18μs/adapt via WASM) - Cross-domain transfer learning with Thompson Sampling (ADR-084) - Regret tracking with log-log regression growth classification Phase 3 — Safety (make agents trustworthy): - Sheaf-gated test validation via prime-radiant CohomologyEngine - Coherence-gated agent actions (3-filter pipeline, ADR-083) - Witness chain audit trail with SHA-256 hash linking - HNSW health monitoring with spectral analysis Phase 4 — Differentiation (make AQE unique): - Cognitive container export/import (RVF v2) - DAG attention scheduler for dependency-aware test ordering - CNN visual regression with spatial pooling embeddings - Behavior tree orchestration (Sequence/Selector/Parallel) - Reasoning QEC with majority-vote consensus - Browser QE dashboard scaffolding Also: remove obsolete v2-to-v3-migration test (source already removed), add ADR-081 through ADR-085, research docs, benchmarks, and integration test suites for all 4 phases. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Profa
|
416c97ab39 |
feat(learning): implement Six Hats Tier 0-2 improvements and fix test flakiness
Tier 0 (Bug Fixes):
- Fix promotion threshold inconsistency (standardized on 3)
- Add embedding backfill script and ghost pattern health check
- Add FTS5 virtual table with hybrid vector/text search
Tier 1 (Quick Wins):
- Binary reward assignment (+1/-1/0) for outcome-based learning
- At-least-one learning guarantee per session
- Temporal decay on search results (30-day half-life)
- Configurable promotion activity window
- Enable coherence gate by default
Tier 2 (Medium Investments):
- Pre-compaction flush hook to prevent knowledge loss
- OPD remediation hints ("bad because X, fix by Y")
- Per-agent MCP tool scoping for security isolation
- Daily Markdown log tier for audit trail
- Proof-of-Quality CLI command with SHA-256 attestation
- Devil's advocate scoring fixes (FTS normalization, cap, decay, freshness)
Test Fixes:
- Fix queen governance flakiness in domain-handlers tests
- Fix cross-suite singleton contamination in handler-factory and e2e tests
- Fix mcp-server domain tool timeout and invocation failures
610 test files pass, 0 failures, 18,589 tests green.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Profa
|
ec2c243172 |
fix(embeddings): complete 384-dim standardization across tests and providers
- Update default embed model from nomic-embed-text to all-MiniLM-L6-v2 in Ollama LLM provider - Fix 15 test files with hardcoded 768-dim references - Update mock model names in Ollama health checks - Update comments referencing old model name Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Profa
|
2253d05618 |
feat: flatten project structure — archive v2, promote v3 to root
Remove the nested v3/ subdirectory by moving all active source code, tests, configs, and assets directly to the project root. Archive v2/ (dead code, 847MB) to v2-archive.tar.gz. Key changes: - Move v3/src/ → src/, v3/tests/ → tests/, v3/packages/ → packages/ - Move v3/tsconfig.json, v3/vitest.config.ts to root - Merge v3/package.json into root package.json (ESM, scripts, deps, exports) - Merge v3/scripts/ into scripts/, v3/implementation/ into docs/implementation/ - Update all CI workflows to remove "cd v3 &&" and v3/ path prefixes - Fix all init installer path resolution (3-level-up → 2-level-up) - Fix hooks to use "node ./dist/cli/bundle.js" instead of bare "aqe" - Update settings.json ADR/DDD directories to docs/implementation/ - Fix helper scripts (statusline, adr-compliance, ddd-tracker, guidance-hooks) - Update 58 agent definitions: aqe/v3/ → aqe/ memory namespaces - Update 15 guidance shards: v3/src/ → src/ path references - Fix sync interfaces: remove v3/ and v2/ path references - Fix test path calculations (process.cwd() + '..' no longer needed) - Fix flaky perf test threshold (500ms → 1000ms for CI environments) - Fix complexity-analyzer test to match ADR-051 keyword-based eligibility Build passes, 17,901 tests pass, CLI and MCP bundles verified. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |