mirror of
https://github.com/ruvnet/ruflo.git
synced 2026-09-14 14:01:28 +08:00
d759af6d6b
* feat(agenticow): complete verb surface (ingest/query/diff/lineage/status) + nativeAnn + 0.2.4
Step 1 of the agenticow integration: the lifecycle verbs (branch/
checkpoint/rollback/promote) existed but the read/write verbs that make
a branch usable did not — you could create a COW branch via MCP but
couldn't populate or read it. Adds:
- agenticow_ingest — write {id?,vector,text?} records into a branch/base
- agenticow_query — k-NN across the full COW lineage (parent ∪ edits)
- agenticow_diff / _lineage / _status — introspection
- nativeAnn option on agenticow_branch (Rust dual-graph ANN fast path)
- targeted rollback via optional checkpointId
Floor bumped agenticow ~0.2.3 → ~0.2.4 (the upstream fix — text payloads
now survive the save/load round-trip our wrapper does on every call;
ruvnet/agenticow#3, published 0.2.4). 9 tools total; tests 10/10.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(swarm): per-agent COW memory branches via agenticow (agenticow step 2)
Wire the agenticow COW-memory primitive into the swarm agent lifecycle so a
swarm agent can own an isolated ~162-byte Copy-On-Write branch of a shared
.rvf base instead of a full copy — the structural fix for the v3.14.4
worktree-bloat (3.3 GB from full-copy per-agent snapshots).
SwarmMemoryBranches service (src/services/swarm-memory-branches.ts):
- branchForAgent(base, agentId): fork a 162-byte COW branch (nativeAnn),
idempotent, persisted to .claude-flow/swarm/cow-branches.json
- promoteAgent(agentId): merge the agent's edits back into base, then delete
the branch (call on success)
- discardAgent(agentId): drop the branch, pure-fs, works even degraded
(call on failure)
- Non-fatal + CLAUDE_FLOW_NO_COW_MEMORY kill switch; agenticow lazy-loaded so
it stays off the CLI startup path (--help unchanged at ~0.09s).
Seam + wiring (src/mcp-tools/agent-tools.ts):
agent_spawn stores agents as pure JSON metadata today — it never forks/copies
an .rvf, so there is no full-copy to replace inline. Wired in opt-in: pass
`memoryBase` to agent_spawn to fork a per-agent branch (recorded on the
AgentRecord); agent_terminate promotes (promoteMemory:true) or discards it.
Default behavior is unchanged.
Shared loader (src/mcp-tools/agenticow-loader.ts): extracted loadAgenticow /
resolveMemoryPath / manifestFor / validateLabel / openWithLineage from
agenticow-tools.ts so the MCP verbs and the service share one optional-dep
dance (behavior-preserving; existing agenticow-tools tests unchanged).
Tests (__tests__/swarm-memory-branches.test.ts): kill-switch + degraded
no-op contract (always-on) and a real branch -> ingest -> promote/discard
lifecycle against a temp .rvf (skipIf agenticow absent), following the
agenticow-tools.test.ts conventions.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(agenticow): speculative branch-and-promote for A/B memory exploration (step 4)
Adds a self-contained SpeculativeExploration module + `agenticow_speculate`
MCP tool that composes on top of the existing agenticow COW verbs (fork /
promote). Fan out N candidate approaches, each on its own 162-byte COW branch
of a shared base .rvf; explore/score each independently; promote the winner's
branch into base and discard the losers by deleting their branch files. The
memory-state analogue of the worktree-per-agent pattern.
- src/agenticow/speculative-exploration.ts — generic `explore(base, {label,fn}[],
score)` core + `exploreFromPath()` lifecycle wrapper. Returns
{winner, scores, promoted, ...}.
- src/mcp-tools/_agenticow.ts — shared COW helpers (loadAgenticow / openWithLineage
/ manifestFor / resolveMemoryPath / validateLabel). ADDED alongside
agenticow-tools.ts (kept byte-identical) to minimize merge conflict with a
sibling branch.
- src/mcp-tools/agenticow-speculate-tools.ts — `agenticow_speculate` MCP tool
(declarative candidates + probe-query / count scoring). Optional-dep graceful
degradation per ADR-150.
- Registered in mcp-client.ts + mcp-tools/index.ts.
- __tests__/agenticow-speculate-tools.test.ts — 3-candidate deterministic-scorer
test asserting winner promoted into base and loser branch files discarded
(skipIf agenticow absent), following agenticow-tools.test.ts conventions.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(autopilot): checkpoint/rollback gate for loop ticks (agenticow step 3)
Add CheckpointGate — an agenticow-backed checkpoint/rollback bracket for
autopilot loops and long-horizon workflows. Before a risky tick that mutates
.rvf memory, take an O(1) agenticow checkpoint (162 bytes); if the tick
regresses (throws, or the caller's verdict says the outcome is worse), roll
memory back to the checkpoint — O(edits-since-checkpoint), not an O(N) rebuild.
- New src/services/checkpoint-gate.ts: CheckpointGate.guard(memPath, label, fn)
plus low-level checkpoint()/rollback(). Lazy-loads agenticow (optional dep,
ADR-150) — zero startup cost, graceful degradation to unguarded pass-through
when the package is absent, the CLAUDE_FLOW_AGENTICOW_DISABLE kill-switch is
set, or no memory path is configured. Non-fatal: an agenticow failure never
masks the tick's own result.
- Opt-in wiring into the autopilot loop (commands/autopilot.ts autopilotCheck):
checkpoint the configured .rvf right before a re-engaged tick; roll back on
the loop's regression signal (stall auto-disable). Activates only when
CLAUDE_FLOW_AUTOPILOT_CHECKPOINT_MEM points at an .rvf the loop mutates.
The loop's memory mutation is out-of-process, so this brackets across ticks
rather than wrapping an in-process fn; guard() is available for callers that
do have an in-process fn.
- Unit test (__tests__/checkpoint-gate.test.ts): 10 cases — real temp .rvf
checkpoint→mutate→rollback restore, success-keeps, throw-rollback-rethrow,
and degraded/kill-switch pass-through. skipIf(agenticow absent), following
agenticow-tools.test.ts conventions.
Co-Authored-By: RuFlo <ruv@ruv.net>
* docs(adr): ADR-170..173 — agenticow substrate, provenance oracle, Fable harness, remote distillation
Records the architectural decisions of the agenticow + distillation
governance integration:
- ADR-170: agenticow COW memory as agent-scoped workspace substrate
(9 verbs, per-agent branches, speculative, checkpoint gate, one
shared optional-dep loader; honest 'no full-copy to replace' scope)
- ADR-171: provenance-tiered evaluation oracle (execution → fable →
proxy, each label tagged; promote-gated on real clearance; causal
failure receipts; never proxy-as-gold — ADR-169 applied to labels)
- ADR-172: cost-disciplined Fable advisor harness via claude -p
(clean-cwd + batching = $1.56→~$0.02/item; judge + GEPA reflector)
- ADR-173: remote GPU distillation via weight-eft over SSH
(dry-run default, --execute --yes gate, parameterized host, hard
honesty rule — no false 'trains a model' claim)
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(neural): tiered `resolved` oracle for distill/weight-EFT SFT data
Replace the single structural-confidence proxy for the per-trajectory
`resolved` boolean with a 3-tier labeler where every label carries honest
provenance (ADR-169 — never present a proxy as ground truth).
- TIER 1 `oracle:test-exec` (distill-oracle.ts): real mechanical eval for
trajectories with a SWE-bench FAIL_TO_PASS spec or a metaharness/darwin
bench-suite case. Runs over SSH on a PARAMETERIZED remote (--remote / env
RUFLO_DISTILL_REMOTE, never hard-coded). DRY-RUN by default: emits the
ssh/darwin-bench/eval command plan + preflight and touches nothing; only
execute:true runs the real eval, behind a wrapped non-fatal probe.
- TIER 2 `judge:fable` (fable-harness.ts): cost-disciplined headless Fable
LLM-as-judge for trajectories with no mechanical spec. Runs `claude -p`
from a FRESH EMPTY temp cwd (no CLAUDE.md — the 5x cost driver), carries
the judge role via --append-system-prompt, BATCHES 20 items/call (~$0.02
/item vs ~$1.56 from the project dir), and enforces --max-budget-usd.
Off by default; second entry point reflectFailures() feeds GEPA/evolve.
- TIER 3 `proxy:structural`: existing output-verifier confidence, weakest,
clearly labeled.
Interface consumed by distill:
labelResolved(trajectories, opts) -> [{...traj, resolved, resolvedBy,
resolvedConfidence?, resolvedReason?}]. Default (no opts) = dry-run oracle
preflight + proxy fallback: ZERO spend, no SSH exec, no Fable call.
Everything ADR-150 optional/graceful.
Tests mock `claude -p` (spawn) and the SSH exec — assert command
construction, batching, provenance tagging, tier fallback order, and the
zero-external-call default. One opt-in live smoke behind RUFLO_FABLE_LIVE=1.
37 passing, 1 skipped (live), $0.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(neural): weight-eft $0 export slice + opt-in run-transcript capture path
Ship the ADR-150 weight-eft slice: turn ruflo run history into AUDITED
training DATA + a cost-Pareto measurement + a GPU plan. This does NOT train a
model and does NOT reduce escalation — weight-eft's own `train` never spawns
and no GPU tune has run. `resolved` is an explicitly-marked PROXY (ruflo has
no SWE-bench gold oracle).
Capture path (the blocker):
- NEW src/ruvector/run-transcript-recorder.ts — opt-in (CLAUDE_FLOW_RUN_TRANSCRIPTS=1,
off by default; PII/retention surface mirroring the router trajectory recorder)
recorder that persists full run records {instance_id, model, tier, resolved,
resolved_source, messages[], model_patch} to .swarm/run-transcripts.jsonl.
- Seam wired in src/mcp-tools/agent-execute-core.ts next to the existing
trajectory-outcome hook — the one place a run's transcript + model + tier +
resolved-proxy are all known. resolved_source='api-success' (weakest proxy),
model_patch='' (single-shot execute produces no diff) — both stamped honestly.
- router-trajectory.ts documents the companion split + adds unifiedRecorderStatus().
Distill command + service:
- NEW src/services/weight-eft.ts — optional-dep wrapper (dynamic import, graceful
{degraded:true}; local type mirrors, no static @metaharness import). Archive
builder (records → DarwinTrajectory[]), runExport/runPlan/runEval, plus a
spend-gated remote-GPU train path (buildRemoteTrainInvocation + runRemoteTrain:
DRY-RUN default, real compute only behind --execute --yes; ssh/rsync/ruvllm;
host parameterized via --remote / $RUFLO_DISTILL_REMOTE, never hard-coded).
- `neural distill export|plan|eval|train` in neural.ts.
- @metaharness/weight-eft ~0.1.1 added to optionalDependencies.
Tests (18 in __tests__/weft-export.test.ts): archive-builder mapping, export E2E
(real weight-eft → sft.jsonl/dpo.jsonl/report), degraded path (absent dep →
{degraded:true}), remote-train command construction + dry-run/refused/preflight
gates, and capture-module opt-in gating. tsc build clean; CLI startup unaffected.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* merge: consolidate step-2/step-4 loaders into agenticow-loader.ts
* merge: step 3 (checkpoint gate) + consolidate 3rd loader into agenticow-loader.ts
All agenticow consumers (verbs, swarm branches, speculative, checkpoint
gate) now import one canonical loader — the first-class compat layer.
checkpoint-gate's inline copies removed; delegates to shared loadAgenticow.
* merge: oracle + weft + wire ADR-171 promote-gate & causal receipts (#23)
explore() and agenticow_speculate now enforce the ADR-171 promotion gate:
- a winning branch is promote-INELIGIBLE unless cleared by oracle:test-exec
or an explicitly-accepted judge:fable — proxy:structural can NEVER clear,
even when it claims cleared:true (fail-closed)
- requireClearance:true fail-closes without a gate — score alone never
graduates work into base
- every discarded loser + an ineligible/failed winner emits a causal
receipt {label, score, diff, outcome, provenance, reason}
- result carries promotedBy + promotionDecision for auditability
Back-compat: no clearance gate → legacy score-only promotion, honestly
tagged 'unverified' (never masquerading as ground truth). Tests 9/9.
Co-Authored-By: RuFlo <ruv@ruv.net>
* security(distill): bare dry-run is fully offline — gate ssh preflight behind --preflight
Adversarial RC finding: the remote-train dry-run ran a read-only 'ssh
host true' reachability probe, contacting the host without --execute.
Now the bare dry-run contacts NOTHING (prints commands only); the
read-only reachability/GPU probes require an explicit --preflight (or
--execute). Closes the 'no implicit remote execution' gate.
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(ci): regenerate v3/pnpm-lock.yaml for agenticow ^0.2.4 + @metaharness/weight-eft
Lock drifted from cli/package.json — had agenticow ~0.2.4 (pkg says
^0.2.4) and was MISSING @metaharness/weight-eft entirely, failing
--frozen-lockfile → cascading 34 CI job failures (the #2540→#2552
lesson). Regenerated; frozen-lockfile now clean.
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(ci): ADR-112 guidance clause on the 5 new agenticow verb descriptions
ingest/query/diff/lineage/status said 'Use to/after/before'; the ADR-112
audit requires the literal 'Use when' guidance signal. Reworded; audit
360/360 pass.
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(ci): register CLAUDE_FLOW_RUN_TRANSCRIPTS* as escape-hatch env vars (ADR-125/130)
The ADR-173 run-transcript capture path adds opt-in, off-by-default
background-recorder env toggles (PII/retention surface, mirroring the
existing CLAUDE_FLOW_ROUTER_TRAJECTORY* recorder). They have no
user-facing command, so they're env-only by design — registered as
known escape hatches like their router-trajectory analog. Audit passes.
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(ci): tilde-pin @metaharness/weight-eft + agenticow (anti-caret guard 17z73)
The metaharness smoke's iter-110 anti-caret regression guard requires all
@metaharness/* optional deps to be tilde-pinned (~) for reproducibility
(ADR-150 review-round-1). My earlier npm-lock revert accidentally flipped
@metaharness/weight-eft (and agenticow) to caret (^). Restored to tilde;
lock regenerated, frozen-lockfile clean.
Co-Authored-By: RuFlo <ruv@ruv.net>
203 lines
7.9 KiB
TypeScript
203 lines
7.9 KiB
TypeScript
/**
|
||
* Tests for agenticow MCP tools.
|
||
*
|
||
* Two surfaces under test:
|
||
* 1. The HAPPY PATH — agenticow is installed (real package). The tool round-trips
|
||
* through actual COW branch / checkpoint / rollback / promote semantics on
|
||
* a temp .rvf file.
|
||
* 2. The DEGRADED PATH — agenticow is NOT installed (we vi.mock() it as
|
||
* ERR_MODULE_NOT_FOUND). Every tool must return `{degraded: true}` with no
|
||
* throw, matching the metaharness-tools / testgen-tools contract.
|
||
*
|
||
* Why both: agenticow is in `optionalDependencies`. CI runs with deps installed
|
||
* (happy path) but production users may not have it — the degraded path is
|
||
* the load-bearing architectural rule (zero hard runtime dep).
|
||
*/
|
||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||
import { mkdtempSync, rmSync, existsSync, statSync, readdirSync } from 'node:fs';
|
||
import { join } from 'node:path';
|
||
import { tmpdir } from 'node:os';
|
||
|
||
import { agenticowTools } from '../src/mcp-tools/agenticow-tools.js';
|
||
|
||
function findTool(name: string) {
|
||
const t = agenticowTools.find(t => t.name === name);
|
||
if (!t) throw new Error(`tool not found: ${name}`);
|
||
return t;
|
||
}
|
||
|
||
function dirBytes(p: string): number {
|
||
if (!existsSync(p)) return 0;
|
||
const st = statSync(p);
|
||
if (st.isFile()) return st.size;
|
||
let total = 0;
|
||
for (const e of readdirSync(p)) total += dirBytes(join(p, e));
|
||
return total;
|
||
}
|
||
|
||
// Detect agenticow availability at module scope (top-level await is allowed
|
||
// in vitest ESM modules). We need this before the `it.skipIf(...)` calls,
|
||
// which evaluate at definition time.
|
||
let havePkg = false;
|
||
try { await import('agenticow'); havePkg = true; } catch { havePkg = false; }
|
||
|
||
describe('agenticow MCP tools — happy path (real package)', () => {
|
||
let workdir: string;
|
||
let basePath: string;
|
||
|
||
beforeAll(() => {
|
||
workdir = mkdtempSync(join(tmpdir(), 'agenticow-tools-'));
|
||
basePath = join(workdir, 'base.rvf');
|
||
});
|
||
|
||
afterAll(() => {
|
||
try { rmSync(workdir, { recursive: true, force: true }); } catch { /* ignore */ }
|
||
});
|
||
|
||
it('exposes 9 tools (lifecycle + read/write verbs)', () => {
|
||
const names = agenticowTools.map(t => t.name).sort();
|
||
expect(names).toEqual([
|
||
'agenticow_branch',
|
||
'agenticow_checkpoint',
|
||
'agenticow_diff',
|
||
'agenticow_ingest',
|
||
'agenticow_lineage',
|
||
'agenticow_promote',
|
||
'agenticow_query',
|
||
'agenticow_rollback',
|
||
'agenticow_status',
|
||
]);
|
||
});
|
||
|
||
it.skipIf(!havePkg)('ingest → query round-trips vectors + text across a COW branch (issue: text persistence)', async () => {
|
||
const ingest = findTool('agenticow_ingest');
|
||
const query = findTool('agenticow_query');
|
||
const branch = findTool('agenticow_branch');
|
||
const v = (s: number) => Array.from({ length: 8 }, (_, i) => Math.sin(s + i));
|
||
const base = join(workdir, 'rw-base.rvf');
|
||
const br = join(workdir, 'rw-branch.rvf');
|
||
|
||
const ing = await ingest.handler({ path: base, records: [
|
||
{ id: 1, vector: v(1), text: 'alpha' }, { id: 2, vector: v(2), text: 'beta' },
|
||
], dimension: 8 }) as any;
|
||
expect(ing.success).toBe(true);
|
||
expect(ing.ingested.accepted).toBe(2);
|
||
|
||
// branch with the native ANN fast path, then ingest a branch-only vector
|
||
const b = await branch.handler({ basePath: base, branchPath: br, label: 'rw', nativeAnn: true }) as any;
|
||
expect(b.nativeAnn).toBe(true);
|
||
await ingest.handler({ path: br, records: [{ id: 3, vector: v(3), text: 'gamma' }] });
|
||
|
||
// query the branch: sees base ∪ branch, and text survives (0.2.4 fix)
|
||
const q = await query.handler({ path: br, vector: v(1), k: 3 }) as any;
|
||
expect(q.success).toBe(true);
|
||
const hit = q.hits.find((h: any) => h.id === 1);
|
||
expect(hit).toBeDefined();
|
||
expect(hit.text).toBe('alpha');
|
||
});
|
||
|
||
it.skipIf(!havePkg)('diff / lineage / status report branch state', async () => {
|
||
const ingest = findTool('agenticow_ingest');
|
||
const diff = findTool('agenticow_diff');
|
||
const lineage = findTool('agenticow_lineage');
|
||
const status = findTool('agenticow_status');
|
||
const v = (s: number) => Array.from({ length: 8 }, (_, i) => Math.cos(s + i));
|
||
const p = join(workdir, 'introspect.rvf');
|
||
await ingest.handler({ path: p, records: [{ id: 10, vector: v(1), text: 'x' }], dimension: 8 });
|
||
|
||
const d = await diff.handler({ path: p }) as any;
|
||
expect(d.success).toBe(true);
|
||
expect(Array.isArray(d.diff.added)).toBe(true);
|
||
|
||
const lin = await lineage.handler({ path: p }) as any;
|
||
expect(lin.success).toBe(true);
|
||
expect(lin.lineage.some((n: any) => n.role === 'working' || n.role === 'base')).toBe(true);
|
||
|
||
const st = await status.handler({ path: p }) as any;
|
||
expect(st.success).toBe(true);
|
||
expect(st.status.dimension).toBe(8);
|
||
});
|
||
|
||
it('new read/write verbs degrade + validate like the lifecycle verbs', async () => {
|
||
// path-traversal rejection is shared via resolveMemoryPath
|
||
const query = findTool('agenticow_query');
|
||
await expect(query.handler({ path: '../../etc/passwd', vector: [1, 2] })).rejects.toThrow(/disallowed/);
|
||
const ingest = findTool('agenticow_ingest');
|
||
await expect(ingest.handler({ path: 'ok.rvf', records: [] })).rejects.toThrow(/non-empty/);
|
||
});
|
||
|
||
it('every tool has a JSON-schema input + handler', () => {
|
||
for (const t of agenticowTools) {
|
||
expect(t.inputSchema.type).toBe('object');
|
||
expect(t.inputSchema.properties).toBeDefined();
|
||
expect(typeof t.handler).toBe('function');
|
||
}
|
||
});
|
||
|
||
it.skipIf(!havePkg)('agenticow_branch creates a tiny COW branch on a real base', async () => {
|
||
const branch = findTool('agenticow_branch');
|
||
// Use a small dim for test speed
|
||
const branchPath = join(workdir, 'b1.rvf');
|
||
const result: any = await branch.handler({
|
||
basePath, branchPath,
|
||
label: 'test-branch',
|
||
dimension: 32,
|
||
});
|
||
expect(result.success).toBe(true);
|
||
expect(result.degraded).toBeUndefined();
|
||
expect(result.label).toBe('test-branch');
|
||
expect(existsSync(branchPath)).toBe(true);
|
||
// The size win is the load-bearing claim — assert it's tiny
|
||
expect(dirBytes(branchPath)).toBeLessThan(2048);
|
||
});
|
||
|
||
it.skipIf(!havePkg)('agenticow_checkpoint persists state across reopen, agenticow_rollback restores it', async () => {
|
||
const cp = findTool('agenticow_checkpoint');
|
||
const rb = findTool('agenticow_rollback');
|
||
|
||
// checkpoint must persist via the .agenticow.json manifest. The base file
|
||
// was created earlier by the branch test (which writes the manifest).
|
||
const cpResult: any = await cp.handler({ path: basePath, label: 'cp-1' });
|
||
expect(cpResult.success).toBe(true);
|
||
expect(cpResult.label).toBe('cp-1');
|
||
// Manifest file should now exist next to the .rvf
|
||
expect(existsSync(`${basePath}.agenticow.json`)).toBe(true);
|
||
|
||
// Rollback now finds the checkpoint via the persisted manifest
|
||
const rbResult: any = await rb.handler({ path: basePath });
|
||
expect(rbResult.success).toBe(true);
|
||
expect(rbResult.rolledBack).toBe(true);
|
||
});
|
||
|
||
it('agenticow_branch rejects path traversal in basePath', async () => {
|
||
const branch = findTool('agenticow_branch');
|
||
await expect(branch.handler({
|
||
basePath: '../../../../etc/passwd',
|
||
branchPath: join(workdir, 'evil.rvf'),
|
||
label: 'evil',
|
||
dimension: 32,
|
||
})).rejects.toThrow(/disallowed characters/);
|
||
});
|
||
|
||
it('agenticow_branch rejects a malformed label', async () => {
|
||
const branch = findTool('agenticow_branch');
|
||
await expect(branch.handler({
|
||
basePath,
|
||
branchPath: join(workdir, 'b2.rvf'),
|
||
label: 'evil; rm -rf /',
|
||
dimension: 32,
|
||
})).rejects.toThrow(/may only contain/);
|
||
});
|
||
|
||
it('agenticow_branch requires dimension when creating a new memory file', async () => {
|
||
const branch = findTool('agenticow_branch');
|
||
const freshBase = join(workdir, 'nonexistent.rvf');
|
||
await expect(branch.handler({
|
||
basePath: freshBase,
|
||
branchPath: join(workdir, 'b3.rvf'),
|
||
label: 'fresh',
|
||
// no dimension
|
||
})).rejects.toThrow(/dimension is required/);
|
||
});
|
||
});
|