Files
rUv 70e2339469 feat(federation): ADR-111 Phases 4-6 — firewall projection + witness chain + MCP tools (#1895)
* feat(federation): ADR-111 Phases 4-6 — firewall projection + witness chain + MCP tools

Phases 4-6 of ADR-111. Phase 7 (cross-OS WG mesh bringup) is operator-mediated
and lives outside the package — staged but not auto-executed per CLAUDE.md
destructive-actions guidance.

## Phase 4 — Trust-graded firewall projection
`domain/services/wg-firewall-service.ts` (new)
- Two backends: linux-nftables (atomic `nft -f`, table `inet ruflo_fed`,
  default DROP, interface-scoped) and darwin-pf (anchor-scoped pfctl).
- Projects WG_NETWORK_GATES → concrete rule lines per peer.
- UNTRUSTED excluded (no mesh IP), peers without wgMeshIP skipped.
- Defense-in-depth: regex-validated rule args (no shell metachars in
  pubkey/IP/iface).
- Returns rule file content + the load command the operator runs —
  service never shells out.
- 16 unit tests.

## Phase 5 — Witness attestation chain
`domain/services/wg-witness-service.ts` (new)
- Append-only chain of Ed25519-signed entries. Each entry hash-links to
  the previous (`prevHash` = sha256 of previous canonical encoding).
- Canonical encoding: sorted keys, no whitespace, omitted-on-undefined.
- `verifyWitnessChain` walks the chain and reports `failedAt` + reason
  (`broken-chain-link` | `invalid-signature-or-hash`).
- `attestWgCommand` adapter maps WgCommand.verb → event type.
- 11 unit tests (canonicalization stability, deterministic hash,
  chain link integrity, signature-tamper detection).

## Phase 6 — Operator MCP tools
`mcp-tools.ts` (extended)
- `federation_wg_status` — per-peer mesh state (trust, meshIP, AllowedIPs,
  suspended/evicted flags) via `WgMeshService.summarize()`.
- `federation_wg_attest` — operator-signed witness chain entry. v1 stub
  returns wire-up instructions until plugin.ts integrates WgWitnessService.
- `federation_wg_keyrotate` — emits a new X25519 keypair (public key only
  via MCP; private key write is operator-mediated) + the upgrade checklist.
  Requires `confirm: true` (destructive — drops existing tunnels until peers
  fetch updated manifest).
- `createMcpTools` gets optional `getWgMesh` getter (additive, no break).
- `FederationCoordinator.listPeers()` added as public discovery accessor.

## Tests
- 543/543 pre-existing + new tests pass (513 + 27 phase 4/5 + 3 plugin
  test count assertions for the new MCP tools).

## Not in this PR (Phase 7 — operator-mediated)
- Real WG mesh bringup on mac↔ruvultra over tailscale: stage configs,
  operator-approved `wg-quick up`, peer-add via federation manifests,
  SUSPEND → confirm L3 isolation → reactivate restores, witness-verify
  the change log. Tracked alongside #1879.

Co-Authored-By: RuFlo <ruv@ruv.net>

* fix(federation): security — validate peer wg fields before splice (ADR-111)

Manual security audit of ADR-111 Phases 1-6 found a real vulnerability in
Phase 2's WgMeshService that the existing tests did not catch.

## The bug

`WgMeshService.buildInterfaceConfig()` and `summarize()` spliced
`peer.metadata.wgPublicKey` / `wgMeshIP` / `wgEndpoint` into the wg-quick
config string *without validation*. The ADR's threat model explicitly
includes "compromised federation peer with valid WG key" — that peer
signs their own manifest, so the Ed25519 signature only proves origin,
not content safety.

Concrete attack: a compromised peer publishes

  wgEndpoint = "host:51820\n[Peer]\nPublicKey = attacker\nEndpoint = ..."

and the operator's wg-quick config ends up with an extra [Peer] block.
Cross-peer lateral movement, bypassing the per-peer trust gates.

Phase 4 (WgFirewallService) already validated meshIP via
`assertSafeRuleArg`. Phase 2 didn't — fixed here.

## The fix

New `readSafePeerWgFields(peer)` returns the validated triple or null.
Three regexes:
  - wgPublicKey: ^[A-Za-z0-9+/]{43}=$            (base64 X25519, 32 raw bytes)
  - wgMeshIP:    ^d.d.d.d/32$ + octet range check
  - wgEndpoint:  ^(hostname|ipv4|[ipv6]):port$  + port range check

Mismatched fields → peer is skipped entirely (no [Peer] block, no
AllowedIPs, summarize() returns empty strings). Silent skip beats partial
write because partial writes can leak attacker content into operator
review while looking syntactically valid.

## Tests

7 new adversarial cases under `WgMeshService — readSafePeerWgFields
(compromised-peer defense)`:
  - newline injection in wgEndpoint
  - wrong-length / non-base64 wgPublicKey
  - out-of-range octet in wgMeshIP
  - missing /32 CIDR
  - port out of range
  - IPv6 bracket positive case
  - summarize() also sanitizes (regression: it used the same unsafe path)

Full suite: 550/550 pass.

Co-Authored-By: RuFlo <ruv@ruv.net>

* feat(federation): ADR-111 Phase 7 staging helper

Generates WG keypair + wg-quick config + per-OS firewall rules on the
current host, naming a single peer. Outputs go to /tmp/adr-111-stage.
Nothing loaded into the kernel — wg-quick up / nft -f / pfctl -f are
operator-mediated.

Usage:
  node scripts/phase7-stage.mjs <localId> <peerId> <peerPk> <peerIP> <peerEndpoint>

Workflow:
  1. Run on host A with placeholder peer values → emit host-A pubkey
  2. Run on host B with placeholder peer values → emit host-B pubkey
  3. Re-run on both with the OTHER host's pubkey → final config files
  4. Operator reviews + activates per the printed checklist

Co-Authored-By: RuFlo <ruv@ruv.net>

* fix(federation): ADR-111 phase7-stage idempotent — reuse key file

Without persistence, every run generates a fresh keypair, which breaks
the cross-feed workflow (run on host A → capture pubkey → run on host B
with A's pubkey → A regenerates a different key on its next run).

Now reads /tmp/adr-111-stage/wg-key-<nodeId>.json if present, falls back
to fresh generation if not. Operator can delete the file to force
rotation.

Co-Authored-By: RuFlo <ruv@ruv.net>

* docs(federation): user guide + Phase 7 bringup procedure + README link

Adds:
  - docs/federation/README.md — full user guide (when to use, MCP tools,
    trust levels, breaker, ADR-111 mesh layer, claude -p headless usage)
  - docs/federation/phase7-mesh-bringup.md — step-by-step cross-OS WG
    mesh activation procedure with operator review checklist
  - README.md — link to docs/federation/ from the existing Federation
    section

Co-Authored-By: RuFlo <ruv@ruv.net>

---------

Co-authored-by: Reuven <cohen@ruv-mac-mini.local>
2026-05-10 23:03:19 -04:00
..

Ruflo Federation — User Guide

Cross-installation agent peering with built-in cost limits, circuit breaker, signed envelopes, and (as of alpha.14) opt-in WireGuard mesh layer governed by federation trust.

This guide walks through what federation is, when to use it, and how to set it up. For the architectural backstory and per-phase release notes see the companion gist.

What federation does

Federation lets two or more Ruflo installations — your mac, a server, a teammate's laptop — discover each other, exchange signed manifests, and send messages between them with bounded cost and per-peer trust gates. Key properties:

  • Ed25519 identity — each node holds a private key; peers exchange Ed25519-signed manifests. No central directory.
  • Five-level trust ladder — UNTRUSTED → VERIFIED → ATTESTED → TRUSTED → PRIVILEGED. Each level unlocks a wider set of operations (discovery, send, share-context, remote-spawn, …).
  • Per-peer budget + circuit breaker — bounded tokens/USD per peer. Sustained failures auto-SUSPEND; further failures EVICT. ADR-097.
  • PII pipeline + audit trail — every cross-peer envelope passes through PII detection. Every state transition is auditable.
  • Real wire transport — WSS with permessage-deflate compression, optional cert pinning, stream multiplexing. ADR-104.
  • Optional WG mesh layer — opt-in opaque packet-layer reachability that follows federation trust changes. Compromised peer auto-isolated at L3 when the breaker fires. ADR-111.

When to use federation

Use case Fit
Two laptops collaborating on a project, want bounded cost sharing + audit ✅
Personal home server agent ↔ travel laptop ✅
Team of 5 engineers sharing memory/skills across machines ✅
Mobile / Windows / sub-50 peers with NAT issues ✅ over Tailscale + federation
Public-internet exposed agent endpoints ✅ with TLS cert pinning (ADR-107)
Internal HR/finance multi-agent workflow with strict access tiers ✅ with PRIVILEGED gating + audit
Replacing Slack/Discord — no ❌ federation is for agent-to-agent, not human chat
Untrusted-internet messaging without identity vetting ❌ — trust ladder must be bootstrapped out-of-band

Quick start

1. Install the plugin

npx ruflo@latest                                     # if you don't have it yet
npx ruflo plugins install @claude-flow/plugin-agent-federation

Or directly via npm:

npm i @claude-flow/plugin-agent-federation@latest    # currently 1.0.0-alpha.14

2. Initialize a node

npx claude-flow@v3alpha agent spawn -t federation --name fed-1

Or via the MCP tool federation_init:

{
  "tool": "federation_init",
  "params": {
    "nodeId": "my-mac",
    "endpoint": "ws://my-mac.tailnet:9100",
    "agentTypes": ["coder", "tester"]
  }
}

This generates an Ed25519 keypair (persisted to .claude-flow/federation/keys-<nodeId>.json, mode 0600), publishes a signed manifest, and starts the discovery service.

3. Join a peer

{
  "tool": "federation_join",
  "params": {
    "endpoint": "ws://other-host.tailnet:9100"
  }
}

The handshake exchanges manifests, verifies Ed25519 signatures, and establishes the initial trust level (UNTRUSTED until the trust-evaluator records enough successful interactions to promote).

4. Send a message

{
  "tool": "federation_send",
  "params": {
    "targetNodeId": "other-host",
    "messageType": "agent-handoff",
    "payload": { "task": "Investigate the failing integration test" }
  }
}

Budget and trust gates apply: if the peer is below ATTESTED, only discovery/status/ping go through. If the breaker has the peer SUSPENDED, the send short-circuits with PEER_SUSPENDED.

MCP tools at a glance

Tool What it does Trust gate
federation_init Initialize this node —
federation_join Join a peer by endpoint —
federation_peers List discovered peers —
federation_send Send a typed message to a peer per-peer (varies)
federation_query Synchronous query → response ATTESTED+
federation_status Current node + peer trust summary —
federation_trust View / adjust trust levels operator
federation_audit Read audit log operator
federation_breaker_status Per-peer state, when changed, why —
federation_evict Operator manual evict operator
federation_reactivate Operator manual reactivate operator
federation_report_spend Report cost of a completed call integrator
federation_consensus Federated proposal across peers varies
federation_wg_status (ADR-111) Per-peer mesh state —
federation_wg_attest (ADR-111) Operator-signed witness entry operator
federation_wg_keyrotate (ADR-111) Rotate WG keypair operator + confirm:true

Trust levels — what each unlocks

Level Capabilities (federation) WG reachability (if ADR-111 active)
UNTRUSTED discovery Excluded from mesh — drop all
VERIFIED + status, ping Discovery port (9100) only
ATTESTED + send, receive, query-redacted + federation messaging (9101-9199)
TRUSTED + share-context, collaborative-task + ssh (22), services (80/443)
PRIVILEGED + full-memory, remote-spawn Full mesh

Trust is earned via repeated successful interactions (the TrustEvaluator tracks score + interaction count). Promotion thresholds are documented in domain/entities/trust-level.ts.

Circuit breaker

If a peer's failure ratio or cost spend exceeds the policy:

ACTIVE → SUSPENDED → EVICTED
   ↑          │
   └─────────operator-only─────────┘
  • SUSPEND: peer's outbound sends short-circuit (PEER_SUSPENDED). Existing sessions continue; new sends rejected. Auto-eviction on continued failures.
  • EVICT: peer's outbound sends short-circuit (PEER_EVICTED). Session terminated. Operator must explicitly reactivate.
  • The breaker does not auto-reactivate. The integrator's health probe is responsible for confirming the peer is healthy and calling federation_reactivate.

Policy is tunable; defaults are conservative (50+ samples needed before suspend, high failure ratio threshold).

ADR-111 — WireGuard mesh (opt-in, since alpha.14)

Federation today treats network connectivity as the integrator's problem (Tailscale, LAN, wss://+pinning). That works but trust changes don't propagate to the L3 layer — a peer EVICTED in federation stays in the tailnet until an admin manually removes it.

ADR-111 closes that gap with an optional in-tree WG mesh:

  • WG keypair generated alongside the federation key
  • Mesh IP derived deterministically from nodeId (sha256 → 10.50.0.0/16 host portion, with collision-handling probe loop)
  • WG identity published inside the same Ed25519-signed manifest
  • Federation breaker SUSPEND → wg set ... allowed-ips "" (soft-block at L3)
  • Federation breaker EVICT → wg set ... remove (terminal)
  • Operator reactivate → AllowedIPs restored
  • Each mutation entered into an append-only Ed25519-signed witness chain

Phases shipped in alpha.14:

  • Phase 1 — Manifest extension + key generation
  • Phase 2 — WgMeshService (no shell — emits configs + commands)
  • Phase 3 — Coordinator/breaker wiring
  • Phase 4 — Firewall projection (nftables / pf) — PR #1895
  • Phase 5 — Witness attestation chain — PR #1895
  • Phase 6 — Operator MCP tools — PR #1895

Phase 7 — operator-mediated: See docs/federation/phase7-mesh-bringup.md for the cross-OS bringup procedure (mac ↔ ruvultra over Tailscale).

Enabling ADR-111

Set config.wgMesh: true in your federation plugin config, then run the staging helper:

node v3/@claude-flow/plugin-agent-federation/scripts/phase7-stage.mjs \
  <localNodeId> <peerNodeId> <peerPubkey> <peerMeshIP> <peerEndpoint>

The script generates /tmp/adr-111-stage/:

  • wg-key-<nodeId>.json (mode 0600 — your private WG key)
  • ruflo-fed.conf (the wg-quick interface config)
  • ruflo-fed.nft or ruflo-fed.pf (firewall projection)

After review, the operator manually activates:

sudo install -m 0600 /tmp/adr-111-stage/ruflo-fed.conf /etc/wireguard/ruflo-fed.conf
# Linux:
sudo nft -f /tmp/adr-111-stage/ruflo-fed.nft
# macOS:
sudo pfctl -a ruflo-fed -f /tmp/adr-111-stage/ruflo-fed.pf
sudo wg-quick up ruflo-fed

Using claude -p headless mode

claude -p (print/pipe mode) can drive federation MCP tools non-interactively. Each invocation processes its prompt and exits — for a persistent federation listener, run a long-lived MCP server instead.

On the originating host (mac mini):

claude -p --model haiku --max-budget-usd 0.20 --output-format text \
  "Federation MCP tools to verify cross-machine peering health: name 3. One line."
# → Three Federation MCP tools for peering health verification:
#   federation-init (keypair generation), federation-status (peers/trust/metrics),
#   federation-audit (compliance filtering).

On the peer host (ruvultra) — requires interactive /login first since claude -p reads stored credentials:

# First time only — operator-mediated:
claude   # then /login

# After that:
claude -p --model haiku --max-budget-usd 0.20 \
  "Run federation_status MCP and report peer count."

Workflow for cross-machine task handoff:

# Host A — kick off a federated task:
claude -p --model sonnet --output-format json --resume <session> \
  "Use federation_send to dispatch this task to ruvultra: analyze the failing test"

# Host B (ruvultra) — receive + work:
claude -p --resume <session> "Continue handling the federated task"

The federation plugin handles signing, PII gating, breaker, and audit on every send. The two claude -p invocations don't share state directly — they communicate exclusively through the federation envelope channel.

Anti-goals — when NOT to use federation

  • Replacement for Slack/Discord. Federation moves agent envelopes, not human chat.
  • Public internet without identity vetting. Trust ladder bootstrapping is your responsibility.
  • NAT traversal magic. Use Tailscale (or Headscale) for that; federation rides on top.
  • A general-purpose RPC framework. It's specifically for AI agents with cost-aware budgeting.

Where things live

What Path
Plugin source v3/@claude-flow/plugin-agent-federation/src/
Tests v3/@claude-flow/plugin-agent-federation/__tests__/
ADRs v3/docs/adr/ADR-{097,104,105,106,107,109,110,111}-*.md
Phase 7 staging script v3/@claude-flow/plugin-agent-federation/scripts/phase7-stage.mjs
Witness signing plugins/ruflo-core/scripts/witness/

Releases

Version What landed
1.0.0-alpha.9 First user-visible release — see announcement gist
1.0.0-alpha.10 ADR-097 Phases 2.a-4 + ADR-104 transport + ADR-109 inbound dispatcher
1.0.0-alpha.11-12 ADR-109 sig verify, ADR-104 compression, ADR-107 TLS cert pinning
1.0.0-alpha.13 ADR-104 stream multiplexing + ADR-110 MemorySpendReporter
1.0.0-alpha.14 ADR-111 Phases 1-3 (WG mesh foundation)
1.0.0-alpha.15 (in flight) ADR-111 Phases 4-6 (firewall + witness + MCP tools) — PR #1895
  • ADR-097 — budget + circuit breaker
  • ADR-104 — WSS transport + multiplexing
  • ADR-105 — state snapshot/replay
  • ADR-106 — discovery mechanisms
  • ADR-107 — TLS + cert pinning
  • ADR-109 — inbound dispatch + sig verify
  • ADR-110 — production SpendReporter
  • ADR-111 — WG mesh layer

Support