Files
Dragan Spiridonov 4c0bdfafb9 feat(observability): daemon pidfile fix + learning loop-health (#488 Phase 1)
Phase 1 of the #488 production-readiness work, scoped to additive changes
that don't need new ADRs. Two parts:

B.1 — Daemon pidfile points at the npx wrapper, not the MCP server
=================================================================

When neither `node_modules/.bin/aqe-mcp` nor the local
`node_modules/agentic-qe/dist/mcp/bundle.js` exists, the generated
`.agentic-qe/workers/start-daemon.cjs` falls back to
`npx --yes agentic-qe mcp`. `npx` exits as soon as it has forked the real
bundle, so `child.pid` is the wrapper PID and the pidfile becomes stale
within seconds. `aqe daemon status` then misreports liveness, and the
idempotency check at the top of the script spawns duplicate daemons —
the reporter hit this twice in two days.

Adds `require.resolve('agentic-qe/dist/mcp/bundle.js')` as a 3rd
candidate (try/catch-guarded) before the npx fallback. Globally-installed
shops (`npm install -g agentic-qe` — the typical case the npx fallback
was silently covering) now spawn the real bundle directly; `child.pid`
is the long-lived MCP server.

The npx fallback remains as a last resort for environments where neither
local nor global install is reachable. When taken, it now logs a WARNING
to daemon.log so operators can see the degraded pidfile contract.

B.2 — `learning:loop-health` observability
==========================================

Operators previously had no way to answer "is the self-learning loop
closing end-to-end?" without DB-mining `dream_cycles`, `kv_store`, and
`routing_outcomes` separately. Cumulative counters in `aqe learning
stats` grow forever and can't distinguish "healthy loop" from "loop
wedged 3 hours ago with stale totals".

Adds a single `learning:loop-health` kv key with per-component liveness:

  { overallLastSuccess, bootedAt,
    components: {
      bridge?:          { lastSuccessAt, ticksSinceBoot, ... },
      learningWorker?:  { lastSuccessAt, ticksSinceBoot, ... },
      dreamScheduler?:  { lastSuccessAt, ticksSinceBoot, ... },
    } }

Writers (best-effort — observability MUST NOT crash the observed
component):
  - CapturedExperienceBridge.drainSafe — every poll, success or failure
  - LearningConsolidationWorker.doExecute — end of every tick
  - DreamScheduler reserved for #488 Phase 2 (the field stays
    undefined until Phase 2 wires the scheduler in-kernel)

Surface:
  - new `aqe learning loop-health [--json]` CLI subcommand with
    component verdicts (live/stale/never-ran) based on per-component
    staleness thresholds (bridge: 30s; worker/scheduler: 2h)
  - distinct from the existing `aqe learning health` (per-domain
    regret analysis); the new command is pipeline liveness, not
    agent outcome quality

Tests:
  - 12 tests in tests/unit/learning/loop-health.test.ts for the
    utility: fresh record creation, tick counters, error capture +
    clear-on-success, multi-component isolation, overallLastSuccess
    max-computation, throw-safety when the memory backend is broken,
    staleness verdicts
  - 4 tests in tests/unit/init/phases/10-workers-daemon-pidfile.test.ts
    asserting the require.resolve probe is present, try/catch-guarded,
    ordered before the npx fallback, and that the warning string is
    in place

541/541 tests pass across CLI commands, workers, bridge, learning,
and init/phases suites; build clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 13:11:32 +00:00
..