mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
Dragan Spiridonov
af4140efb9
Apply Phoenix Architecture principles to AQE — regenerate internals, conserve the interface. ADR-113 (evals are oracles, durable-first tests, mutation-score gate): - src/validation/oracle-eval.ts grades a generated test by running it against a reference impl + operator mutants (reuses qe-arena, ADR-104); rejects assertion-less output - oracle mode wired into scripts/run-skill-eval.ts (skips under the simulating runner; runs live) - qe-test-generation rewritten durable-first; its eval flipped to oracle - src/feedback/regenerability-gate.ts: mutation-score + regenerability gate (warn-default, opt-in block) - live-validated: Claude sonnet-4-6 clears the oracle 5/5; weak local-model output correctly fails ADR-114 (conservation layer, Phoenix essay 14): - src/validation/conservation-guard.ts + scripts/conservation-guard.ts guard CLI commands, output schemas, and dashboard API against breaking removals (deprecation registry exempts) - policy doc + baselines in verification/conservation/; wired into invariant-check.yml non-blocking LLM provider fixes: - OpenRouterProvider.type was 'openai' but registered under 'openrouter' -> ProviderManager failover never found it; fixed to 'openrouter' end-to-end - Gemini default model gemini-1.5-pro was retired -> gemini-2.5-flash; models/pricing refreshed from the live Gemini ListModels API - scripts/provider-health.ts (npm run providers:health) Cheaper-model eval lanes: - scripts/oracle-model-bench.ts + openrouter-models.ts; openai/gpt-oss-120b clears the oracle ~50-70x cheaper than Claude Sonnet; guide in docs/guides/cheaper-model-eval-lanes.md Verification: npm run build ok; 115 new+touched tests pass; 724 LLM tests pass; lint + parity clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
46 lines
620 B
JSON
46 lines
620 B
JSON
{
|
|
"surface": "cli-commands",
|
|
"count": 39,
|
|
"entries": [
|
|
"agent",
|
|
"arena",
|
|
"audit",
|
|
"brain",
|
|
"ci",
|
|
"code",
|
|
"completions",
|
|
"coverage",
|
|
"daemon",
|
|
"domain",
|
|
"eval",
|
|
"fleet",
|
|
"health",
|
|
"heartbeat",
|
|
"hooks",
|
|
"hypergraph",
|
|
"init",
|
|
"learning",
|
|
"llm",
|
|
"mcp",
|
|
"memory",
|
|
"pipeline",
|
|
"platform",
|
|
"plugin",
|
|
"protocol",
|
|
"prove",
|
|
"quality",
|
|
"routing",
|
|
"ruvector",
|
|
"security",
|
|
"skill",
|
|
"status",
|
|
"sync",
|
|
"task",
|
|
"test",
|
|
"token-usage",
|
|
"upgrade",
|
|
"validate",
|
|
"workflow"
|
|
]
|
|
}
|