Two follow-up fixes after re-running the broader test suite (2064 tests):
1. tests/unit/shared/llm/router/hybrid-router.test.ts:619 — the
pre-existing test "should not retry non-retryable errors" was
asserting the OLD short-circuit-on-non-retryable behavior that
Fix#3 in the audit corrected. Updated to assert the new (correct)
semantics: non-retryable errors don't re-try the SAME provider but
DO fall through to the NEXT provider in the fallback chain.
Renamed accordingly. Both providers fail non-retryable so the
request still rejects, preserving the original "rejects" assertion.
2. README.md "LLM Providers" section was under-documented relative to
the wired implementation. Updated to:
- List all 7 ExtendedProviderType providers with their env var names
(including the GOOGLE_API_KEY alias added in Phase 1)
- Document `aqe llm config --set` persistence path
- Document the AQE_LLM_ROUTER_DISABLED kill-switch
- Mention that apiKey can't be saved via CLI (env-only)
- Drop Groq from the table since it's not actually in the supported
provider list
Verification
- npx tsc --noEmit: clean
- 2063/2063 unit + integration tests pass (1 skipped, no failures)
- Includes 38/38 hybrid-router tests after the assertion update
Wires the foundation for ADR-043's HybridRouter into the kernel boot
sequence so domain plugins can finally receive an llmRouter dependency.
Phase 1 lands the kernel/CLI/provider changes; Phase 2 will plumb the
router through all 13 domains.
Background: the LLM-enhancement code paths added in ADR-051
(isLLMAnalysisAvailable() across 14 services) were unreachable because
no construction site ever passed an llmRouter — services accepted the
optional dependency but no caller injected it. ProviderManager +
HybridRouter were only instantiated from the `aqe llm advise` CLI
subcommand.
Changes
- src/shared/llm/providers/gemini.ts: read GOOGLE_API_KEY as a third
fallback after GOOGLE_AI_API_KEY and GEMINI_API_KEY, so users with
the conventional Google key name don't need to alias it themselves.
- src/shared/llm/router/config-store.ts (new): load/save
.agentic-qe/llm-config.json with env-detection layering
(defaults < disk < env < explicit override). Strips apiKey fields
defensively before persisting. Filters out providers the
ProviderManager doesn't yet construct (onnx) to keep boot logs
clean.
- src/shared/llm/llm-router-service.ts (new): builds a HybridRouter
from the loaded config, picks primary/fallbacks from the available
set honoring the user's defaultProvider when possible. Accepts an
injected ProviderManager for tests.
- src/kernel/interfaces.ts: extends QEKernel with llmRouter? and
KernelConfig with llmRouter.enabled ('auto'|true|false) +
configOverride + providerManager test-injection.
- src/kernel/kernel.ts: kernel-singleton llmRouter built during
initialize(), exposed via kernel.llmRouter, forwarded to all 13
DOMAIN_FACTORIES as a 4th arg. Phase 2 wires each factory through
to its coordinator/services; for now the wrappers accept-and-discard.
- src/cli/commands/llm-router.ts: replaces the
`// would be persisted in real implementation` stub with real
read/write through the config-store. `aqe llm config --set` now
survives a process restart. Adds AQE_CONFIG_ROOT env var as a test
seam so tests don't mutate the real .agentic-qe/.
Tests (110 total, all passing)
- gemini.test.ts: +3 new tests for each env-var alias, existing
tests updated to clear all three names
- config-store.test.ts (new, 29 tests): load/save/merge precedence,
env detection, secret stripping, atomicity
- llm-router-service.test.ts (new, 10 tests): pickEnabledProviders,
pickPrimaryAndFallbacks, end-to-end with injected mock provider
- kernel-llm-router.test.ts (new, 4 tests): kernel boot with router
enabled=false / auto-no-providers / injected mock, chat() round-trip
- llm-router.test.ts (CLI): +2 persistence-to-disk tests, env-isolated
- tests/mocks/index.ts: shared createMockLLMProvider() helper
Verification
- npx tsc --noEmit: clean
- 110 tests pass; no regressions in existing kernel.test.ts (34/34)
- Stderr noise from PersistentSONAEngine init is pre-existing and
unrelated to this change.
What this commit does NOT do
- Does NOT change behavior for any user yet — DOMAIN_FACTORIES
wrappers still discard the llmRouter param. Phase 2 lights up the
domain services.
Refs ADR-043 (vendor-independent LLM), ADR-051 (LLM-enhanced services)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Commit 3 of 6 for ADR-093 Opus 4.7 migration. Adds effort-level plumbing
with xhigh as the fleet-wide default and max override on security agents.
* src/shared/llm/effort-resolver.ts (new): resolveEffortLevel() implements
the ADR-093 priority chain:
runtime override > per-agent frontmatter > QE_EFFORT_LEVEL env
> config/fleet-defaults.yaml > DEFAULT_EFFORT_LEVEL (xhigh)
Plus downgradeEffort() for models that don't support xhigh.
Minimal regex-based YAML parse avoids a new dependency.
* config/fleet-defaults.yaml (new): effort_level: xhigh.
* src/shared/llm/interfaces.ts: GenerateOptions gains optional
`effort?: 'low' | 'medium' | 'high' | 'xhigh' | 'max'`.
* src/shared/llm/providers/claude.ts: resolve effort per request and
include in /v1/messages body; silently downgrade to 'high' when the
target model lacks supportsEffortXHigh capability.
* .claude/agents/v3/qe-security-auditor.md,
qe-pentest-validator.md,
qe-security-scanner.md: add `effort: max` frontmatter per user
direction. Also corrects stale `advisor.model: claude-opus-4-6`
(non-existent ID) to `claude-opus-4-7` on the two agents that had it.
Note: fleet has 3 security/pentest qe-* agents (qe-security-auditor,
qe-security-scanner, qe-pentest-validator). The Cyber Verification
application lists 4 (including qe-security-reviewer which does not
exist in .claude/agents/v3/). If the submitted application needs an
amendment, re-file with the correct 3-agent list.
* tests/unit/shared/llm/effort-resolver.test.ts (new): 13 tests
covering the priority chain, invalid-input fallback, YAML read,
and downgrade helper.
Verification:
* 0 tsc errors (npx tsc --noEmit)
* 677/677 LLM unit tests pass (+13 new)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
125 test files had beforeEach/beforeAll but no afterEach/afterAll,
leaking state between tests and causing order-dependent failures.
Added cleanup to 104 files (skipped 4 archived, 16 import-only):
- 2 critical DB handle leaks (witness-chain): added db.close()
- 7 high-risk service files: added vi.restoreAllMocks()
- 7 beforeAll files: added matching afterAll
- 88 medium/low risk: added afterEach with appropriate cleanup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update default embed model from nomic-embed-text to all-MiniLM-L6-v2
in Ollama LLM provider
- Fix 15 test files with hardcoded 768-dim references
- Update mock model names in Ollama health checks
- Update comments referencing old model name
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Dragan Spiridonov