Files
proffesor-for-testing__agen…/tests/unit/packaging/dependency-surface.test.ts
Dragan Spiridonov 9b6de39792 fix: resolve issues #565, #567, #568, #569 (deps, LLM wiring, agent routing, coverage honesty)
Four independent user-reported issues, each verified end-to-end through the real
MCP protocol server and adversarially reviewed via qe-court.

#565 — HIGH CVE reachable by consumers
`@huggingface/transformers` was an *optional* peerDependency, but npm >=7
auto-installs optional peers — `optional` only suppresses the error when the peer
cannot be resolved. Every `npm install agentic-qe` therefore pulled
onnxruntime-node -> adm-zip@0.5.x (GHSA-xcpc-8h2w-3j85), and no resolvable
version of that chain is unaffected. Dropped the peer (it stays a devDependency
and an opt-in for local in-process embeddings; both load sites already degrade
with an actionable message). Consumer audit: 4 HIGH -> 0.

#567 — test_generate_enhanced never invoked the LLM
The DI factory behind the task-executor path was registered as `(memory) => ...`,
so `llmRouter` never reached TestGeneratorService and the ADR-051 branch was
unreachable regardless of provider configuration. Threaded the router through the
registry -> resolveTestGeneratorService -> getTestGenerator (now async), honored
the documented `aiEnhancement` opt-out, and surfaced `llmEnhanced` /
`generationMode` / `note` so a silent fallback is diagnosable. `llmEnhanced` now
reflects what the LLM actually produced — a broken provider (401/timeout/empty)
no longer reports AI-enhanced output it did not produce. The CLI was unaffected
(it goes through the domain plugin, which was already wired).

#568 — per-agent provider routing had no on-disk surface
Added `agentOverrides` to `.agentic-qe/llm-config.json` with keyed-object merge,
validation that drops bad entries individually and strips leaked API keys, and
resolution ahead of the category defaults. Materialized the overrides into
routing rules the router actually evaluates, and threaded `agentType` through all
nine router-calling domain services — without it a parsed override built a rule
that never fired. Also added the missing `codex` entry to ALL_PROVIDER_TYPES,
which had made that provider unselectable through the router.

#569 — coverage_analyze_sublinear reported fabricated data for Rust
Coverage now carries provenance (`estimated` / `measured` / `coverageMethod`),
uncollected branch and function metrics report `null` instead of 0 or 100, gaps
derived from an estimate carry low confidence and an `estimated` flag, and Rust
crates are measured with `cargo llvm-cov` where available. Test code (`tests/`,
`benches/`, brace-matched `#[cfg(test)]` blocks) is excluded from production
coverage, and line numbers can no longer exceed the file length. Security scans
now state their analysis depth so "zero findings" is not read as "verified clean".

Adversarial review (qe-court, cross-vendor) found and this commit fixes: an
anchoring bug that dropped `latest_parser.rs` as a test file; brace matching that
counted braces inside string literals and comments; `#[cfg(test)]` on a
non-module item swallowing the following production function; crate detection
that skipped instrumentation for subdirectory targets, over-claimed nonexistent
targets, and hijacked polyglot subtrees; and a Cobertura fallback that rounded a
measured 47% to 0%.

ADR-125 (on-disk agent routing overrides) and ADR-126 (measurement provenance)
document the decisions; ADR-115 gains an amendment on optional peers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:59:49 +00:00

84 lines
3.1 KiB
TypeScript

/**
* Packaging-surface guards (issue #565, ADR-115 amendment 2026-07-23).
*
* These assert facts about what a *consumer* of `npm install agentic-qe` ends up
* with. They are cheap manifest reads — no install, no network — and exist so a
* well-meaning "let's declare the peer again" edit fails in CI instead of on a
* user's machine three weeks later.
*/
import { describe, it, expect } from 'vitest';
import { readFileSync } from 'fs';
import { join } from 'path';
const repoRoot = join(__dirname, '..', '..', '..');
const pkg = JSON.parse(
readFileSync(join(repoRoot, 'package.json'), 'utf-8')
) as {
peerDependencies?: Record<string, string>;
peerDependenciesMeta?: Record<string, { optional?: boolean }>;
devDependencies?: Record<string, string>;
optionalDependencies?: Record<string, string>;
overrides?: Record<string, string>;
};
const lock = JSON.parse(
readFileSync(join(repoRoot, 'package-lock.json'), 'utf-8')
) as {
packages: Record<
string,
{ version?: string; dev?: boolean; peer?: boolean; optional?: boolean }
>;
};
describe('consumer dependency surface (#565)', () => {
it('does not declare @huggingface/transformers as a peer dependency', () => {
// npm >=7 auto-installs OPTIONAL peers — `optional: true` only suppresses
// the resolution error. Declaring this peer put onnxruntime-node ->
// adm-zip@0.5.x (GHSA-xcpc-8h2w-3j85) in every consumer's tree, and no
// resolvable version of that chain is unaffected.
expect(pkg.peerDependencies ?? {}).not.toHaveProperty(
'@huggingface/transformers'
);
expect(pkg.peerDependenciesMeta ?? {}).not.toHaveProperty(
'@huggingface/transformers'
);
});
it('keeps @huggingface/transformers available for our own dev/test runs', () => {
// Dropping the peer must not break the in-process embedding path locally.
expect(pkg.devDependencies ?? {}).toHaveProperty('@huggingface/transformers');
});
it('leaves the transformers/onnx/adm-zip chain dev-only in the lockfile', () => {
for (const path of [
'node_modules/@huggingface/transformers',
'node_modules/onnxruntime-node',
'node_modules/adm-zip',
]) {
const entry = lock.packages[path];
expect(entry, `${path} missing from lockfile`).toBeDefined();
expect(entry.dev, `${path} is reachable by consumers`).toBe(true);
}
});
it('resolves adm-zip to a version without GHSA-xcpc-8h2w-3j85', () => {
const version = lock.packages['node_modules/adm-zip']?.version ?? '0.0.0';
const [major, minor] = version.split('.').map(Number);
// Advisory range is <0.6.0.
expect(major > 0 || minor >= 6).toBe(true);
});
it('retains every declared optionalDependency in the lockfile', () => {
// Guards the known dependabot failure mode: a lock regenerated on one
// platform prunes the other platforms' @ruvector native binaries, and
// `npm ci` then fails with EUSAGE everywhere else.
for (const name of Object.keys(pkg.optionalDependencies ?? {})) {
expect(
lock.packages[`node_modules/${name}`],
`optionalDependency ${name} was pruned from the lockfile`
).toBeDefined();
}
});
});