mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
Dragan Spiridonov
2988cbcc7c
H1: detectVibium reads stderr too, extracts semver
- Many Go CLIs (including some Vibium versions) write --version to stderr.
detectVibium used to read only stdout and silently report 'unknown',
polluting the init log with "vibium unknown (already installed)".
- Fix: read both streams, prefer stdout, extract bare semver via regex
so output is consistent regardless of where the binary printed it.
- Tests: 5 new regression tests covering stderr fallback, stdout
preference, verbose-format extraction, prerelease tags, fallback to
first whitespace token.
- Note: result.version is now bare semver (26.3.18) not v-prefixed.
Two pre-existing tests updated to match the new contract.
H2: instructions_in_html_comment dead-code regex
- TreeWalker(SHOW_COMMENT).textContent yields the inner text without
the <!-- --> delimiters, so the pattern that requires the literal
`<!--` prefix could never fire. Eval tc009 was passing only by
accident (ignore_previous_instructions fired on the same fixture).
- Fix: re-wrap each comment node in `<!-- ... -->` before adding to
the hidden bucket. Pattern can now actually match.
- Tests: 2 new regression tests asserting wrapped form fires AND
unwrapped form does NOT fire (the bug condition).
H4: e2e-flow-verifier frontmatter regression — INVALID FINDING
- Verified against `git show main:` — the skill always had minimal
frontmatter on main (just name + description + user-invocable: true)
and has no evals/, schemas/, or scripts/ directories. It is a
user-invocable runbook-style skill, not trust-tier-3 validated.
No metadata to restore. Marked closed-as-invalid.
H6: log-before-spawn for the 3-minute Vibium install
- spawnSync(npm install -g vibium) blocks the event loop for up to
3 minutes on a cold cache while Chrome for Testing downloads. The
user saw zero output and would Ctrl-C thinking init hung.
- Fix: pre-flight detect call (5s timeout) short-circuits when vibium
is already installed (the common case, no banner). When the install
is actually about to run, log a clear "this can take 1-3 minutes,
downloads Chrome for Testing lazily" message BEFORE the spawn.
H7: scoped scripts/package.json survives npm pack — VERIFIED
- `npm pack --dry-run` confirms both
.claude/skills/qe-browser/scripts/package.json (225B) AND
assets/skills/qe-browser/scripts/package.json (225B)
are included in the published tarball. The CommonJS override that
keeps the helper scripts loadable from an ESM project root will
survive `npm install` after publication. No fix needed.
H8: eval yaml uses unsupported runner fields
- The structured assertion fields I invented (exit_code, json_fields,
severity_at_least, candidate_count_at_least) are NOT understood by
src/validation/parallel-eval-runner.ts which expects must_contain
keyword matching. The yaml as written is unrunnable.
- Honest fix: relabel evals/qe-browser.yaml as `status: design-spec`
with a HONESTY NOTE explaining it's a specification, not a runnable
eval. Two paths to make it runnable documented (extend runner OR
rewrite as must_contain).
- Parallel artifact: scripts/smoke-test.sh — a runnable bash smoke
test against the same pinned httpbin.org fixtures. THIS is the
script that gates PR-reopen per ADR-091 Phase 3.
Bonus from B2/H5 already in Phase 1:
- assert.js now uses ?? instead of || for the `expected` field
Verification:
- 50/50 qe-browser tests pass (was 36 in initial PR; +14 regressions)
- 392/392 init tests pass (was 387; +5 from H1)
- npm run build: clean
- npm pack --dry-run: scoped package.json confirmed in tarball
- eslint on all touched source files: clean
- bash -n smoke-test.sh: syntax OK
Phase 3 (real Vibium smoke test) still gating PR reopen per ADR-091.
Branch intentionally not pushed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>