mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
Dragan Spiridonov
05dcfe92dd
Addresses the v3.9.1–v3.9.4 init regression series by building the verification layer that #401's post-mortem identified as structurally missing. The load-bearing changes are the real-public-repo corpus, the --json contract, and the restructured error handling in phase 06 that stops lying about success when a non-critical phase fails. What landed ----------- * `aqe init --json` — structured InitResult output with schema version 1, stricter exit code semantics (exit 1 on any step.status='error', not just on critical failures). Stable contract for the gate and other CI consumers. See InitJsonOutput in src/cli/handlers/init-handler.ts. * Phase 06 no longer swallows errors as `{status:'skipped',entries:0}`. Exceptions now propagate to BasePhase.execute() and surface as real step errors in result.steps[]. Fixed a resource leak where kgService and memory handles were leaked on the old throw path — now always released via try/finally. * `tests/fixtures/init-corpus/` — 4 pinned real public repo fixtures: - tiny-ts (developit/mitt@b240473) - mid-ts (sindresorhus/p-queue@55d306b) - multi-lang-real (ruvnet/RuView@2a05378 — contains the exact 28,745 byte / 776 line ruview_live.py that triggered the v3.9.4 hang, sha256-pinned to detect content drift) - self-dogfood (agentic-qe HEAD) * 22 gate assertions per fixture (A1–A22) including per-step JSON error inspection (A10), snapshot-based KG threshold with tolerance (A12), skills/agents/MCP/CLAUDE.md/workers/config.yaml presence (A13–A20), subthreshold stall detection (A21), and a second init pass that exercises the delta-scan path of phase 06 — the actual surface that hung in v3.9.1 ruview, never reached on a first init (A22). * `.github/workflows/npm-publish.yml` — new pre-publish-gate job that runs the corpus against the freshly-built tarball before publish; new tests-on-tag-sha job that re-runs fast unit tests on release events (replaces a buggy gh-api conditional caught by devils-advocate review). * `.github/workflows/post-publish-canary.yml` — fires on real publishes only (not workflow_dispatch dry runs), polls npm CDN via `npm install --dry-run` (actual CDN edge, not just registry metadata), runs the corpus against the published package, opens P0 issue on failure via heredoc-based body. * Watchdog effectiveness unit test (tests/unit/init/phases/code-intelligence-watchdog.test.ts) — 4 cases proving the async-stall branches of runBoundedScan work and documenting the sync-block limitation that requires AQE_SKIP_CODE_INDEX or (future) worker isolation. * `.github/PULL_REQUEST_TEMPLATE.md` — single load-bearing checkbox: every PR failure mode must have a test or tracking issue. Honor-system today; CI enforcement tracked in #408. * AQE_SKIP_CODE_INDEX promoted from "temporary escape hatch" to permanent supported flag in docs, CLI help, and source comments. Verification ------------ * `npm run typecheck` — clean * `npx eslint` on all touched files — clean (plus fixed two pre-existing unused-var warnings in init-handler.ts) * `aqe security --sast` on all touched source + workflows — 0 findings * `tests/unit/init/` — 377/377 pass * `tests/unit/init/phases/code-intelligence-watchdog.test.ts` — 4/4 pass * Local gate against `npm pack`'d tarball — 4/4 fixtures pass all 22 assertions including double-init delta path and sha256 regression marker Failure paths independently verified (F1–F5 injection tests): * F1 wrong sha256 → A3 fires, exit 2, actionable diagnostic * F2 absurd expectedKgEntries → A12 fires, exit 2, shows actual vs expected * F3 expectedElapsedSec=0 → A21 fires on slow fixture, A22 fires on double-init — same `(( elapsed > soft_cap ))` math proven in both paths * F4 injected throw in runCodeIntelligenceScan → reproduces the exact v3.9.3 failure shape (`success:true` at orchestrator level, phase 06 step.status='error'). A7 catches via strict exit code, A10 catches as defense-in-depth via jq inspection. Both layers verified. * F5 expectedSkillsInstalled=999 → A13 fires with OR-logic showing json=84, disk=84 vs expected 999 Deferred with tracking issues ----------------------------- * Phase 06 worker_threads isolation — #407 * PR template CI enforcement — #408 * VERIFICATION.md + release-notes matrix — #409 * Chaos workflow for pathological inputs — #410 * Mirror init-corpus tarballs to GitHub Releases — #411 What is NOT verified -------------------- * The CI workflows have never actually executed in a real Actions run. YAML syntactically validated via js-yaml, job dependency graph traced, but the first real execution is the first push of this branch. * The canary on a real publish is unexecuted. First execution is the first real release after merge. Refs: #401, #407, #408, #409, #410, #411 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1.3 KiB
1.3 KiB
Summary
Verification
Failure modes
Required check (issue #401)
- Every failure mode mentioned in this PR description has either (a) a test that exercises it, or (b) a linked tracking issue. "Unlikely" is not an acceptable substitute. If you wrote "I don't think this can happen but...", that sentence is a failure mode and needs a test or an issue link.
Optional context
- Linked issues: #
- Trust tier change (if any): tier N → tier M (because ...)
- Affects published API or CLI surface: yes / no
- Touches the init flow /
npm-publish.yml/tests/fixtures/init-corpus/: yes / no- If yes: did you run
./tests/fixtures/init-corpus/run-gate.shlocally? yes / no / not applicable
- If yes: did you run