proffesor-for-testing__agentic-qe/tests/fixtures/pr-body/valid.md at benchmarks

skills/proffesor-for-testing__agentic-qe

mirror of https://github.com/proffesor-for-testing/agentic-qe.git synced 2026-09-19 08:45:47 +08:00

Files

T

Dragan Spiridonov 70713b0b45 feat(pr-template-check): CI enforcement for the #401 failure-modes checkbox

Replaces the honor-system PR template checkbox with a parser that fails
any PR whose body either (a) unchecks or deletes the load-bearing
failure-modes checkbox, or (b) contains dismissal phrases like
"I believe it's unlikely" without a same-paragraph tracking issue
reference. Both rules hard-fail.

Covers the 3 phrases explicitly named in #408 plus 8 additional
dismissal patterns that shipped or could ship the same class of
reasoning that caused the v3.9.1–v3.9.4 init regression series.

- scripts/check-pr-body.cjs: standalone Node parser, ESM-safe via .cjs,
  zero dependencies, exported for self-test. Skips the template's own
  checkbox paragraph when scanning for forbidden phrases — the stock
  instruction text quotes the phrase as a teaching example and would
  otherwise trip the linter on every clean PR.
- .github/workflows/pr-template-check.yml: two jobs. lint-pr-body runs
  the parser against the actual PR body (dependabot[bot] allowlisted).
  self-test runs the parser against 11 fixtures and asserts both
  exit code and a content-specific substring in the output, so a
  parser regression fails loudly instead of silently passing bad PRs.
- tests/fixtures/pr-body/: 3 valid fixtures (clean, tracked-phrase,
  tracked-edge-case), 6 invalid fixtures (unchecked, missing section,
  forbidden phrase, multi-violation, highly-unlikely, can't-see-how),
  2 historical fixtures (the actual v3.9.3 PR body which fails for
  missing section, plus a synthetic body quoting #401's postmortem
  reference which fails for forbidden phrase).

Rollout: workflow is intentionally not required at merge time on
initial landing. Maintainer should promote it to a required status
check in branch protection once it has been observed to behave
correctly on live PRs — see the header comment in
.github/workflows/pr-template-check.yml.

Verified locally with the same self-test logic the workflow will run:
11/11 fixtures pass with content-aware assertions, and a mutation
test (breaking the checkbox-state logic) correctly causes the valid
fixture to fail, proving the self-test would catch a parser regression.

Closes #408

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-08 08:56:09 +00:00

1.1 KiB

Raw Permalink Blame History

Summary

Adds a new retry helper for flaky network calls in the fetcher layer. No behavior change for non-error paths.

Verification

Unit tests cover retry-on-500, retry-on-network-error, and the max-retry cap
Ran npm test src/fetcher — 14/14 pass
Hand-verified the retry count via debug logs on a killed-then-restarted server

Failure modes

Upstream returns 500 repeatedly → tested (max-retry cap + final error surface)
Network disconnect mid-request → tested (the retry loop picks it up on next attempt)
Upstream returns 200 with malformed JSON → not a retry path; parse errors bypass the helper

Required check (issue #401)

Every failure mode mentioned in this PR description has either (a) a test that exercises it, or (b) a linked tracking issue. "Unlikely" is not an acceptable substitute. If you wrote "I don't think this can happen but...", that sentence is a failure mode and needs a test or an issue link.

Optional context

Linked issues: #
Trust tier change (if any): none
Affects published API or CLI surface: no
Touches the init flow / npm-publish.yml / tests/fixtures/init-corpus/: no

1.1 KiB Raw Permalink Blame History