* Clarify evaluation progress label Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: d35e7696-f69c-4898-ad3e-c75fb02b993d * Distinguish queued and running evaluations Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: d35e7696-f69c-4898-ad3e-c75fb02b993d --------- Copilot-Session: d35e7696-f69c-4898-ad3e-c75fb02b993d
7.2 KiB
PR Triage Workflows
Three GitHub Actions workflows keep open PRs moving without manual nudging:
pr-triage-batch.yml— hourly orchestrator (cron17 * * * *). Enumerates open non-draft PRs, computes a deterministic state for each, and dispatches the per-PR worker (or the malicious-code scanner). No labels, no model calls; the only comment it posts is a one-time idempotency marker when it dispatches the malicious-code scanner. Also hosts the deterministic weekly stale-PR sweep (stale-sweepjob, cron17 4 * * 1) — see Stale-PR sweep.pr-triage.yml— per-PR worker (workflow_dispatch). Re-validates the PR's state, reconciles a singlepr-state/*label, and performs at most one of: trigger evaluation (by dispatchingevaluation.yml), ping the author, or ping maintainers. Cool-down (default 4 days) is enforced via marker comments.pr-malicious-scan.agent.md— per-PR malicious-code scanner (gh-aw). Static diff review for untrusted contributors. Reports findings as code-scanning alerts and an optional comment; never executes PR head code.
Architecture
flowchart TD
Cron["cron: every hour"] --> Batch["pr-triage-batch.yml<br/>(orchestrator)"]
WCron["cron: weekly (Mon)"] --> Sweep["pr-triage-batch.yml<br/>(stale-sweep job)"]
Batch -->|workflow_dispatch| Worker["pr-triage.yml<br/>(per-PR worker)"]
Batch -->|workflow_dispatch| Scan["pr-malicious-scan.agent.lock.yml<br/>(per-PR scanner)"]
Worker -->|workflow_dispatch: pr_number| Eval["evaluation.yml<br/>(existing)"]
Worker -->|adds pr-state/* label| PR[("PR")]
Worker -->|posts ping comment| PR
Scan -->|code-scanning alert + comment| PR
Sweep -->|warn / close stale| PR
PR -.->|human adds label: evaluate-now| Eval
Entry points into evaluation.yml
Four entry points feed the gate job, all sharing a per-PR concurrency group
so overlapping triggers collapse to a single run. Each binds the run to one
specific reviewed commit (never the live branch head), so evaluation always
runs the exact commit the maintainer approved:
- The
/evaluate <sha>slash command (issue_comment) — humans. The conversation-comment payload carries no commit id, so an explicit SHA is required and must belong to the PR; a bare/evaluateonly posts guidance pointing to the review flow. /evaluateinside a submitted PR review (pull_request_review [submitted]) — humans; the recommended path. Bound toreview.commit_id(the exact commit reviewed), so no SHA needs to be typed.- The
evaluate-nowlabel (pull_request_target [labeled]) — humans. Thegatejob consumes (removes) the label so reapplying re-fires. Bound to the head SHA carried in the label event payload. workflow_dispatchwith apr_numberinput — the triage worker. The worker runs asgithub-actions[bot], and label events emitted byGITHUB_TOKENdo not start workflows (GitHub's recursion guard), so the bot cannot use entry point 3.workflow_dispatchis exempt from that guard, so the worker dispatchesevaluation.ymldirectly. A dispatched run checks out the default branch by default (github.shaismain's tip, not the PR head) and its metadata doesn't record the target PR, so the worker matches the run byevaluation.yml's run name (Evaluate PR #<n> @ <sha7>) for idempotency. The PR's head travels in thehead_shainput, and the gate resolves that short SHA to the exact commit (it does not re-read the live PR head).
State machine (worker)
Order of evaluation; first match wins:
| Order | Condition | State | Label | Action |
|---|---|---|---|---|
| 1 | draft, or mergeable_state == unknown |
skip |
— | none |
| 2 | non-bot && non-trusted && no malicious-scan marker on head | needs-malicious-scan |
— | dispatch scanner |
| 3 | CHANGES_REQUESTED || unresolved threads > 0 || mergeable_state == dirty |
needs-author-attention |
waiting-on-author |
author-ping |
| 4 | eval == success && APPROVED |
ready-for-merge |
ready-to-merge |
maintainer-ping/C |
| 5 | eval == success && REVIEW_REQUIRED/none |
ready-for-review |
waiting-on-review |
maintainer-ping/A |
| 6 | eval == success && other decision | in-review |
pr-state/in-review |
reconcile only |
| 7 | eval == pending && run exists for head | evals-in-progress |
pr-state/evals-in-progress |
reconcile only |
| 8 | otherwise | ready-for-eval |
pr-state/ready-for-eval |
eval-trigger |
Trusted = OWNER / MEMBER / COLLABORATOR. Bots are short-circuited as trusted.
Cool-down and idempotency
Each ping variant writes a hidden HTML marker into its comment. The worker fetches prior bot comments and:
- If a marker for the same variant exists within
COOLDOWN_DAYS(default 4), the new comment is suppressed. - A first-ping age gate (default 30 min after PR creation) prevents pings on freshly opened PRs; it is bypassed once any prior ping marker exists.
Marker shapes:
<!-- pr-triage:fingerprint=author-ping:{sha7}:{yyyy-mm-dd} --><!-- pr-triage:fingerprint=maintainer-ping/{A,B,C}:{sha7}:{yyyy-mm-dd} --><!-- pr-malicious-scan:fingerprint={sha7}:{yyyy-mm-dd} -->
Labels owned by these workflows
State labels (exactly one is reconciled at a time). Where the existing label
taxonomy already covered a state, the workflow reuses it rather than introducing
a duplicate pr-state/* name:
pr-state/ready-for-eval(existing — queued, missing, or failed evaluation)pr-state/evals-in-progress(new — evaluation pending for the current head)waiting-on-review(existing — reused forready-for-review)ready-to-merge(existing — reused forready-for-merge)waiting-on-author(existing — reused forneeds-author-attention)pr-state/in-review(new)
Triggers and opt-outs:
evaluate-now— applied to fire evaluation; removed by the gate after consumption.no-stale— opt-out of stale-PR closure (honored by thestale-sweepjob) and of author/maintainer pings in the worker.
Stale-PR sweep
pr-triage-batch.yml includes a deterministic stale-sweep job that replaces the
former agentic close-stale-prs.agent.md. It runs weekly (cron 17 4 * * 1) and
on manual workflow_dispatch with stale_sweep=true, and executes
.github/scripts/pr-stale-sweep.sh — no
model calls, no tokens.
Policy (unchanged from the agentic version):
- Considers every open PR, including drafts.
- "Last activity" is the most recent non-bot comment or review; if there is
none, it falls back to the PR's
created_at.updated_atand all[bot]activity are ignored so the bot's own warning never resets the timer. - created ≤ 30 days ago → skip (too new).
- 30 days < inactivity ≤ 37 days → post a stale warning (once; guarded by a
<!-- pr-triage:stale-warning -->marker). - inactivity > 37 days → close the PR with a closing comment.
- Exempt: the
no-stalelabel; authorsdotnet-maestro[bot]/dotnet-maestro.
Inputs (via workflow_dispatch): stale_sweep (run the sweep), dry_run (log
decisions without writing), stale_max (hard cap on warn+close writes, default 25).