Commit Graph

6 Commits

Author SHA1 Message Date
Amaury Levé f6f2751abd Clarify evaluation progress PR label (#1082)
* Clarify evaluation progress label

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d35e7696-f69c-4898-ad3e-c75fb02b993d

* Distinguish queued and running evaluations

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d35e7696-f69c-4898-ad3e-c75fb02b993d

---------

Copilot-Session: d35e7696-f69c-4898-ad3e-c75fb02b993d
2026-08-27 14:41:18 +00:00
Jan Krivanek 15c336709e Fold Close Stale PRs into deterministic pr-triage stale sweep (#928)
Replace the agentic close-stale-prs.agent.md with a deterministic
stale-PR sweep hosted in pr-triage-batch.yml. The new stale-sweep job
runs .github/scripts/pr-stale-sweep.sh weekly (cron 17 4 * * 1) and on
manual dispatch (stale_sweep=true), warning about and closing PRs with no
non-bot activity for 30/37 days. Preserves the original policy (no-stale
and maestro exemptions, drafts included, non-bot activity timer) without
model calls or token cost.

Deletes close-stale-prs.agent.md and its compiled lock file, and updates
docs/design/pr-triage-workflows.md.

Fixes #915

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-07-23 16:04:13 +02:00
Jan Krivanek 0c0f6f0263 Fix PR-triage eval trigger: dispatch evaluation.yml instead of bot label (#746)
The triage worker added the 'evaluate-now' label via GITHUB_TOKEN, but label events emitted by GITHUB_TOKEN do not start workflows (GitHub's recursion guard), so evaluation.yml's pull_request_target:[labeled] entry point never fired for the bot (repro: PR #745). workflow_dispatch and repository_dispatch are the only token-initiated events exempt from that guard.

The worker now dispatches evaluation.yml directly via 'gh workflow run' with a pr_number input, routed through the existing gate job so the path is identical to /evaluate. A dispatched run's head_sha is the default branch (not the PR head), so idempotency now matches the deterministic run name 'Evaluate PR #<n> @ <sha7>'. The 'evaluate-now' label remains a valid human entry point. Worker granted actions:write for 'gh workflow run'.
2026-06-12 13:50:33 +00:00
Jan Krivanek 2604067914 Fix the auto-evaluation triggering in pr review agentic workflows (#728) 2026-06-05 13:36:52 +02:00
Jan Krivanek 01050ea83c Fix pr-malicious-scan: repeat-spam + integrity-filter blocks (#722)
* Fix malicious-scan repeat-spam + integrity-filter blocks

Root causes (observed on PR #237):
1. Agent's emitted add_comment body did not include the HTML marker line
   (<!-- pr-malicious-scan:fingerprint=... -->), so both the orchestrator's
   pre-dispatch check and the agent's own Step 1 idempotency lookup failed
   to find a prior scan for the same head SHA. Result: hourly re-dispatch.
2. The github MCP tools (pull_request_read, list_pull_requests,
   search_pull_requests) are blocked by the gh-aw integrity filter on PRs
   from non-approved authors -- exactly the population this scanner targets.
   Result: 'Integrity filter blocked N items' notes in every comment.

Fixes:
- pr-malicious-scan.agent.md: drop the github MCP toolset, add 'gh' to the
  bash allowlist, and instruct the agent to use 'gh api' for all PR data
  reads (PAT-authenticated, not subject to the integrity filter).
- Strengthen Step 5: the HTML marker MUST be the first line of the comment
  body. Add a defense-in-depth note that the orchestrator also accepts the
  visible-body sentinel.
- pr-triage-batch.yml + pr-triage-act.sh: match prior scans by EITHER the
  HTML marker OR the visible-body sentinel ('Automated diff scan' + backticked
  sha7), so a missing marker on a previously-emitted comment no longer
  triggers re-dispatch.

Workflow disabled remotely while this lands.

* Orchestrator-only dispatch + integrity-filter opt-out

Replace the per-push pull_request_target trigger and the gh-api workaround
with the documented gh-aw pattern:

- pr-malicious-scan.agent.md: drop pull_request_target; trigger only via
  workflow_dispatch from the orchestrator. Restore the github MCP toolset
  with min-integrity: none (the documented level for spam-detection /
  analytics workflows; safe-outputs still gates every mutation). Drop the
  'gh' bash hack and visible-body sentinel requirements.

- pr-triage-batch.yml: orchestrator now posts a deterministic
  '<!-- pr-malicious-scan:dispatched=SHORT --> ' comment BEFORE calling
  gh workflow run. That comment is the source of truth for 'a scan has
  been initiated for this head SHA' and survives every agent-side failure
  mode (PAT outage, integrity block, dropped HTML marker). Dedup matches
  either that orchestrator marker OR the agent's own fingerprint marker.

- pr-triage-act.sh: drop the visible-body-sentinel fallback; match the
  orchestrator dispatched marker plus the agent fingerprint marker.

Validated: gh aw compile clean; bash -n clean for both worker script and
orchestrator embedded script; markdownlint clean; dedup query and POST
api tested live against PR #713.
2026-06-04 12:30:55 +02:00
Jan Krivanek d9e4a5e113 PR triage workflows: orchestrator, worker, and evaluate-now label (#716)
* PR triage workflows: orchestrator, worker, and evaluate-now label

Implements docs/design/pr-triage-workflows.md:
- pr-triage-batch.yml: hourly orchestrator that classifies open PRs
- pr-triage.yml + pr-triage-act.sh: per-PR worker (state recompute,
  label reconciliation, eval-trigger, ping comments with cool-down)
- evaluation.yml: gate job now also handles pull_request_target [labeled]
  with the evaluate-now label as a second entry point alongside /evaluate

* Add temporary push triggers for testing pr-triage workflows

* test: live-run pr-triage worker once

* test: re-run worker for cool-down check

* fix: age gate uses created_at and applies only before first ping

* Remove temporary test triggers and inline test marker

* Add pr-malicious-scan agent workflow; replace design doc with brief overview

- New: .github/workflows/pr-malicious-scan.agent.md + compiled .lock.yml.
  Static diff scanner for external (non-trusted) PR contributors. Triggers
  on pull_request_target [opened/synchronize/reopened] and workflow_dispatch.
  Surfaces findings as code-scanning alerts plus a single maintainer-ping
  comment per head SHA when high-severity / workflow-tamper / supply-chain
  hits. Never executes PR head code.
- docs/design/pr-triage-workflows.md replaced with a brief overview + diagram.
  The full implementation plan is kept locally as
  docs/design/pr-triage-workflows-plan.md (gitignored).
- pr-triage-batch.yml's existing dispatch-scanner branch now resolves to
  the new scanner; orchestrator unchanged.

* Fix markdownlint MD038 (pipe inside code span) in malicious-scan agent
2026-06-03 20:01:19 +00:00