* chore: upgrade gh-aw runtime * fix: paginate stale pull request cleanup Upgrade the generated agentic workflow assets and ensure stale PR discovery includes every result page and draft pull requests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 968a22c2-327f-4d26-8f86-1c59bcd323ea * Improve dotnet-test eval coverage and efficiency Address remaining high-confidence items from #899 by bounding the code-testing pipeline and adding eval coverage for grade-tests and find-untested-sources. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e430fee9-d3df-4ef5-85a4-745ae4b17046 * fix: address agentic workflow review Pin the Copilot setup checkout action and include the cutoff date in stale PR search results. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 968a22c2-327f-4d26-8f86-1c59bcd323ea * Improve test migration skill guidance Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5e19d263-02a6-45b1-9cfb-424fa4d10863 * fix: add fixture namespace imports Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a464e6e4-3e45-41fe-b17d-887c8cb8a448 * test: assert GetOrderById grade Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a464e6e4-3e45-41fe-b17d-887c8cb8a448 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
12 KiB
PAT Pool
Selects a random Copilot PAT from a numbered pool of secrets. This addresses limitations that arise from having a single PAT shared across all agentic workflows, such as rate-limiting.
This is a stop-gap workaround. As soon as organization/enterprise billing is available to the dotnet org, this approach will be removed from our workflows.
Repository Onboarding
To use Agentic Workflows in a dotnet org repository:
- Follow the instructions for Configuring Your Repository | Agentic Authoring | GitHub Agentic Workflows. Use
gh awv0.71.5 or newer, which supports the agent job dependencies required for this implementation. - Copy the
pat_pool.mdandpat_pool.README.mdfiles into the repository under.github/workflows/shared. - Merge those additions into the repository and then follow the instructions for the PAT Creation and Usage below.
Install or upgrade the gh aw CLI and check the version
gh extension install github/gh-aw --force
gh aw --version
Environment
Create an environment for the agentic workflows:
- Configuring these settings requires repo admin permission
- https://github.com/dotnet/{repo}/settings/environments
- Recommended Name: copilot-pat-pool
- Recommended Deployment branches and tags: Protected branches only
This environment is used for all agentic workflows, restricting agentic workflows to the repo's protected branches and preventing the workflows from accessing secrets defined for other environments.
PAT Management
Team members provide PATs into the pool with secret names matching the pattern of {pool-name}_{0-9}, such as COPILOT_PAT_0.
Use this link to prefill the PAT creation form with the required settings:
- Resource owner is your user account, not an organization.
- Copilot Requests (Read) must be the only permission granted.
- 8-day expiration must be used, which enforces a weekly renewal.
- Repository access set to Public repositories only.
The Token Name does not need to match the secret name and is only visible to the owner of the PAT. It's recommended to use a token name indicating the PAT is used for dotnet org agentic workflows. The Description is also only used for your own reference.
Team members providing PATs for workflows should set weekly recurring reminders to regenerate and update their PATs in the PAT pool. With an 8-day expiration, renewal can be done on the same day each week.
PAT Pool Secrets
For a PAT pool that is specific to an environment, PATs can be added to repositories as Environment Secrets for the environment created above. This requires repo admin permission.
- Settings >
- Environments >
- copilot-pat-pool (or other environment name) >
- Add environment secret (or edit your existing secret)
- Enter your secret name of
COPILOT_PAT_{0-9}and paste in your PAT
- Enter your secret name of
- Add environment secret (or edit your existing secret)
- copilot-pat-pool (or other environment name) >
- Environments >
This can also be accomplished using the gh CLI, specifying the repo and environment arguments.
# Register the PAT secret. This will prompt for you to paste the PAT.
gh secret set "<pool_name>_<0-9>" --repo <org>/<repo> --env "copilot-pat-pool"
It's also helpful to record who owns each PAT within the pool. To capture which team member is associated with each PAT, a <pool_name>_<0-9>_<username> "sidecar secret" can be added alongside the PAT secret to make the username for the PAT pool entry visible. This sidecar secret must have a non-empty value, but it's never consumed, so any value is sufficient.
# Record a sidecar secret that presents who owns this PAT.
gh secret set "<pool_name>_<0-9>_<username>" --body "<username>" --repo <org>/<repo> --env "copilot-pat-pool"
Workflow Output Attribution
Team members' PATs are only used for the Copilot requests from within the agentic portion of the workflow. All outputs from the workflow use the github-actions[bot] account token. Issues, PRs, comments, and all other content generated by the workflow will be attributed to github-actions[bot]--not the team member's account or token.
Usage
The pat_pool.md workflow import defines a custom job with a pat_number output. Consuming workflows need two additions to their frontmatter to import this job and use the PAT number to override the COPILOT_GITHUB_TOKEN passed to the workflow's agent job.
# ###############################################################
# Select a PAT from the pool and override COPILOT_GITHUB_TOKEN.
# Run agentic jobs in an isolated `copilot-pat-pool` environment.
#
# When org-level billing is available, this will be removed.
# See `shared/pat_pool.README.md` for more information.
# ###############################################################
imports:
- uses: shared/pat_pool.md
with:
environment: copilot-pat-pool
environment: copilot-pat-pool
engine:
id: copilot
env:
COPILOT_GITHUB_TOKEN: ${{ case(needs.pat_pool.outputs.pat_number == '0', secrets.COPILOT_PAT_0, needs.pat_pool.outputs.pat_number == '1', secrets.COPILOT_PAT_1, needs.pat_pool.outputs.pat_number == '2', secrets.COPILOT_PAT_2, needs.pat_pool.outputs.pat_number == '3', secrets.COPILOT_PAT_3, needs.pat_pool.outputs.pat_number == '4', secrets.COPILOT_PAT_4, needs.pat_pool.outputs.pat_number == '5', secrets.COPILOT_PAT_5, needs.pat_pool.outputs.pat_number == '6', secrets.COPILOT_PAT_6, needs.pat_pool.outputs.pat_number == '7', secrets.COPILOT_PAT_7, needs.pat_pool.outputs.pat_number == '8', secrets.COPILOT_PAT_8, needs.pat_pool.outputs.pat_number == '9', secrets.COPILOT_PAT_9, 'NO COPILOT PAT AVAILABLE') }}
The COPILOT_GITHUB_TOKEN expression can be collapsed onto a single line if desired. gh-aw compile automatically wires pat_pool into the activation and agent jobs' needs: graph because of the needs.pat_pool. references within the engine.env property.
gh aw compile <workflow-name> --schedule-seed <org>/<repo>
Specifying the environment
The environment must be specified both to the pat_pool.md import and to the containing workflow to ensure both jobs access the PAT pool from the same environment. The copilot-pat-pool environment name is recommended as the isolated environment for agentic workflows that use the PAT pool.
Customizing the pool
The import declares 10 optional inputs (COPILOT_PAT_0 through COPILOT_PAT_9), each defaulting to secrets.COPILOT_PAT_# of the matching number. To point a workflow at a different pool of repository secrets, use the parameterized uses/with form when importing and pass the substitute secrets as the COPILOT_PAT_# inputs:
imports:
- uses: shared/pat_pool.md
with:
COPILOT_PAT_0: ${{ secrets.MY_TEAM_PAT_0 }}
COPILOT_PAT_1: ${{ secrets.MY_TEAM_PAT_1 }}
# Unspecified inputs default to `secrets.COPILOT_PAT_#` lookups
The secrets passed via with: must match the secrets referenced in the consuming workflow's case expression that overrides COPILOT_GITHUB_TOKEN--both sides need to agree on which secret backs each COPILOT_PAT_# slot. Update the case expression accordingly:
engine:
id: copilot
env:
COPILOT_GITHUB_TOKEN: ${{ case(needs.pat_pool.outputs.pat_number == '0', secrets.MY_TEAM_PAT_0, needs.pat_pool.outputs.pat_number == '1', secrets.MY_TEAM_PAT_1, ..., 'NO COPILOT PAT AVAILABLE') }}
This approach aligns with GitHub's documented guidance for passing secrets between workflows, where the pat_pool job returns a PAT number and the case statement acts as a secret store to look the PAT secret up based on the selected number.
Design / Security
There are several details of this implementation that keep our workflows and repositories safe.
- Secrets adhere to existing trust boundaries. The pool of PAT secrets is
provided to a dedicated step within the
pat_pooljob. That job runs afterpre_activationand contains only the trusted checkout and action steps--no untrusted context or input is within scope. Theselect-pat-numberaction only references the secret values to determine which are non-empty, filtering the secret numbers to those with values. - The
pat_pooljob emits only a number, never a secret. Its sole output,pat_number, is the 0-9 index of the selected PAT (or empty when the pool is empty). The actual secret materializes only later, in the activation job'sengine.envmapping, where thecase()expression resolves the number to the matching secret. This follows GitHub's guidance for passing secrets between jobs or workflows, with thecasestatement acting as a very simple secret store. - The
select-pat-numberaction does not require any permissions. It reads only theCOPILOT_PAT_#environment variables passed to it and writes only toGITHUB_OUTPUT. The job that hosts it setspermissions:to the workflow defaults (no elevated scopes). - The implementation uses supported Agentic Workflow extensibility hooks.
Defining a custom job inside an imported workflow file is
supported by
gh aw compile. gh-aw automatically wirespat_poolinto the activation job'sneeds:graph based on theneeds.pat_pool.outputs.pat_numberreferences inengine.env. The secret override capability supplies theCOPILOT_GITHUB_TOKENvalue viaengine.envrather than the default secret of the same name.
Each of the references below contributed to the design and implementation to ensure a secure and reliable design.
Known Issues
The pat_pool import integration requires that the workflow's compilation results in a pre_activation job. If nothing in your workflow definition produces a pre_activation job, a compilation error will be received.
✗ Failed workflows:
✗ <workflow-name>.md
.github\workflows\<workflow-name>.md:1:1: error: failed to generate YAML: failed to build and validate jobs: job dependency validation failed: job 'pat_pool' depends on non-existent job 'pre_activation'
To work around this, add on.permissions: {} (or another on.* setting that forces pre_activation, such as skip-if-no-match) to your workflow, which forces a no-op pre_activation job to be generated.
on:
permissions: {}
See: Activation 'needs' does not incorporate jobs in engine.env expressions (github/gh-aw#30790)
References
- Agentic Workflows CLI Extension
- Agentic Authoring
- Authentication
- Agentic Workflow Imports
- Custom Steps
- Custom Jobs
- Job Outputs
- Engine Configuration
- Engine Environment Variables
- Update agentic engine token handling to use user-provided secrets (github/gh-aw#18017)
- Case Function in Workflow Expressions
- Passing a secret between jobs or workflows