No CI job ran the shell-docs vitest suite, and none of the prompt files were
in the docs workflow's paths filter, so the assertions these tests make were
verified only on a developer's machine. That is the same gap the retired
prose had: nothing measured it and nothing caught it rotting.
The job is scoped to five files. The whole shell-docs suite is not green on
main, so running it here would gate every prompt change on unrelated
failures.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The coverage ratchet added for OSS-1036 never ran in CI. No workflow runs the
shell-docs vitest suite: `test_integration-docs.yml` runs exactly one test
file, `test_unit-showcase.yml` covers harness and shell-dashboard only, and
`showcase_build_check.yml` matches `showcase/**` but only builds images. On top
of that, neither path filter listed the snippet files the gate reads, so a new
`frontend-tools-setup.mdx` triggered nothing.
Add a scoped job and widen both filters. The job is deliberately limited to the
two setup-concept test files, because the full shell-docs suite is not green on
main and would gate every snippet change on unrelated failures.
The harness symlink step is the same one showcase/shell-docs/Dockerfile uses:
`generate-registry.ts` imports the catalog fold out of the harness tree, which
resolves `js-yaml` by walking up from `showcase/harness/`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four workflow steps installed CLI tools ad-hoc via `npm install -g`, which
zizmor's `adhoc-packages` audit flags (install outside a lockfile). Replace
each with a lockfile-managed or pinned-action install, preserving behavior:
- aimock (test_integration-docs, test_e2e-showcase-on-demand): invoke the
workspace-pinned @copilotkit/aimock `llmock` bin from the frozen lockfile
(already a dep of @copilotkit/showcase-scripts) instead of `npm install -g`.
Kept lockfile-devDep rather than the CopilotKit/aimock composite action:
the action wraps the newer config-only `aimock` CLI and can't do the
multi-`--fixtures` / `--validate-on-load` / `/__aimock/health` invocation
these jobs need.
- claude-code (social_copy-generator): pin @anthropic-ai/claude-code as a root
devDependency, install from the frozen lockfile, invoke via its documented
cli-wrapper.cjs entrypoint. Kept lockfile-devDep rather than
anthropics/claude-code-action: the job uses claude as a scripted `-p` CLI,
not PR/issue automation.
- oxfmt (static_quality): already a root devDependency; install from the frozen
lockfile and put node_modules/.bin on PATH instead of `npm install -g`.
- ruff (static_quality): switch `pipx install` to the pinned official
astral-sh/ruff-action@278981a (v4.1.0) with the same 0.15.13 version.
zizmor --min-severity low --config .github/zizmor.yml .github/workflows:
before: exit 12, 4 adhoc-packages findings
after: exit 0, 0 adhoc-packages findings, 0 unpinned-uses (no findings)
Comprehensive CI/CD security hardening pass over all 33 workflows.
Action pinning
- Every `uses:` is now pinned to a 40-char commit SHA with a `# vX.Y.Z`
comment alongside (167 occurrences resolved). Tag-style refs like `@v4`
are mutable and have been used in past supply-chain attacks (e.g.
tj-actions/changed-files in March 2025) to repoint widely-used actions
to malicious commits.
- Removed redundant `version: "10.13.1"` hardcodes from `pnpm/action-setup`
call sites so the action inherits from package.json `packageManager`
(one source of truth).
Automated maintenance
- Added `.github/dependabot.yml` for the `github-actions` ecosystem so
SHA pins stay current. Without this, pins go stale fast and new
upstream advisories never reach us. Minor/patch bumps are grouped;
major bumps stay separate so they get a real review.
Static analysis
- Added `.github/zizmor.yml` configuration and
`.github/workflows/security_zizmor.yml` (blocking on PR, runs on push
to main, weekly schedule for advisory drift). zizmor catches the
well-known classes of Actions footguns: template injection from
untrusted input, dangerous triggers, unpinned uses, excessive token
scopes, secret exfil patterns.
- All 28 high-severity and 54 medium-severity findings from the baseline
scan are remediated. Each suppression in zizmor.yml carries a
per-finding justification comment so future maintainers can audit the
trust assumption.
Workflow hardening (from zizmor + manual audit)
- Added `persist-credentials: false` to every `actions/checkout` except
the 7 workflows that legitimately push back to the repo via the
workflow token (release tagging, auto-formatting, docs-sync, registry
updates). Each retained credential persistence carries a
`persist-credentials required: ...` comment explaining the call site.
- Routed every attacker-controllable expansion (`github.head_ref`,
`github.event.pull_request.head.repo.full_name`, `inputs.*`,
step outputs) through `env:` and referenced as quoted shell variables.
Eliminates 17 template-injection vectors in fork-PR-reachable
workflows.
- Added per-job `permissions:` blocks across 14 workflows; demoted
broad workflow-level `id-token: write` to the specific Depot-runner
jobs that need it; narrowed `pull-requests: write` /
`actions: write` to the jobs that actually call those APIs.
Audit-driven fixes
- `publish-release.yml` build job: dropped `token:` and added
`persist-credentials: false`. The subsequent `Upload workspace` step
was packing `.git/config` (with the persisted GITHUB_TOKEN) into a
1-day-retention artifact downloadable by anyone with `actions:read`.
- `auto_merge_showcases.yml`: team-membership check now authorizes on
the PR AUTHOR (`pull_request.user.login`), never `context.actor` —
the actor is whoever triggered the latest event, so a team member
synchronizing or reopening an outsider's PR would otherwise
green-light auto-merge of code they didn't author.
- `static_quality.yml`: pinned ruff to a specific version so a
compromised release can't land on the next PR run with the
persisted-credentials write token in the format job.
- `showcase_capture-previews.yml`: switched the args-string construction
to a bash array so a slug or demo value containing whitespace or shell
metacharacters stays a single argument rather than being re-tokenized
by the shell.
Benjamin Taylor