Commit Graph

5 Commits

Author SHA1 Message Date
Nathan Colosimo 5c4eef0a97 chore: upgrade to pnpm 11.24.0 (#3901)
* chore: upgrade to pnpm 12

* fix: support pnpm 12 in CI

* fix: enable pnpm 12 on Vercel

* refactor: simplify pnpm 12 setup

* refactor: target pnpm 11.24.0

* refactor: let pnpm setup own CI installs

* refactor: limit workspace Node versions

* fix: complete pnpm 11 CI migration
2026-09-01 12:50:26 -07:00
Karthik Kalyan ee61817865 ci: pin third-party GitHub Actions to commit SHAs (#2050)
Major-version refs like `@v2`/`@v5` resolve to mutable refs on the
upstream repos — sometimes a tag, sometimes a branch (e.g. marocchino
keeps `v1`/`v2`/`v3` as branches), and dawidd6 force-pushes the bare
`v6` tag forward outside of releases. A compromised maintainer account
could push new code that our CI picks up on the next run with
GITHUB_TOKEN (or, for changesets/action, NPM_TOKEN) in hand.

Pin all third-party `uses:` references to full commit SHAs with a
trailing version comment so the upstream release is still visible to
reviewers. Dependabot/Renovate can keep these fresh going forward.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 00:13:40 +00:00
Nathan Rajlich 28dc089eef ci: fix VERCEL_WORKFLOW_SERVER_* env var ternary on main (#1859)
* ci: fix VERCEL_WORKFLOW_SERVER_* ternary so main actually unsets them

In GitHub Actions expressions, '' is falsy, so the original
`cond && '' || secrets.X` pattern always fell through to the secret
regardless of branch. The result was that pushes to main were sending
preview workflow-server values to production, causing 'invalid_url'
errors on `x-vercel-workflow-api-url` across all e2e jobs.

Flip the condition so the secret sits in the truthy branch and ||
correctly selects '' on main.

* ci: skip pnpm cache in matrix-generation jobs

The Get Test Matrix and Get Community Worlds Matrix jobs only run a
small Node script to emit a JSON matrix; they never run `pnpm install`.
With `cache: 'pnpm'` set on actions/setup-node, the post-job cache save
step fails with 'Path Validation Error' because the pnpm store path was
never created, marking the whole job as failed.

Add a cache-pnpm input to setup-workflow-dev (default true) and opt out
in the two matrix-generation jobs.
2026-04-28 10:26:04 -07:00
Nathan Rajlich 69af0c1374 ci: upgrade pnpm/action-setup to v5 and read version from package.json (#1785)
* ci: upgrade pnpm/action-setup to v6 and read version from package.json

Removes hardcoded pnpm version (10.14.0) from all workflows and instead
reads the version from the packageManager field in package.json, so CI
stays in sync with the version used locally.

* ci: update setup-workflow-dev composite action to use pnpm/action-setup@v6

Also removes the pnpm-version input since the action now reads the
version from package.json#packageManager.

* ci: downgrade pnpm/action-setup to v5

v6 installs pnpm 11 RC/beta, which has a regression
(pnpm/pnpm#11264, pnpm/action-setup#225/#227/#228) that causes
'ERR_PNPM_BROKEN_LOCKFILE: expected a single document in the stream'
when the project's packageManager pins a 10.x pnpm version. v5 is the
latest stable release before v6 and supports reading the version from
package.json#packageManager.
2026-04-17 01:17:27 +00:00
Pranay Prakash 5dd15452cd Test and benchmark community worlds against e2e tests (#482)
* Add new github workflow

* Enable pull_request trigger for community worlds workflow

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add community worlds manifest and generation scripts

- Add community-worlds.json manifest as single source of truth
- Add scripts/generate-community-worlds-workflow.mjs to generate CI workflow
- Add scripts/generate-community-worlds-docs.mjs to generate docs section
- Update aggregate-benchmarks.js to load community worlds dynamically
- Add pnpm generate:community-worlds script
- Update docs/deploying/world/index.mdx with community worlds

The manifest-based approach allows:
- E2E tests to be auto-generated from the manifest
- Benchmark aggregation to include community worlds
- Docs to stay in sync with tested worlds

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix YAML syntax error - quote strings starting with @

The @ symbol has special meaning in YAML, so package names like
@workflow-worlds/turso need to be quoted.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix Redis health-cmd quoting, remove unpublished starter world

- Quote health-cmd when it contains spaces (fixes Docker arg parsing)
- Remove @workflow-worlds/starter as it's not published to npm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add benchmarks and summary job for community worlds

- Add build job to share artifacts between benchmark jobs
- Add benchmark jobs for Turso, MongoDB, and Redis worlds
- Update summary job to show both E2E and benchmark status matrix
- Add left border/indent to sidebar child items for visual hierarchy
- Update workflow generator to support benchmark generation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Reuse build artifacts for E2E tests

E2E jobs now depend on the shared build job and download
artifacts instead of rebuilding packages from scratch.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add Worlds Ecosystem dashboard to docs

- Create worlds-manifest.json with official and community worlds
- Add aggregate-worlds-data.mjs script for processing E2E and benchmark results
- Create WorldsDashboard, WorldCard, and BenchmarkChart components
- Add /docs/worlds page showing compatibility status and performance
- Include sample data for development

The dashboard shows:
- E2E test progress per world (pass/fail/skip counts)
- Benchmark performance comparison across all worlds
- Filter by official vs community worlds

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix CI to output JSON test results and add Jazz world

- Update workflow generator to output JSON test results from vitest
- Upload E2E results as artifacts for parsing in summary job
- Summary job now shows actual pass/fail/skip counts per world
- Add Jazz world to worlds-manifest.json (requires external credentials)
- Add update-worlds-status.yml workflow to auto-update dashboard data
- Update TypeScript types to support null lastRun and metrics

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Refactor community worlds to use reusable workflows

Instead of a generated workflow file, integrate community world testing
directly into tests.yml and benchmarks.yml using reusable workflows.

- Add reusable workflows for E2E tests: e2e-community-world.yml (no services),
  e2e-community-world-mongodb.yml, e2e-community-world-redis.yml
- Add reusable workflows for benchmarks: benchmark-community-world.yml,
  benchmark-community-world-mongodb.yml, benchmark-community-world-redis.yml
- Update tests.yml to call reusable workflows for Turso, MongoDB, Redis
- Update benchmarks.yml to include community world benchmarks in summary
- Delete generated community-worlds.yml and generator script

This approach:
- Inherits proper Rust/SWC setup from the main workflows
- Keeps all CI in the established patterns
- Makes adding new community worlds straightforward

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix benchmark timing file naming for community worlds

Add WORKFLOW_BENCH_BACKEND env var support to bench.bench.ts so community
world benchmarks generate timing files with the correct backend suffix
(e.g., bench-timings-nextjs-turbopack-turso.json instead of -local.json).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add @workflow-worlds/starter to community worlds test matrix

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Unify worlds manifest and add dynamic GitHub API fetching

- Merge community-worlds.json into worlds-manifest.json with type field
- Add server-side data fetching from GitHub API for worlds dashboard
- Remove static worlds-status.json, fetch CI artifacts dynamically
- Update all references to use unified manifest format
- Remove obsolete community-worlds.yml and update-worlds-status.yml workflows

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Skip community worlds for non-nextjs-turbopack in benchmark summary

Community worlds only run against nextjs-turbopack, so hide the
"missing" rows for Express and Nitro frameworks in the benchmark
comparison tables.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix stream benchmark detection to check for actual TTFB data

The previous check `!== null` incorrectly returned true for undefined,
causing all benchmarks to show TTFB columns. Now explicitly checks
for a number type.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Hide Worlds Ecosystem page from sidebar

The page is still accessible via direct link at /docs/worlds but
won't appear in the navigation until it's been further iterated on.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix review comments: trailing newline and division by zero

- Add trailing newline when replacing Community Worlds section in docs
- Fix division by zero in WorldCard benchmark calculation when metrics is empty

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Consolidate community world workflows with service-type parameter

- Create setup-workflow-dev composite action for common setup steps
- Add service-type input to benchmark-community-world.yml and e2e-community-world.yml
- Use conditional job execution (if: inputs.service-type == 'mongodb') to handle different services
- Update benchmarks.yml and tests.yml to pass service-type parameter
- Delete redundant workflow files:
  - benchmark-community-world-mongodb.yml
  - benchmark-community-world-redis.yml
  - e2e-community-world-mongodb.yml
  - e2e-community-world-redis.yml

Reduces workflow files from 11 to 7 and eliminates ~500 lines of duplicated YAML.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Generate community world test matrix from worlds-manifest.json

Replace hardcoded community world jobs with dynamic matrix generation using
scripts/create-community-worlds-matrix.mjs. This allows adding/removing
community worlds by editing the manifest instead of multiple workflow files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Apply suggestion from @vercel[bot]

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

* Add Samples column and separate local/production benchmarks

- Add Samples column to all benchmark tables showing iteration count
- Separate benchmark results into Local Development and Production sections
- Add explanatory context for each section (localhost vs Vercel deployment)
- Add GitHub action step summaries to e2e community world tests
- Create aggregate-e2e-results.js script for parsing vitest JSON output

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Remove obsolete generate-community-worlds-docs script

The Worlds Ecosystem page now fetches from worlds-manifest.json at runtime,
making this script unnecessary. The npm script also referenced a non-existent
workflow generator script.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Maximize composite action usage and reorganize benchmark output

- Update setup-workflow-dev composite action with optional Rust, install-dependencies, and install-args inputs
- Update tests.yml to use composite action in unit, e2e-vercel-prod, getTestMatrix, e2e-local-*, and getCommunityWorldsMatrix jobs
- Update benchmarks.yml to use composite action in build, benchmark-local, benchmark-postgres, benchmark-vercel, and getCommunityWorldsMatrix jobs
- Reorganize benchmark output to group by benchmark test with local/production tables within each benchmark
- Remove invalid $schema reference from worlds-manifest.json

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add beads stealth mode stuff (for personal claude memory - will remove stealth if people want)

* Add E2E test results PR comment summary

- Add pr-comment-start job to create/update PR comment when tests start
- Add artifact uploads to all e2e test jobs (vercel-prod, local-dev, local-prod, local-postgres, windows)
- Update e2e-community-world.yml with consistent artifact naming (e2e-community-*)
- Add summary job to aggregate all e2e results and update PR comment
- Extend aggregate-e2e-results.js with --mode aggregate for multi-job PR summary
- Group results by category (Vercel Production, Local Development, etc.)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add step summaries to all e2e test jobs

Add "Generate E2E summary" step to each individual e2e job:
- e2e-vercel-prod
- e2e-local-dev
- e2e-local-prod
- e2e-local-postgres
- e2e-windows

Each job now outputs pass/fail/skip counts to GITHUB_STEP_SUMMARY.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Consolidate community world workflows to single job

Replace 3 mutually exclusive jobs (e2e/e2e-mongodb/e2e-redis) with a single
job that starts services via docker run when needed. This eliminates the
skipped job entries that appear in the GitHub Actions UI.

- Use conditional docker run steps instead of services: block
- Add health check loops to wait for service readiness
- Add cleanup step to stop containers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix extractWorldId to handle community world artifact naming

Add handling for `e2e-results-community-{world}` pattern so community
world test results are properly extracted (e.g., `e2e-results-community-turso`
now correctly extracts `turso` instead of `community-turso`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
2025-12-02 15:12:44 -08:00