Pranay Prakash 54ba1888cd fix: compare benchmarks against PR base branch instead of main (#560) ...

* fix: compare benchmarks against PR base branch instead of main

- Use github.event.pull_request.base.ref instead of hardcoded main
- Remove search_artifacts: true to ensure most recent baseline is used
- For stacked PRs, this compares against the parent PR's baseline

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: group failed e2e tests by category and app in summary

Instead of listing each failed test as a separate item, group them by:
1. Category (world): e.g., "Community Worlds", "Vercel Production"
2. App (framework): e.g., "mongodb", "turso", "nextjs-turbopack"

This makes the summary much more readable when there are many failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: ensure local E2E tests always produce JSON output

- Add 'fastify' to app detection list in aggregate-e2e-results.js
- Change && to ; so e2e tests run even if dev.test.ts fails
- This ensures local-dev, local-prod, and local-postgres categories
  appear in the E2E summary comment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: ensure local E2E tests always produce JSON output

- Add 'fastify' to app detection list in aggregate-e2e-results.js
- Change && to ; so e2e tests run even if dev.test.ts fails
- This ensures local-dev, local-prod, and local-postgres categories
  appear in the E2E summary comment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: publish CI results to GitHub Pages for docs

- Add generate-docs-data.js script to create JSON summaries from CI artifacts
- Add publish-results job to tests.yml and benchmarks.yml workflows
- Update docs/lib/worlds-data.ts to fetch from GitHub Pages URLs
- Results published to https://vercel.github.io/workflow/ci/

This allows the docs worlds page to display actual test/benchmark
results without requiring a GITHUB_TOKEN.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: correct outputFile path for local E2E test artifacts

The --outputFile path was using ../../ which placed files outside the
repo because pnpm run test:e2e executes from workspace root, not from
the cd'd workbench directory. This prevented local-dev, local-prod, and
local-postgres test results from being uploaded as artifacts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: show green checkmark for skipped tests instead of warning

Skipped tests are intentional and shouldn't show as warnings in the
E2E test summary comments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: use collapsible sections in benchmark PR comment

Wrap each benchmark, stream benchmarks section, and summary tables in
<details> toggles to make the PR comment more compact and readable.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add Vercel observability links to benchmark PR comments

- Store runId in benchmark timing data
- Add project-slug to Vercel benchmark matrix
- Pass WORKFLOW_VERCEL_PROJECT_SLUG env var to benchmarks
- Store Vercel metadata (teamSlug, projectSlug, environment) in timing files
- Generate observability deep links for each Vercel world benchmark
- Show observability links below Production (Vercel) tables

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: use correct Vercel project slugs for observability links

- nextjs-turbopack → example-nextjs-workflow-turbopack
- nitro-v3 → workbench-nitro-workflow

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

2025-12-06 18:30:25 -08:00

8.5 KiB

Raw Permalink Blame History

View Raw