* Refactor e2e tests for errors
* Improve step error tests to check workflow return value
Step error workflows now catch the error and return message/stack,
making assertions cleaner. Tests verify both:
- Workflow return value (caught error message)
- CLI step result (original stack with function names)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add assertion for stack trace in caught step error
With the fix in step.ts that propagates original stack traces,
we can now verify that caught step errors include function names
directly in the workflow return value (not just via CLI).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* logging
* Fix: When a step handler is re-invoked after max retries are exhausted, the step_failed event doesn't include a stack trace, breaking stack trace propagation for this edge case.
This commit fixes the issue reported at packages/core/src/runtime/step-handler.ts:127-146
## Stack trace loss in step_failed event when max retries are exceeded
**What fails:** Step handler does not include stack property in step_failed event when step is re-invoked after max retries exhausted, causing FatalError in workflow to lose original stack trace
**How to reproduce:**
1. Create a step that always fails with an error that includes a stack trace
2. Exhaust all retry attempts (e.g., maxRetries = 3, so 4 total attempts)
3. The step handler is re-invoked (attempt > maxRetries + 1)
4. This triggers the edge case at lines 127-146 in step-handler.ts
5. A step_failed event is created with fatal: true but without stack property
6. When step.ts processes this event (lines 103-104), the stack remains unset since event.eventData.stack is undefined
**Result:** FatalError created in workflow has default stack (from step.ts) instead of original error stack from step execution
**Expected:** Stack trace should be propagated consistently across all error paths. All other step_failed events with fatal: true include stack property (lines 311-313 and 351-353), but edge case was missing it.
**Root cause:** When step handler is re-invoked after max retries exhausted (attempt > maxRetries + 1), the code creates a step_failed event without including step.error?.stack, which contains the previous error information from the last failed attempt. Other code paths capture this correctly.
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
* add withData
* Enable source maps for step bundles and validate in e2e tests
- Move sourcemap generation from intermediate workflow bundle to steps bundle
- Enhance step error tests to validate function names and source files in stack traces
- Remove isLocalDeployment() checks since source maps now work in dev mode
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add changeset for step bundle source maps
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add changeset for core package e2e test improvements
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Enable source maps in CI e2e tests
Add NODE_OPTIONS="--enable-source-maps" to all e2e test jobs to ensure
stack traces show original source file paths.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix step error source map checks for local prod builds
Add hasStepSourceMaps() helper that correctly identifies when source maps
are expected to work:
- Vercel prod: works (production builds have proper source maps)
- Local dev: works (DEV_TEST_CONFIG is set, uses step bundle with inline source maps)
- Local prod: doesn't work (nitro/bundler output doesn't preserve source maps)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add hasWorkflowSourceMaps() utility for vite-based framework exception
- Add isViteBasedFramework() helper to detect vite, sveltekit, astro apps
- Add hasWorkflowSourceMaps() to check if workflow errors have source maps
(known issue: vite-based frameworks in local deployments don't preserve them)
- Refactor e2e.test.ts to use the new utility instead of inline check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* prevent sourcemap checks in nextjs and sveltekit (known offenders)
* improve source maps matrix check
* fix matrix again
* ugh more matrix ignoring
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
* [docs] Add PR template for OSS contributions
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
* Add line
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
* Empty
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
---------
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
* perf: add 5s delay to benchmark steps to simulate real work
* perf: add realistic workloads to benchmark steps
Add realistic workloads to benchmark step functions:
- doWork() - 1 second delay to simulate real computation
- stressTestStep() - 1 second delay to simulate real computation
- genBenchStream() - generates ~5KB of data in 50 chunks
- transformStream() - uppercases stream content (renamed from doubleNumbers)
Add "Slurp Time" metric to stream benchmarks measuring time from
first byte to complete stream consumption, complementing TTFB.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(e2e): increase dev test timeouts for Windows
Windows file watching and rebuilding is slower than macOS/Linux,
causing the 10s timeouts to fail. Increased to 30s to accommodate.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(ci): treat community world failures as warnings, not errors
Community world tests/benchmarks are now non-blocking:
- Removed from has_failures check in both tests.yml and benchmarks.yml
- Added separate has_warnings output for community failures
- Split PR comment notices: ❌ for failures, ⚠️ for community warnings
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: apply PR review suggestions
- Fix chunk size calculation: chunkSize - 11 for ~100 bytes per chunk
- Remove redundant metadata check (always truthy)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
The publish-results job was being skipped because it depends on the
summary job which uses `if: always()`. When a job uses `always()`,
downstream jobs need to also use `always()` in their condition,
otherwise GitHub Actions skips them when any upstream job fails.
This was causing e2e-results.json to never be published to gh-pages.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: compare benchmarks against PR base branch instead of main
- Use github.event.pull_request.base.ref instead of hardcoded main
- Remove search_artifacts: true to ensure most recent baseline is used
- For stacked PRs, this compares against the parent PR's baseline
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: group failed e2e tests by category and app in summary
Instead of listing each failed test as a separate item, group them by:
1. Category (world): e.g., "Community Worlds", "Vercel Production"
2. App (framework): e.g., "mongodb", "turso", "nextjs-turbopack"
This makes the summary much more readable when there are many failures.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: ensure local E2E tests always produce JSON output
- Add 'fastify' to app detection list in aggregate-e2e-results.js
- Change && to ; so e2e tests run even if dev.test.ts fails
- This ensures local-dev, local-prod, and local-postgres categories
appear in the E2E summary comment
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: ensure local E2E tests always produce JSON output
- Add 'fastify' to app detection list in aggregate-e2e-results.js
- Change && to ; so e2e tests run even if dev.test.ts fails
- This ensures local-dev, local-prod, and local-postgres categories
appear in the E2E summary comment
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: publish CI results to GitHub Pages for docs
- Add generate-docs-data.js script to create JSON summaries from CI artifacts
- Add publish-results job to tests.yml and benchmarks.yml workflows
- Update docs/lib/worlds-data.ts to fetch from GitHub Pages URLs
- Results published to https://vercel.github.io/workflow/ci/
This allows the docs worlds page to display actual test/benchmark
results without requiring a GITHUB_TOKEN.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: correct outputFile path for local E2E test artifacts
The --outputFile path was using ../../ which placed files outside the
repo because pnpm run test:e2e executes from workspace root, not from
the cd'd workbench directory. This prevented local-dev, local-prod, and
local-postgres test results from being uploaded as artifacts.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: show green checkmark for skipped tests instead of warning
Skipped tests are intentional and shouldn't show as warnings in the
E2E test summary comments.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: use collapsible sections in benchmark PR comment
Wrap each benchmark, stream benchmarks section, and summary tables in
<details> toggles to make the PR comment more compact and readable.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: add Vercel observability links to benchmark PR comments
- Store runId in benchmark timing data
- Add project-slug to Vercel benchmark matrix
- Pass WORKFLOW_VERCEL_PROJECT_SLUG env var to benchmarks
- Store Vercel metadata (teamSlug, projectSlug, environment) in timing files
- Generate observability deep links for each Vercel world benchmark
- Show observability links below Production (Vercel) tables
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: use correct Vercel project slugs for observability links
- nextjs-turbopack → example-nextjs-workflow-turbopack
- nitro-v3 → workbench-nitro-workflow
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* init
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* not using my prototype fastify plugin
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* renamed workbench dir
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* lockfile update
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* removed
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* symlinked workflows dir
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* added readme
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* symlinked client page
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* scope nitro route to allow renderer fallback
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* allow empty json bodies
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* comments
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* improve error handling and cleanup in /api/trigger
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* consistent server replies
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* expect json res
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* fix json res for e2e test 3,4,5,11
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* fix fastify streaming responses
Stream web streams the same way express or hono do. Read the web reader and write each json chunk to reply.raw. Avoids fastifys async iterator quirks that reordered frames and sent undefined data in the output stream tests
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* fix stream responses for e2e tests
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* clearer comments
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* fix streaming res content type. passes all core e2e tests
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* updated dev deps fastify bench
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* fastify logo (light/dark mode)
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* update url
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* fastify docs
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* Apply suggestions from code review
Co-authored-by: Adrian <me@adriandlam.com>
* Update docs/content/docs/getting-started/fastify.mdx
Co-authored-by: Adrian <me@adriandlam.com>
* Update docs/content/docs/getting-started/fastify.mdx
Co-authored-by: Adrian <me@adriandlam.com>
* Update docs/content/docs/getting-started/fastify.mdx
Co-authored-by: Adrian <me@adriandlam.com>
* Update docs/content/docs/getting-started/fastify.mdx
Co-authored-by: Adrian <me@adriandlam.com>
* Update docs/content/docs/getting-started/fastify.mdx
Co-authored-by: Adrian <me@adriandlam.com>
* Update docs/content/docs/getting-started/fastify.mdx
Co-authored-by: Adrian <me@adriandlam.com>
* Update docs/content/docs/getting-started/fastify.mdx
Co-authored-by: Adrian <me@adriandlam.com>
* Sree Narayanan <sreeaadhi07@gmail.com>
DCO Remediation Commit for Sree Narayanan <sreeaadhi07@gmail.com>
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: c3b6cf9bc4
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 3628b82c1e
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 600117f95a
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 9111151b48
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: aceb6a8ceb
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: b93df58edb
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 628718750e
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 7f523e8013
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* Apply suggestions from code review
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
Co-authored-by: Adrian <me@adriandlam.com>
* nit: standardized package.json
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* mom, my commits are in vercel CI
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* Update docs/content/docs/getting-started/fastify.mdx
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
Co-authored-by: Adrian <me@adriandlam.com>
* Update workbench/fastify/package.json
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
* passing workflow error object to err responses
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* Update workbench/fastify/nitro.config.ts
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
Co-authored-by: Adrian <me@adriandlam.com>
* Update workbench/fastify/src/index.ts
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
Co-authored-by: Adrian <me@adriandlam.com>
* feat: add start pg world plugin
* import createReadStream to serve index.html
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* test: add fastify deploy
* fix: add postgres world dep
* use symlink
* chore(deps): remove unused deps from fastify
* docs: update fastify
* Sree Narayanan <sreeaadhi07@gmail.com>
DCO Remediation Commit for Sree Narayanan <sreeaadhi07@gmail.com>
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 54c3e8a50a663c8e9aacf9c95618f59af1bd2e4c
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 423f722209c64382e86f9f8821253dc4facffe96
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: f76061627fe8d320ce498d44d0a3234abb6432ab
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: a269b8aa8d2fcf0b605cce5a1486cac481cca16e
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: b3cedf7defdb7f97b37862bf2a736d41a6bc5ec8
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 596cf21e79bc82a73b56acb9cab1fc33dbfde975
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 0ea1aa7d95e375d1f04e5501254a9fda815c7694
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: dbc8638b4a952c303e35035204cfdca43b99a059
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* remove fromNodeHandler inside fastify app
* debug print to see stream chunks
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* use readfile and resolve for root route
* moving index.html to /public
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* added back symlinked index.html (to copy during prebuildstep)
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* prebuild and postbuild script to copy and remove index.html
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* Sree Narayanan <sreeaadhi07@gmail.com>
DCO Remediation Commit for Sree Narayanan <sreeaadhi07@gmail.com>
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: b78bf227f4
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: e7a456548e
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 37ee9ddaa6
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 72912dc279
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 4ebda5b505
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: ccb690f367
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 6b80913d38
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: c9fb3ebd80
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* docs: remove preset in fastify docs
* move public/index.html symlink
* Revert "move public/index.html symlink"
This reverts commit 081d29fe16e7817912d69f1b3a3bcd3a38000d5a.
* removed stream output chunk debug log
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* Sree Narayanan <sreeaadhi07@gmail.com>
DCO Remediation Commit for Sree Narayanan <sreeaadhi07@gmail.com>
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: df5dfc9849af3cd9e818cd816feec5586137eae7
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 809bacb88461da5e92aec5d31d936222407593b6
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: b270d02ee69ac072eaa4538a31ed110edf852cc2
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: cb73ae5b1f3f7059f4c432b8226f8eb9b0492724
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 2ee3af65040e969b91a491b2bb285cd2ed447e10
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: cab1381b0e08e3453a3e5a66f55d02a8effff0e3
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 9c7471b2fba60079c71951433d9d0649d8399ba3
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 4583452a27c704ba474080d0c560a1b1b9724058
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
* lockfile
* fix
* Sree Narayanan <sreeaadhi07@gmail.com>
DCO Remediation Commit for Sree Narayanan <sreeaadhi07@gmail.com>
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 40af716dca
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 672e756cc1
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 33b9b3e1f4
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 38ca6e24e9
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: b1ada4ae63
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 472c3394fd
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: fd154df028
I, Sree Narayanan <sreeaadhi07@gmail.com>, hereby add my Signed-off-by to this commit: 7f1bebe93d
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
---------
Signed-off-by: Sree Narayanan <sreeaadhi07@gmail.com>
Co-authored-by: Adrian <me@adriandlam.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
* Add new github workflow
* Enable pull_request trigger for community worlds workflow
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add community worlds manifest and generation scripts
- Add community-worlds.json manifest as single source of truth
- Add scripts/generate-community-worlds-workflow.mjs to generate CI workflow
- Add scripts/generate-community-worlds-docs.mjs to generate docs section
- Update aggregate-benchmarks.js to load community worlds dynamically
- Add pnpm generate:community-worlds script
- Update docs/deploying/world/index.mdx with community worlds
The manifest-based approach allows:
- E2E tests to be auto-generated from the manifest
- Benchmark aggregation to include community worlds
- Docs to stay in sync with tested worlds
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix YAML syntax error - quote strings starting with @
The @ symbol has special meaning in YAML, so package names like
@workflow-worlds/turso need to be quoted.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix Redis health-cmd quoting, remove unpublished starter world
- Quote health-cmd when it contains spaces (fixes Docker arg parsing)
- Remove @workflow-worlds/starter as it's not published to npm
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add benchmarks and summary job for community worlds
- Add build job to share artifacts between benchmark jobs
- Add benchmark jobs for Turso, MongoDB, and Redis worlds
- Update summary job to show both E2E and benchmark status matrix
- Add left border/indent to sidebar child items for visual hierarchy
- Update workflow generator to support benchmark generation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Reuse build artifacts for E2E tests
E2E jobs now depend on the shared build job and download
artifacts instead of rebuilding packages from scratch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add Worlds Ecosystem dashboard to docs
- Create worlds-manifest.json with official and community worlds
- Add aggregate-worlds-data.mjs script for processing E2E and benchmark results
- Create WorldsDashboard, WorldCard, and BenchmarkChart components
- Add /docs/worlds page showing compatibility status and performance
- Include sample data for development
The dashboard shows:
- E2E test progress per world (pass/fail/skip counts)
- Benchmark performance comparison across all worlds
- Filter by official vs community worlds
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix CI to output JSON test results and add Jazz world
- Update workflow generator to output JSON test results from vitest
- Upload E2E results as artifacts for parsing in summary job
- Summary job now shows actual pass/fail/skip counts per world
- Add Jazz world to worlds-manifest.json (requires external credentials)
- Add update-worlds-status.yml workflow to auto-update dashboard data
- Update TypeScript types to support null lastRun and metrics
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor community worlds to use reusable workflows
Instead of a generated workflow file, integrate community world testing
directly into tests.yml and benchmarks.yml using reusable workflows.
- Add reusable workflows for E2E tests: e2e-community-world.yml (no services),
e2e-community-world-mongodb.yml, e2e-community-world-redis.yml
- Add reusable workflows for benchmarks: benchmark-community-world.yml,
benchmark-community-world-mongodb.yml, benchmark-community-world-redis.yml
- Update tests.yml to call reusable workflows for Turso, MongoDB, Redis
- Update benchmarks.yml to include community world benchmarks in summary
- Delete generated community-worlds.yml and generator script
This approach:
- Inherits proper Rust/SWC setup from the main workflows
- Keeps all CI in the established patterns
- Makes adding new community worlds straightforward
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix benchmark timing file naming for community worlds
Add WORKFLOW_BENCH_BACKEND env var support to bench.bench.ts so community
world benchmarks generate timing files with the correct backend suffix
(e.g., bench-timings-nextjs-turbopack-turso.json instead of -local.json).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add @workflow-worlds/starter to community worlds test matrix
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Unify worlds manifest and add dynamic GitHub API fetching
- Merge community-worlds.json into worlds-manifest.json with type field
- Add server-side data fetching from GitHub API for worlds dashboard
- Remove static worlds-status.json, fetch CI artifacts dynamically
- Update all references to use unified manifest format
- Remove obsolete community-worlds.yml and update-worlds-status.yml workflows
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Skip community worlds for non-nextjs-turbopack in benchmark summary
Community worlds only run against nextjs-turbopack, so hide the
"missing" rows for Express and Nitro frameworks in the benchmark
comparison tables.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix stream benchmark detection to check for actual TTFB data
The previous check `!== null` incorrectly returned true for undefined,
causing all benchmarks to show TTFB columns. Now explicitly checks
for a number type.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Hide Worlds Ecosystem page from sidebar
The page is still accessible via direct link at /docs/worlds but
won't appear in the navigation until it's been further iterated on.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix review comments: trailing newline and division by zero
- Add trailing newline when replacing Community Worlds section in docs
- Fix division by zero in WorldCard benchmark calculation when metrics is empty
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Consolidate community world workflows with service-type parameter
- Create setup-workflow-dev composite action for common setup steps
- Add service-type input to benchmark-community-world.yml and e2e-community-world.yml
- Use conditional job execution (if: inputs.service-type == 'mongodb') to handle different services
- Update benchmarks.yml and tests.yml to pass service-type parameter
- Delete redundant workflow files:
- benchmark-community-world-mongodb.yml
- benchmark-community-world-redis.yml
- e2e-community-world-mongodb.yml
- e2e-community-world-redis.yml
Reduces workflow files from 11 to 7 and eliminates ~500 lines of duplicated YAML.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Generate community world test matrix from worlds-manifest.json
Replace hardcoded community world jobs with dynamic matrix generation using
scripts/create-community-worlds-matrix.mjs. This allows adding/removing
community worlds by editing the manifest instead of multiple workflow files.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Apply suggestion from @vercel[bot]
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
* Add Samples column and separate local/production benchmarks
- Add Samples column to all benchmark tables showing iteration count
- Separate benchmark results into Local Development and Production sections
- Add explanatory context for each section (localhost vs Vercel deployment)
- Add GitHub action step summaries to e2e community world tests
- Create aggregate-e2e-results.js script for parsing vitest JSON output
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Remove obsolete generate-community-worlds-docs script
The Worlds Ecosystem page now fetches from worlds-manifest.json at runtime,
making this script unnecessary. The npm script also referenced a non-existent
workflow generator script.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Maximize composite action usage and reorganize benchmark output
- Update setup-workflow-dev composite action with optional Rust, install-dependencies, and install-args inputs
- Update tests.yml to use composite action in unit, e2e-vercel-prod, getTestMatrix, e2e-local-*, and getCommunityWorldsMatrix jobs
- Update benchmarks.yml to use composite action in build, benchmark-local, benchmark-postgres, benchmark-vercel, and getCommunityWorldsMatrix jobs
- Reorganize benchmark output to group by benchmark test with local/production tables within each benchmark
- Remove invalid $schema reference from worlds-manifest.json
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add beads stealth mode stuff (for personal claude memory - will remove stealth if people want)
* Add E2E test results PR comment summary
- Add pr-comment-start job to create/update PR comment when tests start
- Add artifact uploads to all e2e test jobs (vercel-prod, local-dev, local-prod, local-postgres, windows)
- Update e2e-community-world.yml with consistent artifact naming (e2e-community-*)
- Add summary job to aggregate all e2e results and update PR comment
- Extend aggregate-e2e-results.js with --mode aggregate for multi-job PR summary
- Group results by category (Vercel Production, Local Development, etc.)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add step summaries to all e2e test jobs
Add "Generate E2E summary" step to each individual e2e job:
- e2e-vercel-prod
- e2e-local-dev
- e2e-local-prod
- e2e-local-postgres
- e2e-windows
Each job now outputs pass/fail/skip counts to GITHUB_STEP_SUMMARY.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Consolidate community world workflows to single job
Replace 3 mutually exclusive jobs (e2e/e2e-mongodb/e2e-redis) with a single
job that starts services via docker run when needed. This eliminates the
skipped job entries that appear in the GitHub Actions UI.
- Use conditional docker run steps instead of services: block
- Add health check loops to wait for service readiness
- Add cleanup step to stop containers
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix extractWorldId to handle community world artifact naming
Add handling for `e2e-results-community-{world}` pattern so community
world test results are properly extracted (e.g., `e2e-results-community-turso`
now correctly extracts `turso` instead of `community-turso`).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
* Add stream benchmarks and cnealup for benchmark code
* 10m delay for local world race
* potential fix for first bute"
* Improve sticky comment stuff
* local world: ignore controller close errors
* show missing data in benchmark comment
* nitrpicks
* Improve leaderboard
* Add benchmark comparisons against main
* changeset
* Add e2e tests for postgres world
* Add e2e tests for postgres world
- Add new e2e-local-postgres job to test all workbenches against postgres world
- Configure PostgreSQL service container in GitHub Actions
- Update world-postgres build script to chmod +x the setup.js bin
- Use direct path to setup script in CI for reliability
- Tests run in production mode similar to e2e-local-prod tests
* Normalize Workbenches
Normalize trigger scripts across workbenches
fix: include hono in local build test
test: include src dir for test
test: add workflow dir config in test to fix sveltekit dev tests
add temp 7_full in example wokrflow
format
fix(sveltekit): detecting workflow folders and customizable dir
Remove 7_full and 1_simple error
replace API symlink in webpack workbench
Fix sveltekit and vite tests
Fix sveltekit symlinks
Test fixes
Fix sveltekit workflows path
Dont symlink routes in vite
Include e2e tests for hono and vite
fix error tests post normalization
wip - attempted fixes
* Add claude demo command
* fix: normalize workbench tests (#292)
* Proper stacktrace propogation in world
Proper stacktrace propogation in world
* Standardize the error type in the world spec
* Normalize Workbenches
Normalize trigger scripts across workbenches
fix: include hono in local build test
test: include src dir for test
test: add workflow dir config in test to fix sveltekit dev tests
add temp 7_full in example wokrflow
format
fix(sveltekit): detecting workflow folders and customizable dir
Remove 7_full and 1_simple error
replace API symlink in webpack workbench
Fix sveltekit and vite tests
Fix sveltekit symlinks
Test fixes
Fix sveltekit workflows path
Dont symlink routes in vite
Include e2e tests for hono and vite
* fix error tests post normalization
* fix(sveltekit): reading file on hmr delete
* changeset
* fix(vite): add resolve symlink script
* fix(vite): missing building on hmr
* test local builder in vite
* test: increase timeout on hookWorkflow
* test: ignore vite based apps in crossFileWorkflow
* test: fix nitro based apps status codes
* fix: intercept default vite spa handler on 404 workflow routes
* fix: vite hook route returning 422
* test: use 422 for hookWorkflow expected
* test: fix hono returning 404
* chore: add comment to middleware to clarify
* make api route for duplicate case
* revert
* revert: nitro builder
* add back nitro unhandled rejection logic
* test: add hono
* changeset
* fix: unused method
* fix: remove duplicate import
* remove
* chore: add comments to clarify
* test remove vite symlink script
---------
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
* refactor: add top level resolve symlinks script
* fix: cleanup builder directories (#319)
* fix: add sveltekit server routes to builder
* fix: remove root workflow dir check
* fix missing root level workflow route
* Fix: The constructor now hardcodes `dirs: ['src/routes', 'src/lib']` which silently ignores any user\-provided `dirs` option passed to the plugin\, breaking the documented API and removing support for custom workflow directories\.
* Fix: The test expectations don\'t match the new implementation of `getWorkflowDirs()`\. The mock provides `scanDirs` which the new code no longer uses\, and the new implementation adds scanning of `routesDir` and `apiDir` instead\.
* fix(nitro): use src dir
---------
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
* refactor(nitro): use suppressUndefinedRejections
* revert: sveltekit builder
---------
Co-authored-by: Adrian <me@adriandlam.com>
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
* feat: add getPort method for detecting pid port
* lockfile
* update tests and fix getPort usage
* changeset
* docs: update sveltekit getting started
* fix: use pid-port
* fix: not using config port env
* fix: remove unused getPort from world core
* remove unused stuff
* fix: world local config returning port 3000 as fallback
* changeset
* fix: rebase conflicts
* fix: util test missing http import
* changeset
* fix: wrong import for getPort in core runtime
* fix: getPort in @workflow/utils being imported into workflow runtime
* test: simplify sveltekit test
* fix missing import in test
* fix: async await stuff with getPort
* refactor: move getPort to @workflow/utils/get-port
* test: simplfiy getPort tests
* test: fix sveltekit ports
Peter Wielander