* test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident)
Reproduces the o2flow v5 upgrade failure (workflow@5.0.0-beta.26, fixed by
#2752 in beta.28): a plain API route importing defineHook() from the root
`workflow` entry and calling .resume() failed with Turbopack's
"Cannot find module as expression is too dynamic" stub, because the world
registration was tree-shaken out of the route bundle and getWorldLazy()'s
dynamic-import fallback got stubbed.
The bug only manifests when a route bundle loads in isolation (a Vercel
lambda): local `next dev`/`next start` evaluates next.config.ts, whose
workflow/next import chain registers the world process-wide and masks it —
which is why no existing server-driven suite caught it.
- route-bundle-isolation.test.ts: production Turbopack build of the
nextjs-turbopack workbench, then loads ONLY the compiled route bundle in a
bare Node subprocess (cold-lambda simulation) and invokes its POST handler.
Fails with the exact incident error on regressed code; passes on main.
Wired into the build-error-messages CI job.
- e2e: plainModuleDoneHook round-trip through a plain API route on the two
Next workbenches (deployed matrix covers real lambda isolation).
- Workbench fixtures mirroring o2flow: a directive-less defineHook module
shared by a workflow (create) and a plain route (resume). The webpack
workbench gets a real route file because `next dev` (webpack) does not
serve directory-symlinked app routes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
* test: authenticate plain hook resume request
* test: address review — marker-based harness output parsing, changeset summary
- route-bundle-isolation: prefix the harness result line with a unique
marker and locate it explicitly instead of JSON.parse()ing the last
stdout line, so stray logging from the route bundle or the world can't
break parsing; failures now include the full subprocess stdout.
- changeset: add a human-readable summary to the (release-less) changeset.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
---------
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Karthik Kalyanaraman <karthik.kalyanaraman@vercel.com>
Co-authored-by: Karthik Kalyan <105607645+karthikscale3@users.noreply.github.com>
* [world-vercel] Add /run-id sub-export with tagged ULID encode/decode
Encodes a tag bit, 5-bit version, and 6-bit Vercel region ID into a
ULID-shaped string used for workflow run IDs. Tagged values remain
valid 26-char Crockford-Base32 ULIDs so they still sort and round-trip
through any system that accepts ULIDs.
* [world-vercel] Add string-value assertions to run-id tests
Add exact-string expectations for encoded outputs at known inputs,
covering the default region/version pair, numeric region IDs, version
overrides, boundary values (all-zero, all-max), the dirty-input
overwrite case, and the lexicographic-order checks. Also adds an
explicit byte-array expectation for the canonical ULID-spec example
string and an additional first-char-range coverage test for isTagged.
* [world-vercel] Remove internal-repo reference from regions doc comment
* [world-vercel] Address PR review feedback on run-id sub-export
- isTaggedString now fully validates the input as a 26-char Crockford
Base32 ULID (delegating to ulidToBytes) instead of only inspecting
the first character. This fixes false positives on inputs like
'4UUUU...' that have a valid tag-bit position but invalid chars
later in the string.
- isTagged() now accepts `unknown` to match its documented behavior
of safely rejecting non-string inputs without requiring callers to
cast.
- Introduce `RegionKey` for the full set of keys including 'unknown',
and narrow `RegionCode` to `Exclude<RegionKey, 'unknown'>` so the
return type of `lookupRegion` and the `DecodedRunId.region` field
accurately reflect that 'unknown' is never produced. Updates
`encode` to reject 'unknown' as a region code string at runtime
(callers wanting the unknown sentinel should pass numeric 0).
* [world] [core] [world-vercel] Add World.createRunId() and region-aware queue routing
- @workflow/world: add optional createRunId(input?) to the World
interface so worlds can mint run IDs with embedded metadata, and
add an optional 'region' field to QueueOptions for per-message
routing hints.
- @workflow/core: start() now delegates run ID generation to
world.createRunId() when defined (falling back to a monotonic
ULID otherwise), and accepts a new 'runIdInput' option that is
forwarded verbatim to createRunId. When runIdInput.region is a
string, it is also threaded onto the queue options so the initial
workflow message is dispatched to the matching region.
- @workflow/world-vercel: implement createRunId() to mint
region-tagged ULIDs, preferring an explicit runIdInput.region and
falling back to the VERCEL_REGION env var. The queue now resolves
its destination region from (in order): an explicit opts.region,
the region embedded in the payload's tagged run ID, the
VERCEL_REGION env var, and finally a hardcoded 'iad1' default.
This replaces the previous unconditional 'iad1' region passed to
the @vercel/queue client.
Monotonicity within a process is preserved by tracking the last
emitted run ID and bumping the bit immediately above the 11-bit
metadata window when a same-ms collision would otherwise occur,
then re-stamping the requested region/version on top so metadata
remains stable.
* [core] [world] [world-vercel] Pass full StartOptions to World.createRunId
Drop the dedicated 'runIdInput' field on StartOptions and forward the
entire options bag to world.createRunId() instead. This keeps the
public API surface smaller and lets each World pick the fields it
recognises (e.g. world-vercel reads 'region'). The top-level 'region'
option remains on StartOptionsBase and is also threaded onto the
queue's per-call region opt when set.
* Address review feedback: doc fixes and deterministic same-ms tests
- Document the final iad1 fallback in QueueOptions.region (world)
- Correct the World.createRunId doc: start() always passes an object
- Fix the clientOptions comment: the handler client omits region and
relies on SDK auto-detection + the ce-vqsregion header for acks
- Fix a misleading QueueClient-construction comment in queue.test.ts
- Freeze time in the same-ms monotonicity test so it deterministically
exercises the intended path, and add a test covering the
bump-above-metadata fallback when the region changes mid-millisecond
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: keep workflow-server override rewrite-compatible
Export WORKFLOW_SERVER_URL_OVERRIDE while keeping the one-line const shape
that workflow-server's cross-repo e2e test automation rewrites. Update
world-vercel tests to import that exported value for mock origins and URL
expectations instead of duplicating the temporary preview URL.
* fix(world): clear region tag bit before ULID timestamp validation
Region-tagged run IDs set the high bit of the ULID timestamp byte. The
shared world timestamp validator used raw decodeTime(), so current tagged
run IDs appeared thousands of years in the future and were rejected before
reaching workflow-server. Clear the tag bit before decoding, matching the
workflow-server behavior, and cover tagged IDs in tests.
* fix(world-vercel): validate tagged runId timestamps via run-id decode
Keep @workflow/world's ULID helpers generic; they should not know about
world-vercel's region-tagged run ID layout. Instead, world-vercel decodes
its tagged runId to the original ULID before using the shared timestamp
validator for run_created events. Add a world-vercel regression test that a
current sfo1-tagged runId passes validation.
* fix(world-vercel): default run ID region to iad1 instead of unknown
When neither an explicit region option nor VERCEL_REGION is available,
createRunId minted a tagged ULID with the unknown (0) region sentinel,
producing the tagged: true, region: null state. The server already
resolves unknown/untagged runs to DEFAULT_VERCEL_REGION (iad1), so mint
a concrete iad1 tag instead, keeping every run ID self-describing and
routable.
* test(e2e): use verbose reporter + per-test start heartbeat
The default vitest reporter buffers per-file output, so a stalling e2e
test produces no output until its timeout — making CI look like a silent
30-minute hang. Switch the e2e CI invocations to the verbose reporter
(prints each test result as it completes) and emit a '[e2e] ▶ start:'
heartbeat to stdout at the start of every test (bypassing vitest's console
buffering) so a stuck test is immediately identifiable in the live CI log.
* test(world-vercel): point WORKFLOW_SERVER_URL_OVERRIDE at combined 527+529 preview
Temporarily target the workflow-server combined-527-529-preview deployment,
which bundles platform-directed multi-region routing (vercel/workflow-server#527,
incl. the iad1 hook pin) and durable stream state (vercel/workflow-server#529),
so e2e can validate the full multi-region path end-to-end. Revert to empty on main.
* fix(core): region-tag the health-check correlationId
The health-check response is delivered over a Redis stream whose name (and
synthetic run ID) embed the correlationId. Under platform-directed routing
the responding endpoint and the polling reader can be served from different
physical regions; Redis is physical-region-local, so the correlationId must
carry the region for both sides to resolve the same backend.
Generate the correlationId via world.createRunId() (a region-tagged ULID)
when the world provides it, falling back to a plain ULID for worlds that
don't tag IDs (e.g. local, single-region). The synthetic wrun_hc_<id> run ID
then carries the region; workflow-server's region middleware decodes it.
* Address review feedback: validate region overrides, reset server override
- queue: validate opts.region and VERCEL_REGION against the known region
table before routing, ignoring unrecognised codes so a bad override
can't clobber the payload-derived region (Copilot)
- add isKnownRegionCode() runtime guard to run-id/regions
- reset WORKFLOW_SERVER_URL_OVERRIDE to '' (must be empty on main)
- fold the within-PR iad1-default changeset into the main world-vercel
changeset and delete it (review)
- start.test: declare specVersion on createRunId mock worlds now that
the merged world-compatibility check requires it
- cover the new region-validation fall-through paths in queue.test
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(world-vercel): point WORKFLOW_SERVER_URL_OVERRIDE at wave-1 multi-region preview
BRANCH-ONLY — revert the override to '' before merge (lint enforces).
Points this PR's e2e/benchmark runs at the wave-1 multi-region
workflow-server preview (vercel/workflow-server#590: iad1+sfo1+fra1
serving, staging data backends) so region-tagged runs are validated
against real multi-region serving end-to-end.
Also makes the unit-test mock origins in events-v4.test.ts and
trace-propagation.test.ts override-aware (same pattern the rest of
the file and utils.test.ts already use), so the suite passes whether
or not the override is set — these two files were the only spots
hardcoding https://vercel-workflow.com.
* test(e2e): Vercel multi-region suite for start()'s region option
Adds a dedicated e2e suite validating @workflow/world-vercel region
routing end to end, run as its own CI job (e2e-vercel-multi-region)
against the nextjs-turbopack workbench only — deliberately separate
from e2e.test.ts, which runs as a matrix across all worlds/frameworks
where Vercel-specific multi-region behavior doesn't apply.
- workbench/nextjs-turbopack/vercel.json: deploy to iad1+sfo1+fra1 so
region-routed flow messages have a function to land on in each region.
- workflows/99_e2e.ts: regionProbeWorkflow returns the VERCEL_REGION
observed by both the workflow and a step, so tests can assert the run
EXECUTED in the intended region (not just that it was tagged).
- packages/core/e2e/e2e-region.test.ts: per-region cases assert
1) start(..., { region }) mints a region-tagged run ID (decoded via
@workflow/world-vercel/run-id),
2) the workflow + step both observed VERCEL_REGION === region,
3) the server reports the run completed;
plus a concurrent all-regions case guarding against cross-region
misrouting under simultaneous multi-region traffic. Skips on local
deployments.
- .github/workflows/tests.yml: new e2e-vercel-multi-region job
mirroring e2e-vercel-prod's env/deployment-wait, running only the
new suite.
* test(e2e): start region probes in-function; fix getWorld await
The first multi-region CI run surfaced two issues:
1. sfo1/fra1-tagged runs executed in iad1. The suite started runs from
the external test process, which uses the api.vercel.com token proxy
— and the proxy's queues path forwards every send to the region-less
VQS host (the world's proxy-mode resolveBaseUrl ignores the region
argument, and the proxy's x-vercel-vqs-api-url escape hatch only
allowlists vqs-server-*.vercel.sh preview hosts). Production traffic
publishes IN-FUNCTION (direct regional queue routing), so the suite
now triggers start() through a new workbench route
(/api/e2e-region-start) and rehydrates the run with getRun() —
testing the path production actually takes. Proxy-mode regional
queue routing is a known gap to address separately in api-workflow.
2. TypeError on world.runs.get: getWorld() is async and was called
without await.
* test(e2e): cover explicit and implicit region starts in the multi-region suite
With regional VQS routing now working through the api.vercel.com proxy
(vercel/api#79056 + #2789 + this branch's per-send region resolution),
the suite covers both start configurations, asserting the same three
properties for each (region-tagged run ID, execution in the intended
region via VERCEL_REGION echoed in the return value, server-side
completion):
1. EXPLICIT: start(..., { region }) called directly in the vitest
runner — publishes through the token proxy, per-send region carried
by x-vercel-queue-region. Restores the direct-start shape the suite
had originally, plus the concurrent all-regions case.
2. IMPLICIT: dedicated per-region workbench routes
(/api/e2e-region-implicit/{iad1,sfo1,fra1}), each pinned to a single
region via a per-function 'regions' entry in the workbench
vercel.json, calling start() with NO region option — createRunId
derives the tag from the minting function's VERCEL_REGION. The test
also asserts the route reported executing in its pinned region, so
the implicit-tagging assertion can't pass vacuously.
Replaces the interim /api/e2e-region-start route (explicit region via
request body), which existed to work around the pre-#79056 proxy gap.
* Revert WORKFLOW_SERVER_URL_OVERRIDE to '' — wave-1 multi-region serving is in production
workflow-server#590 (iad1+sfo1+fra1 serving) merged and deployed to
production and the e2e backend, so this branch's e2e/benchmark runs no
longer need to target the wave-1 preview. Restores the empty override
the No Test Overrides lint job enforces for merge.
The override-aware unit-test origins (events-v4/trace-propagation)
stay — they are correct under any override value.
* test(e2e): cross-region stream visibility (iad1 writer, sfo1 reader)
Regression coverage for a backend bug that made cross-region stream
reads report zero chunks on IN-PROGRESS streams (completed streams were
unaffected), which forced the multi-region serving rollback.
The new case exercises exactly that geometry:
- crossRegionStreamWorkflow (99_e2e.ts) writes N chunks to the default
output stream, then holds the stream OPEN for 45s before closing —
the in-progress window is the point, since completed streams are the
easy case.
- The e2e starts it with region iad1, waits (same-region, via the
api.vercel.com proxy) until all chunks are written, asserts the run
is still 'running', then reads through a new sfo1-pinned workbench
route (/api/e2e-stream-read/sfo1) that returns getTailIndex() plus
its VERCEL_REGION. The reader's region served none of the stream's
writes, so the reported chunk count must come from the backend's
cross-region stream metadata. The test fails loudly if the route
isn't actually executing in sfo1.
Also bumps the explicit-region test timeout to 120s: the first case in
the file absorbs every cold start at once (fresh workbench instances
in up to three regions plus a cold backend preview) and was observed
just over the 60s default.
BRANCH-ONLY (revert before merge, lint enforces):
WORKFLOW_SERVER_URL_OVERRIDE points at a multi-region backend preview
that includes the fix, so this validates cross-region stream
visibility end-to-end before multi-region serving is re-enabled.
* test(e2e): extend multi-region suite to all 19 provisioned regions
Points the suite at an all-regions backend preview and widens coverage
from the wave-1 trio to every provisioned region:
- Explicit path: a single concurrent all-regions case starts one
tagged run per region (one shared cold-start window instead of 19
sequential ones) and aggregates per-region failures so a single
region's breakage reports alongside the full picture. The trio keeps
its detailed per-region cases and the 9-way concurrent-isolation
case.
- Implicit path: workbench gains a region-pinned
/api/e2e-region-implicit/<region> route per provisioned region (19
total, shared handler), the workbench itself now deploys to all of
them, and the test.each covers the full set with per-case timeouts
for regional cold starts.
- Multi-region CI job timeout 20m -> 35m for the sequential implicit
cases.
BRANCH-ONLY (revert before merge, lint enforces):
WORKFLOW_SERVER_URL_OVERRIDE now targets the all-regions backend
preview instead of the previous (stale, since-merged) fix preview.
* test(e2e): tolerate geo-adjacent execution of queue callbacks
The first all-regions run surfaced a subtle execution-locality
behavior: queue delivery is guaranteed to the tagged region's
dataplane and the delivery callback egresses from that region, but the
consumer invocation's execution region is chosen by where that
callback enters Vercel's edge — and adjacent regions can geo-resolve
to each other's functions. Observed live: kix1-tagged runs (callback
egressing from Osaka) deterministically executing in hnd1/Tokyo on
both the explicit and implicit paths, with tagging, data placement,
and completion all still strictly kix1.
expectRunInRegion now asserts execution lands in the tagged region OR
one of its geographic neighbors (EXECUTION_ADJACENCY), while run-ID
tagging and server-side completion remain strictly the requested
region. Gross misrouting (e.g. kix1 -> iad1) still fails.
* Revert WORKFLOW_SERVER_URL_OVERRIDE to '' — all-regions serving is in production
The all-regions workflow-server rollout is deployed and serving
production traffic from every Vercel region, so this branch's e2e no
longer needs to target a branch preview. Restores the empty override
the No Test Overrides lint enforces for merge.
With this the PR is complete: region-tagged run IDs, region-aware
queue routing, and the multi-region e2e suite (explicit + implicit +
all-regions + cross-region streams) all validate against the
production-default backends.
* docs: fix three stale comments flagged in review
- start.ts: StartOptionsBase.region fallback is iad1, not the unknown
sentinel (createRunId always mints a concrete routable region)
- queue.ts: example used a nonexistent start({ runIdInput }) API; the
real option is start({ region })
- events.ts: decode() clears only the tag bit (top bit of the 48-bit
timestamp field) — it does not restore the original untagged ULID;
reword to say what actually matters for timestamp validation
* test(e2e): cover hook resolve/resume for runs owned by non-iad1 regions
Hooks are resolved by opaque token, which carries no region hint, so
lookup and resume must work regardless of which region owns the run's
data. Exercises the full follow-up-message path on sfo1- and
fra1-tagged runs: create inside the workflow, resolve by token from
the test process, resume twice sequentially, and assert payload order
and completion.
Regression coverage for the failure mode where the first message to a
hook-driven app on a non-iad1 run worked but every follow-up failed
with 'Hook not found'.
---------
Co-authored-by: Peter Wielander <peter.wielander@vercel.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Optimize Next dev HMR rebuilds
* Fix Next dev HMR CI coverage
* Gate dev HMR logs behind opt-in flag
* Match workflow dev build logs to Next style
* Fix Next dev HMR changed-file classification
* Fix Windows port detection
* Relax HMR log wait in dev e2e
* Avoid canary workflow execution cache flakes
* Allow slower Turbopack HMR propagation in e2e
* Scope canary HMR fuzz execution assertions
* Add Platformatic World to worlds-manifest.json
Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>
* ci fixup
Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>
* ci: pin platformatic world image to 0.8.1 and harden community-world runner
Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>
* platforamtic-world version
Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>
* ci: wire generic docker service-type into community benchmark workflow
The shared community-worlds matrix now emits service-type "docker" for any
world with non-builtin or multiple services (e.g. Platformatic, which needs
postgres + the platformatic/workflow image). tests.yml's e2e-community path
already handles it, but benchmarks.yml's benchmark-community path
(label-gated, non-blocking) did not — so a "community-benchmarks" run would
start no services and fail.
Mirror the e2e "Start Docker services" step, package-version pin, and docker
cleanup into benchmark-community-world.yml, and pass `services`/`version`
through from benchmarks.yml.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add a `--url` flag to `inspect`/`web` that prints a run's observability
dashboard deep link to stdout and exits — no browser, no local server —
so scripts and agents can share a link instead of opening a UI.
Fix the Vercel dashboard URL to the current
`…/workflows/runs/<id>?environment=<env>` route (drop the legacy
`/observability` segment) and respect `--env`. Apply the same route fix
to the e2e helpers, CI aggregation scripts, and the nextjs-turbopack
workbench. Document deep-linking in the workflow skill and observability
docs.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Reduce unnecessary CI runtime
* Fix shared E2E artifact extraction path
* Stabilize getWorkflowPort timeout test on Windows
* Preserve UI unit coverage on CI fast path
* ci: extract wait-for-vercel-project to vercel/wait-for-deployment-action
The action's logic was duplicated between this repo and
vercel/workflow-server, which is annoying to keep in sync. Move it to
a standalone repository so both can consume the same pinned build.
Changes:
- Delete .github/actions/wait-for-vercel-project entirely.
- Replace all five `uses: ./.github/actions/wait-for-vercel-project`
references with `uses: vercel/wait-for-deployment-action@<sha>` in:
benchmarks.yml, dispatch-front-workflow-release-pr.yml,
docs-checks.yml, tarballs-checks.yml, tests.yml
- All `with:` inputs (project-slug, environment, timeout,
check-interval, github-token) are unchanged — the new action's
input contract is backwards-compatible.
The new action is ESM-only, targets Node 24, ships a ~12KB bundle
(down from ~830KB in the old in-repo version) by dropping
@actions/core and its transitive undici dependency, and is
unit-tested. See https://github.com/vercel/wait-for-deployment-action.
* ci: bump wait-for-deployment-action to fix/status-context-auto for verification
Repinning to vercel/wait-for-deployment-action#fix/status-context-auto
(SHA 04d46ef) which fixes the broken 'opt-out' heuristic that made
status-context resolution silently disabled for every consumer.
Reproduced in this repo's E2E logs:
Looking for GitHub deployment in environment "Preview – example-workflow"
Deployment ID resolution disabled (status-context is empty)
Deployment ready: https://example-workflow-...labs.vercel.dev
Run E2E Tests: VERCEL_DEPLOYMENT_ID= <-- empty
Will repin to the post-merge main SHA once CI is green.
* ci: bump wait-for-deployment-action pin to merged main SHA
Repinning from the fix/status-context-auto branch (04d46ef) to the
post-merge main SHA (0e2b0c5, vercel/wait-for-deployment-action#4).
The deployment-id resolution fix verified against the prior fix-branch
pin (E2E tests now read VERCEL_DEPLOYMENT_ID=dpl_... correctly across
the matrix; only flaky/unrelated Vercel deployment failures remain).
* ci: grant statuses:read alongside deployments:read
The wait-for-deployment-action also reads the 'Vercel – <slug>'
combined commit status to resolve the dpl_xxx ID. The official
permissions table lists statuses:read for
GET /repos/{owner}/{repo}/commits/{ref}/status.
GitHub Actions doesn't currently support prefilling workflow_dispatch
inputs via URL query params (community/community#51159), so the
"override via workflow_dispatch with this commit SHA" instruction in
the no-backport comment required the reader to go look up the SHA
themselves. Paste the full 40-char SHA into the comment in a fenced
code block so it's one click to copy into the "Commit SHA" input on
the workflow run page.
Major-version refs like `@v2`/`@v5` resolve to mutable refs on the
upstream repos — sometimes a tag, sometimes a branch (e.g. marocchino
keeps `v1`/`v2`/`v3` as branches), and dawidd6 force-pushes the bare
`v6` tag forward outside of releases. A compromised maintainer account
could push new code that our CI picks up on the next run with
GITHUB_TOKEN (or, for changesets/action, NPM_TOKEN) in hand.
Pin all third-party `uses:` references to full commit SHAs with a
trailing version comment so the upstream release is still visible to
reviewers. Dependabot/Renovate can keep these fresh going forward.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(backport): mark not-maintained-on-stable list as exhaustive and require git verification
The backport workflow's AI decision prompt only listed `docs/` (outside
`docs/content/`) and `skills/` as paths not maintained on `stable`, but
didn't flag that list as exhaustive. On at least one commit, the AI
generalized the pattern and incorrectly claimed `tarballs/` was also
not maintained on `stable` (it is) — likely because the commit subject
mentioned "preview tarball" and the changed files included a docs
preview smoke check workflow.
Tighten the prompt to:
- explicitly mark the not-maintained list as exhaustive;
- instruct the AI to run `git ls-tree origin/stable -- <path>` to
verify any other path it wants to cite as main-only;
- warn it not to infer main-only-ness from suggestive names like
"docs", "preview", "tarball", or "workflow".
* ci(backport): link to the backport job run in posted comments and PR body
Each of the comments the backport workflow posts (no-backport,
backport-created, conflict-failure) and the body of the backport PR
it opens now includes a link to the GitHub Actions run that produced
the decision. Makes it easy to jump from the comment straight to the
opencode output, prompt, and AI reasoning when the AI gets a call wrong.
The repo's enterprise `~ALL` required-signatures ruleset rejects the
unsigned commits produced by `peaceiris/actions-gh-pages@v4`, breaking
the benchmark and E2E result publishing jobs on `main`. Replace those
steps with a shared composite action that uses the GraphQL
`createCommitOnBranch` mutation — same pattern already used by
`backport.yml` — so commits are signed automatically by GitHub and
satisfy the rule.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* drop setup-command input from reusable community-world workflows
The community-world matrix is produced by running
scripts/create-community-worlds-matrix.mjs in the fork PR's checkout,
so any field on it is attacker-controlled. Forwarding
matrix.world.setup-command into the reusable workflow and eval-ing it
let a malicious fork PR execute arbitrary shell on the runner.
Replace the pass-through with a hardcoded per-world-id case in the
reusable workflows (only turso currently needs a setup step) and drop
the setup field from the matrix generator.
* rename step to "Per-world setup"
Addresses Copilot review feedback: the step no longer executes an
arbitrary command, so the old name was misleading.
Drops the label-based backport override in favor of workflow_dispatch.
The pull_request_target trigger has security concerns (it runs with
write permissions on PR-controlled events), and we already have a
manual dispatch path that covers the same use case.
Large backport prompts (commit message + diff capped at 200KB) can
exceed Linux's `ARG_MAX` and cause `opencode run` to fail with
"Argument list too long" (exit 126). Redirect the prompt files into
`opencode run`'s stdin instead of passing them on the command line.
* Hoist AI model env, fix opencode external_directory permission, fail loud on AI infra errors
Three related fixes triggered by the failed run on #1935:
1. Hoist the AI model name to a top-level `AI_MODEL` env var
(`anthropic/claude-opus-4.7`); both `opencode run` invocations
now interpolate `vercel/${AI_MODEL}` so the model is specified in
exactly one place.
2. Switch `OPENCODE_PERMISSION` from the bare-string shortcut
`"allow"` to the explicit object form
`{"*":"allow","external_directory":"allow"}`. The shortcut
was observed not to override `external_directory` (which defaults to
"ask" and auto-rejects in non-interactive `opencode run`),
causing the conflict-resolution AI to fail when reading scratch files
it created under `/tmp/`.
3. The `Resolve conflicts with opencode` step no longer uses
`continue-on-error`, and now distinguishes two outcomes via an AI-
written outcome file (`.backport-conflict-outcome.json`):
- `{"status":"resolved"}` — the legitimate clean path; cherry-pick
continues and the backport PR is opened.
- `{"status":"unresolved", ...}` — the legitimate "AI couldn't
do it, hand off to a human" path; `resolved=false` is set and the
conflict-failure comment is posted on the source PR.
- Anything else (missing file, malformed JSON, unknown status) is
treated as an opencode/AI Gateway infra failure: the step exits
non-zero, the workflow fails red, and the misleading
"couldn't resolve" comment is suppressed.
The prompt + scratch files are also moved into the workspace so
opencode never needs `external_directory` access anyway.
* Allow manual workflow_dispatch with ref+model inputs; use AI_MODEL in PR body
Add a `workflow_dispatch` trigger to the backport workflow with two
optional inputs:
- `ref` — commit SHA on `main` to back-port (defaults to `main` HEAD)
- `model` — overrides the default AI model used by opencode for the
decision and conflict-resolution steps (defaults to the workflow's
hardcoded `AI_MODEL`)
The top-level `AI_MODEL` env var now uses
`${{ inputs.model || 'anthropic/claude-opus-4.7' }}` so manual runs
pick up the override without changing anything else.
Manual dispatch (like the `backport-stable` label) always forces a
backport regardless of any AI verdict — the operator's intent is
explicit by virtue of triggering the workflow. The PR body shows
"Triggered manually via `workflow_dispatch`." in that case.
The PR body's conflict-resolution attribution also now interpolates
`${AI_MODEL}` (e.g. "opencode with `anthropic/claude-opus-4.7`")
instead of hardcoding "Claude Opus" so the text stays accurate if the
default model is later changed.
* Address PR review: also detect leftover conflict markers in staged files
The previous `Resolve conflicts with opencode` sanity check used
`git diff --diff-filter=U` to detect unresolved cherry-pick conflicts,
which only catches unmerged index entries. That misses the case where
the AI runs `git add` on a file that still has `<<<<<<<` /
`=======` / `>>>>>>>` markers in its content — git happily stages
the broken file as a normal modification.
Add a second check using `git diff --check --cached`, which emits
`leftover conflict marker` lines when any staged content still has
the standard markers. Grep specifically for that phrase so unrelated
whitespace warnings don't trip the check. Also update the inline
comment to accurately describe what each check covers (per Copilot's
review on #1943).
The repo has an enterprise-level branch ruleset requiring verified
signatures on every ref (`~ALL`), so a normal `git push` of a locally
cherry-picked commit is rejected ("Commits must have verified
signatures"). Replace the `git push` step with a GraphQL
`createCommitOnBranch` mutation, which signs commits automatically
with GitHub's internal key (the same way commits made via the web UI
are signed).
Walks the cherry-pick's diff against the parent (`stable` HEAD),
collects file additions (with binary-safe base64 contents read via
`git cat-file blob`) and deletions, ensures the backport branch
exists on the remote, then runs the mutation. Also updates the manual
conflict-resolution instructions in the failure comment to mention
that local cherry-picks must be signed (`git cherry-pick -S`)
because of the same ruleset.
Caveats:
- Authorship is lost — `createCommitOnBranch` always attributes
commits to the token owner (`github-actions[bot]`). The original
commit SHA is still referenced in the PR body.
- Non-regular files (executable bit, symlinks, submodules) are not
supported by the mutation; the step warns and proceeds with mode
100644 for affected paths.
The previous `OPENCODE_PERMISSION` value (`{"allow":["*"]}`) was the
wrong shape for opencode's permission config and was silently falling
through to defaults. Notably `external_directory` defaults to "ask",
which auto-rejects in non-interactive `opencode run` — causing the
`write` tool to fail when the AI tried to create the decision file
under `/tmp/`. Use the documented form (`"allow"` as the entire
permission config) and also move the decision/diff/prompt files into
the working directory so opencode doesn't need `external_directory`
permission at all.
Additionally, treat any opencode/AI Gateway failure (auth error,
rejected tool call, missing or malformed decision file) as a hard job
failure rather than silently defaulting to "no backport". A previous
expired AI Gateway key produced a green job that simply skipped the
backport without any indication of the underlying infra problem.
* Restructure backport workflow with AI-driven decisions
Run the backport workflow on every push to main and have AI analyze each
commit to decide whether to recommend a backport to stable, instead of
relying on a manual backport-stable label. The action now always opens a
PR for human review and never pushes directly to stable. The
backport-stable label is preserved as a manual override that forces a
backport regardless of the AI verdict.
* Address PR review: randomized output delimiters and updated manual instructions
- Use uuidgen-based delimiters when writing multiline values (PR title,
body, AI reasoning) to $GITHUB_OUTPUT, so user-/model-controlled
content cannot collide with or inject into the heredoc terminator.
- Update the manual conflict-resolution instructions to push a backport
branch and open a PR against stable, matching the new "never push
directly to stable" policy.
- Document the head-commit-only behavior of the push trigger inline in
the workflow.
* Add stable Next.js eager and lazy test coverage
* Address PR review feedback
* Fix eager Next step route builds
* Fix eager Next manifest refreshes
* Fix eager Next e2e stack assertions
* Externalize native step bundle bindings
* Lazy load Vercel world runtime
* Fix Next dev step sourcemap assertions
* Consolidate eager build changesets
* Fix Vercel world tracing in Next deployments
* Externalize Vercel world in Next builds
* Fix webpack tracing for Vercel world deps
* Fix eager workflow route bundling
* Rely on Next server externals
* Introduce structured context-violation errors + Ansi renderer
Phase 1: Add Ansi rendering helpers (frame, hint, note, help, code, inline)
to @workflow/errors, and a chalk mock for readable snapshot tests.
Phase 2: Add four context-violation error classes to @workflow/core
(NotInWorkflowContextError, NotInStepContextError,
NotInWorkflowOrStepContextError, UnavailableInWorkflowContextError)
and apply them to all twelve user-facing throw sites so errors now
include docs links and a structured "what/why/fix" frame.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Address review: tighten changeset, implement ansifyName, harden Ansi
- Tighten phase 1 changeset to a single sentence (per pranaygp review) and switch to double-quoted frontmatter (per Copilot + repo convention).
- Implement `ansifyName` to actually apply dim styling to workflow/ / step/ prefixes; add an `Ansi.dim` helper to `@workflow/errors` so callers don't need to import chalk directly.
- Remove the `void getWorkflowMetadata;` workaround in context-errors.ts by dropping the unused value import (we only needed the type and symbol).
- Render the plain-Error throw in `workflow/get-workflow-metadata.ts` with `Ansi.frame` + docs link so the VM path matches the structured-class styling from the sibling step path (still uses a plain Error to avoid the module-init cycle).
- Guard `buildUnderline` against zero-length markers so a stray empty token can't produce a negative `String.repeat` count.
* Structured runtime logger metadata + fold in replay-timeout logging
Adds a `.child()` and `.forRun(runId, workflowName)` child-logger API to
the structured logger so runtime/step code doesn't have to repeat
`workflowRunId`/`workflowName`/`stepId` on every call. Normalizes error
metadata to structured `errorName` / `errorMessage` / `errorStack` fields
instead of ad-hoc `error: err.message` strings, and adds comments to
silent catches that swallow expected idempotency conflicts.
Also folds in the pending changes from #1812 so that PR can be closed:
- Standardize the console prefix to `[workflow-sdk]`.
- Split the replay-timeout log into a warn-while-retrying vs.
error-when-giving-up, and surface the underlying error when we can't
mark a timed-out run as failed.
- Include the error stack in the "Fatal runtime error during workflow
setup" log and in the top-level user-code workflow error log so the
stack surfaces in flattened log drains.
- Drop the `[Workflows] "<runId>" - ` prefix from
`buildWorkflowSuspensionMessage` — the structured logger now attaches
run context.
Supersedes #1812.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Use double-quoted changeset frontmatter per repo convention
* Add SerializationError + apply to user-facing serialization sites
Phase 4 of friendlier errors: introduce a `SerializationError` class with
an optional `hint` and a docs link (workflow-sdk.dev/err/serialization-failed),
and adopt it at every user-facing serialization boundary in @workflow/core:
- Locked ReadableStream at a workflow boundary
- Unregistered class / missing `classId` / missing `WORKFLOW_DESERIALIZE`
- Attempting to return step functions to clients or call workflow functions
directly
- Webhook `respondWith()` called outside a step
- `dehydrate*` / `getSerializeStream` failures (workflow args/return, step
args/return, stream chunks)
Internal invariants (format prefix length checks, unknown format bytes,
missing `STREAM_NAME_SYMBOL`, encryption key/size guards, etc.) now throw
`WorkflowRuntimeError` instead of plain `Error` so the classifier and logger
treat them consistently.
`formatSerializationError` now returns `{ message, hint }` so the hint
fragment can be rendered with the standard SerializationError framing
instead of being baked into the message string.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Use double-quoted changeset frontmatter per repo convention
* Presentation-only user vs SDK error attribution
Add describeError() that derives attribution and class-aware hints from
existing error classes + RUN_ERROR_CODES — no event data changes. Wire into
step failures, max-delivery exhaustion, run failures, and fatal setup errors
so terminal logs include errorAttribution and a hint for known error types.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Address review: describeError accepts precomputed errorCode + instanceof
- `describeError(err, errorCode?)` now accepts an optional precomputed
`RunErrorCode`. `classifyRunError(err)` only narrows to USER_ERROR /
RUNTIME_ERROR, so the REPLAY_TIMEOUT and MAX_DELIVERIES_EXCEEDED branches
were previously unreachable from the step / run failure log sites.
Callers that know the failure category (runtime.ts for replay timeout and
max-deliveries exhaustion) now pass the code in.
- Context-violation checks use `instanceof` against the actual classes from
context-errors.ts instead of a name-string set. Type-safe + survives
class renames.
- Wire the new hints through to the REPLAY_TIMEOUT and MAX_DELIVERIES_EXCEEDED
log sites so those branches actually render a hint now.
- 3 new tests cover the reachable code paths + precomputed-code override.
- Changeset frontmatter switched to double quotes per repo convention.
* Cosmetic consistency pass on remaining bare throws
Internal invariants now use WorkflowRuntimeError so describeError attributes
them to the SDK: missing startedAt, VM generateKey, closure-vars outside
step context, ENOTSUP. defineHook().resume() formats schema validation
failures as a readable list instead of a JSON blob.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Use double-quoted changeset frontmatter per repo convention
* Data-driven describeRunError + expose via @workflow/core/describe-error
Observability renderers read persisted run_failed / step_failed event data,
not live Error instances. describeRunError takes { errorCode, errorName }
and returns the same { attribution, hint } shape as describeError, so the
CLI and web UI can derive user-vs-SDK framing from the event log directly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Friendlier build-time errors: WorkflowBuildError class + applications
Add `WorkflowBuildError` class in `@workflow/errors` with optional `hint`
for an actionable next step, and apply it in `@workflow/builders` at
user-facing sites: failed esbuild phases, unresolved built-in steps, and
empty esbuild output now throw `WorkflowBuildError` with a hint pointing
at the likely fix. Runtime invariants remain plain `Error`.
* Polish friendlier-errors rendering: drop functionName leak, simplify docs link, redirect stack
- Drop the readonly `functionName` param-property on context-error classes so
util.inspect no longer prints a trailing `{ functionName: 'foo()' }` block.
- Replace the `DocLink` ("label: https://…") shape with a plain `DocsUrl`
template-literal type. Error output now renders a single clean line:
`docs: https://…` (new `Ansi.docs` helper) instead of the noisier
"note: Read more about foo(): https://…".
- Add throw helpers (`throwNotInWorkflowContext`, etc.) that call
`Error.captureStackTrace(err, stackStartFn)` on V8 engines so the top frame
of the thrown error points at the user's call site instead of at the gate
function inside the framework. Callers pass themselves as the boundary.
- Refactor `defineHook()` (both root and `/workflow`) to use named function
closures rather than `this.create`/`this.resume`, since the stack redirect
relies on a stable function identity that survives destructuring.
- Update context-errors.test.ts to snapshot the new `docs:` framing and to
add a regression test asserting the top stack frame is the user call site.
* Consolidate friendlier-errors stack: fix ANSI leak + non-retry semantics
Addresses PR review feedback across the 8-phase friendlier-errors stack and
fixes issues surfaced by manual testing (createHook() inside a step):
- ANSI no longer leaks into .message / .stack. Context-violation errors
now store plain text on .message and render the colored framed form
lazily via [util.inspect.custom] / toString(). Structured logs, log
drains, CBOR-serialized events, and JSON payloads no longer contain
raw \x1B[...m bytes.
- Context violations are now fatal. ContextViolationError sets
fatal = true; FatalError.is(err) recognizes any error with a
fatal: true own property. Calling createHook() from a step no longer
burns three retry attempts on a guaranteed-to-fail context violation.
- Ansi helpers moved to @workflow/errors/ansi subpath so imports from
@workflow/errors no longer pull chalk into consumers that only want
error classes (addresses reviewer VaguelySerious).
- Shared redirectStackToCaller helper in packages/core/src/capture-stack.ts,
used by both context-errors.ts and workflow/get-workflow-metadata.ts
(addresses Copilot review on #1849).
- Structured framed content: ContextViolationError now takes a structured
FramedContent (title segments + detail branches) and renders plain/pretty
from the same source of truth.
Tightens the eight existing phase changesets to 1-2 sentences each and adds
four new scoped changesets (errors-ansi-subpath, context-errors-plain-message,
context-errors-fatal, capture-stack-shared) for the followup fixes, so the
final changelog history stays readable.
* test: update step-handler mocks for scoped forRun() logger
The runtime logger now uses .forRun(runId, name, {stepId, stepName})
to attach scope context, so 409-handling log calls no longer repeat
{workflowRunId, stepId} in every metadata bag — those live on the
scoped logger instance. Update the mock to return itself from forRun()
and tighten assertions to check both the log args (errorName/errorMessage)
and the forRun() scope.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Mark SerializationError fatal + route dehydration through step-failure path
SerializationError now carries readonly fatal = true. Step-return
dehydration is wrapped inside the user-code try/catch so that the
resulting error flows through userCodeFailed → step_failed →
FatalError.is() short-circuit instead of bubbling up as HTTP 500 and
triggering a queue retry loop. Retrying a step that returned a non-POJO
is guaranteed to fail the same way, so this saves ~20s and 3 near-
identical error blocks per serialization failure.
* Add logging snapshot tests + manual-test artifacts
Snapshot tests lock in the exact shape of:
- describeError() payloads (attribution, errorCode, hint) for every
classification — plain Error, SerializationError, context-violation,
WorkflowRuntimeError, REPLAY_TIMEOUT, MAX_DELIVERIES_EXCEEDED.
- The scoped-logger call signature for the two canonical runtime
failure paths (fatal-bubble and hit-max-retries), so refactors of
forRun() / child() metadata merging can't silently change what users
see in their log drains.
SerializationError now also has a direct test for readonly fatal=true
+ FatalError.is() recognition.
pr-artifacts/ contains real log-output snapshots from running the
nextjs-turbopack workbench against five error scenarios. These are
reference material for reviewers and are flagged to be removed before
merge.
* Readable step-fatal logs: inline stack + friendly step/workflow names
The step-level fatal-error log used to embed the full stack trace inside
an `errorStack` string field in the metadata object, so util.inspect
rendered it as a quote-escaped, line-continuation blob when the log
hit the terminal — unreadable in practice. Move framing + stack into
the log *message* (matching the workflow-level log in runtime.ts) and
keep the metadata object compact with only the indexable structured
fields (`errorAttribution`, `errorName`, `errorMessage`, `hint`,
IDs). Log drains still get the same keys; humans now see a readable
stack trace.
Also introduce `formatStepName` / `formatWorkflowName` in
`@workflow/utils` that render machine names
(`step//./workflows/1_simple//add`) as `add (./workflows/1_simple)` in
log framings, using the existing `parseStepName` / `parseWorkflowName`
parsers. Applied to step-fatal, hit-max-retries, exceeded-max-retries,
and workflow-threw log sites.
Artifacts in pr-artifacts/ updated to show the new output shape, and
renamed .log → .md since they're Markdown and IDE previews are nicer
that way.
* Opinionated pretty formatter for runtime structured-log metadata
Replace util.inspect's default object dump (which quote-escapes
multi-line stacks and paragraph hints into a single-line JSON-y blob)
with a workflow-aware formatter that composes the entire log line
into a single string passed to console.error / console.warn.
Highlights of the new output:
- Per-run / per-step IDs render with their parsed friendly names so
users see `wrun_… · simple (./workflows/1_simple)` instead of just
the raw `workflowName: 'workflow//./workflows/1_simple//simple'`.
- Color-coded attribution badge (user error red / sdk error magenta)
paired with the error class in bold.
- Hints render as a paragraph under `hint:` rather than a backslash-
`\n`-escaped string.
- Drops redundant fields (errorStack always; errorMessage when it's
already in the parent message) to avoid double-printing.
- Unknown fields fall through as a sorted `key value` tail so we
never silently drop log information.
@workflow/errors/ansi gains bold/red/magenta helpers used by the
formatter. The web / web-shared packages don't consume stderr — they
read structured event payloads from the World event log — so this is
presentation-only at the runtime layer.
* ci(benchmarks): disable pnpm cache for getCommunityWorldsMatrix
The job never runs `pnpm install` (it just calls `node` against a
checked-in script), so the pnpm store path never exists. The post-job
`actions/setup-node@v4` cache-save then fails with `Path Validation
Error: Path(s) specified in the action for caching do(es) not exist`
and red-X's the entire job even though the matrix step succeeded.
The setup-workflow-dev composite already has a `cache-pnpm` opt-out
input for this exact case — wire it through here.
* Address PR review comments: inspect dedup, cause leak, retry-loop tests
- ContextViolationError: util.inspect(err) duplicated every framed detail
line because the stack-tail strip only sliced the first message line.
V8's Error.stack reads `Name: messageLine1\n messageLine2\n at ...`,
so for our multi-line `title\n╰▶ docs: …` messages every detail line
was getting prepended twice (once in the pretty form, once via the
unsliced message tail). Count the actual message lines and slice past
all of them. Repro test asserts `╰▶ docs:` appears exactly once.
- WorkflowError: stop assigning `cause: undefined` as an enumerable own
property when no cause is provided. Subclasses (every error in this PR)
inherit the parent constructor; the unconditional assignment polluted
`util.inspect(err)` output with `{ cause: undefined, … }` on every
no-cause instance. The `super(...)` call already conditionally sets
`.cause` non-enumerably when `options.cause` is provided.
- step-handler.test.ts: add a regression-gate suite that exercises the
fatal-vs-retryable retry-loop wiring directly. Asserts that an error
with `fatal: true` produces exactly one `step_failed` event with no
`step_retrying`, and that a non-fatal `Error` retries via
`step_retrying` on early attempts and emits `step_failed` once the
retry budget is exhausted. Catches the silent-regression case where
`fatal = true` is removed from a context-violation error class but
the `FatalError.is()` unit tests stay green.
* Consolidate changesets + remove pr-artifacts
Address review feedback to drastically shorten the changesets — fold
the 15 file-by-file entries into a single user-facing changeset for
@workflow/core / errors / builders / utils. Also drop the pr-artifacts/
folder (reviewer-only log captures, no longer needed).
* Polish runtime error logging: layout, stack trim, hint consolidation
Five user-driven fixes from manual smoke-testing of #1849:
1. Logger layout. composeLogLine() now puts the structured-fields block
(attribution badge, run/step IDs, error code) **between** the framing
line and the stack body, instead of after it where 30+ lines of stack
buried the most useful information. The framing stays at the top,
stack at the bottom, structured info readable at a glance.
2. Stack trim. Drops framework-internal frames (`node_modules/.pnpm/`,
`node:internal/`, Turbopack-bundled `node_modules__pnpm_*` chunks,
`_next_dist_*` chunks) and caps the surviving frame count at 6
so the stack stays compact even on heavy async wrappers. Suppressed
runs emit one summary line so users know the trim happened.
3. Wrapper-route noise. The nextjs-turbopack workbench's start route
was catching `WorkflowRunFailedError` rejection on
`Promise.race([readLoop(), run.returnValue])` and re-logging it via
`console.error('Error in workflow stream:', error)` plus
`controller.error(error)` — which then triggered Next.js's
`⨯ failed to pipe response` overlay. The SDK already logs the
failure cleanly upstream and the runId is on the response header, so
the wrapper now closes the SSE stream cleanly on
WorkflowRunFailedError.
4. Consistent framed `╰▶ hint:` / `╰▶ docs:` layout for all errors
that carry a hint or docs slug. WorkflowError, SerializationError,
and WorkflowBuildError now share one `appendFramedDetails` helper
matching the box-drawing structure that ContextViolationError
already used. Was: blank-line-separated `Learn more: <url>`. Now:
one tree, indistinguishable from context-violation rendering.
5. Drop the duplicate logger-side `hint` field. Hints now live on the
error message only — actionable hints get serialized into the event
log, rehydrated on the workflow side, and shown in observability
automatically. The previous logger-only hint duplicated stderr but
never made it past the step boundary.
Updated SerializationError hint to point at the foundations doc
("Ensure you're returning workflow serializable types. Check the
serialization docs to see what's serializable:
https://workflow-sdk.dev/docs/foundations/serialization") instead
of the hardcoded `(plain objects, arrays, primitives, …)` list,
which drifted out of sync as the supported types grew. Same hint
reuses for step args, workflow args/return, stream messages, and
any other site that goes through `formatSerializationError`.
Also retitled the retry summary `3 retries` → `3 max retries` since
"3 retries" next to "4 attempts" was ambiguous (already-happened vs.
budget).
* Trim error-card title + drop machine step name from persisted error
- ErrorStackBlock (web observability): show just the first non-empty
trimmed line of the error message in the card title with single-line
truncation. Multi-line messages (`Failed to serialize step return
value\n╰▶ hint: …`) were rendering the entire framed body in the
title, pushing the copy button off-screen and burying the
scannability of the headline. Full message stays in the body via
the stack (V8 prepends `Name: message` to `Error.stack`), so no
information is lost; hover-tooltip exposes the full title text.
- Persisted error message: drop the `Step "step//./.../foo"` machine
name from `Step failed after N retries: …` and `Step exceeded max
retries (…)` strings. Observability already attributes the event
to a specific step via the UI tree, and the CLI logger emits the
friendly `Step foo (./...) hit max retries` framing on its own
line. Embedding the raw `step//./...` machine name in the persisted
message text was duplicate noise.
* Update .changeset/friendlier-errors.md
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
* Update .changeset/pretty-log-format.md
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
* Update SerializationError snapshot tests for slug-less message
The class no longer attaches a slug-based `╰▶ docs:` line — the
foundations URL is embedded directly in the hint via the
`formatSerializationError` helper in @workflow/core. Update the test
expectations accordingly:
- bare-title case is now a single line (no docs link)
- hint case renders one `╰▶ hint: …` branch (no second branch)
* Update serialization.test.ts hint assertions for foundations URL
Four `should throw error for an unsupported type` cases were still
asserting on the old hardcoded type list. Update to the new hint
phrasing that points at the foundations doc, matching the change in
`formatSerializationError` (`packages/core/src/serialization/errors.ts`).
---------
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
* Pass stale-banner via path: to sticky-pull-request-comment instead of message:
The 'Update existing test comment with stale warning' step inlined the
previous comment body via ${{ steps.get-comment.outputs.previous-results }}
into the action's `message:` input. As the test matrix grows, the
resulting argv can exceed ARG_MAX and the action fails with
'Argument list too long' — observed on a feature branch where the
matrix doubled.
Write the rendered stale-banner message to
$RUNNER_TEMP/stale-comment.md in the github-script step and pass the
path to sticky-pull-request-comment via its `path:` input instead.
This is robust to any future matrix size.
* Apply same fix to benchmarks.yml
Same ARG_MAX hazard exists in the benchmark workflow's stale-warning
step. Apply the identical `path:`-instead-of-`message:` refactor:
- The github-script step now writes the rendered stale-banner to
$RUNNER_TEMP/stale-comment.md and exposes the path as a step output.
- The sticky-pull-request-comment 'Update existing benchmark comment
with stale warning' step uses `path:` instead of inlining
${{ steps.get-comment.outputs.previous-results }} via `message:`.
The final 'Update PR comment with results' step in this workflow
already used `path: benchmark-summary.md`; only the stale-banner
update was inlined.
* Use `github.run_started_at` for stale-comment timestamps
The 'Started at:' label was sourced from `github.event.pull_request.updated_at`,
which is the PR metadata-update timestamp — not the workflow run start
time. That made the displayed timestamp:
- coupled to PR edits (label changes, description edits, etc.) rather
than to the actual CI run, and
- stale on workflow re-runs (an empty re-run would still show the
original PR-update time).
Switch all six occurrences across `tests.yml` and `benchmarks.yml` to
`github.run_started_at`, the canonical "this CI run started at"
timestamp.
* ci: switch Vercel deployment-protection bypass to OIDC Trusted Sources
The e2e, benchmark, and docs-smoke CI jobs previously used the static
`VERCEL_AUTOMATION_BYPASS_SECRET` deployment-protection bypass token
to reach protected Vercel deployments. Switch them over to the new OIDC
Trusted Sources flow: the GitHub Actions runner mints a short-lived
OIDC token via `core.getIDToken()` and forwards it on requests in the
`x-vercel-trusted-oidc-idp-token` header.
Each workbench project (and `workflow-docs`) has been configured with a
matching trusted-source rule:
aud=https://github.com/vercel, repository=vercel/workflow
The shared header helper now lives at `scripts/trusted-sources-headers.mjs`
and is imported by both the e2e/bench tests and the docs smoke script,
removing the previous duplication.
* rename to VERCEL_OIDC_TOKEN and wire through world-vercel
- Rename the env var from VERCEL_TRUSTED_OIDC_TOKEN to VERCEL_OIDC_TOKEN
to match Vercel's convention (also read by @vercel/oidc's
getVercelOidcToken()).
- In @workflow/world-vercel, replace the legacy
VERCEL_WORKFLOW_SERVER_PROTECTION_BYPASS / x-vercel-protection-bypass
flow with VERCEL_OIDC_TOKEN / x-vercel-trusted-oidc-idp-token. The
trusted-source header is attached on every outbound workflow-server
request (both proxied through api.vercel.com and direct).
- Drop the bypass header from the encryption-key and
resolve-latest-deployment fetches: those go to api.vercel.com which
is public.
- Drop VERCEL_WORKFLOW_SERVER_PROTECTION_BYPASS plumbing from tests.yml.
- Update the pending world-vercel changeset to describe the final
trusted-sources flow.
* .
* .
* ci: add statuses:read permission for wait-for-vercel-project action
The action queries /commits/{sha}/status (Commit Statuses API) in addition
to the Deployments API, in order to extract the Vercel `dpl_...` ID. With
an explicit permissions block in place, GITHUB_TOKEN now needs
`statuses: read` or the action 403s when resolving the deployment ID.
Reported by Copilot review on #1882.
* ci(docs): log status code and body when waitForServer times out
Helps diagnose deployment-protection / OIDC-trusted-source bypass
failures (e.g. SSO redirects) on the workflow-docs preview.
* ci(docs): log OIDC token claims (aud, repository, etc.) for diagnostics
Helps determine whether the bypass is failing because of missing
trusted-source config, claim mismatch, or audience mismatch.
* ci(docs): add curl debug step to verify OIDC header reaches Vercel
* .
* ci: remove debug logging now that trusted-sources config is correct
The fetch-failure root cause was the trusted-sources rule format: the
labs workbench projects had been PATCHed with just `to.slugs` (no
`preset`), but Vercel's edge requires the dashboard-form-style
`to.preset: 'all-custom'` field plus `development` in the slug list to
match incoming requests. After re-PATCHing all projects with the
correct format, the bypass works end-to-end.
* ci(docs): debug — test trusted-sources bypass against docs and labs deployments
Trying repository_owner claim added to one labs project to see if that
fixes the bypass.
* ci(docs): revert curl debug step
The GitHub Actions OIDC trusted-sources bypass returns 401 on all tested
projects regardless of claim configuration (including workflow-docs which
was set up via the dashboard). This is not a per-project config issue.
Need to investigate with Vercel team before continuing.
* ci(docs): probe trusted-sources bypass and surface x-vercel-id
Adds a debug step that does two HEAD requests against the docs preview
deployment (with and without the OIDC trusted-sources header) and prints
the response status line plus `x-vercel-id` for each. The proxy-side
trusted-sources changes for GitHub Actions OIDC tokens are rolling out
gradually (~12+ hours), so the edge-node identifier in `x-vercel-id`
helps explain why a request might succeed or fail during the rollout
window.
Also includes `x-vercel-id` in the `waitForServer` timeout error so
post-mortem analysis of failing runs has the same edge-node info.
* ci(docs): drop trusted-sources curl probe — bypass works once proxy fix reaches the serving edge node
The probe served its purpose: confirmed the bypass is functional once
the request lands on a region that has the proxy-side trusted-sources
fix rolled out. The waitForServer error message still surfaces
x-vercel-id for any future rollout-window debugging.
* .
* world-vercel: log outbound OIDC token claims once per process
Adds a one-shot diagnostic that prints the non-sensitive claims of the
OIDC token (`iss`, `aud`, `owner_id`, `project_id`, `environment`,
`sub`, `scope`, `exp`) on the first request that uses bearer auth.
This is invaluable for debugging Vercel deployment-protection
trusted-source rule mismatches: a 401 from the edge tells you nothing
about why the rule didn't match, and the token's claims are the only
thing that determines that. The signature is never logged.
Gated to once per process — Vercel-issued tokens are process-stable for
the lambda's lifetime so further log lines would just be redundant
spam.
* world-vercel: route trusted-sources header through getVercelOidcToken()
The Authorization bearer correctly preferred config.token (a static
Vercel auth token from CLI / Actions runner) and fell back to
getVercelOidcToken() inside a Vercel function. But the trusted-sources
bypass header (x-vercel-trusted-oidc-idp-token) was being read directly
from process.env.VERCEL_OIDC_TOKEN inside getHeaders(). That env var is
the bake-time token, frozen at deployment-creation time — on a project
that has been redeployed after a settings change, it carries stale
claims (e.g. an iss from when the project was briefly in 'global' mode)
that no longer match the workflow-server's trusted-sources rule.
Move trusted-sources header attachment from getHeaders() (sync) to
getHttpConfig() (async) and source it from getVercelOidcToken(). That
function reads getContext().headers['x-vercel-oidc-token'] first — a
freshly minted per-request token that always reflects current project
settings — and only falls back to the env var when that header is
missing.
Bearer auth source remains config.token-first.
Also expand the diagnostic to log claims from BOTH the per-request OIDC
token AND the bake-time env var so the divergence is visible in logs
when debugging future trusted-source mismatches.
Removes the now-misleading getProtectionBypassHeader() helper (its
'read env var directly' semantics were exactly the bug).
* world-vercel: skip OIDC trusted-sources header on proxied path
The two outbound flows have different auth requirements:
1. Proxied (usingProxy=true) — calls api.vercel.com/v1/workflow.
Public endpoint, authenticated with a static Vercel auth token via
config.token. The api-workflow proxy mints its own OIDC token
before forwarding to workflow-server, so the trusted-sources
bypass header on the SDK→proxy hop is meaningless. CLI, GitHub
Actions, and other API-client callers take this path.
2. Direct (usingProxy=false) — runs inside a Vercel deployment
talking straight to workflow-server. workflow-server validates a
Vercel OIDC bearer; Vercel's edge validates the trusted-sources
header. Both must come from getVercelOidcToken() (the per-request
fresh token), not process.env.VERCEL_OIDC_TOKEN (the bake-time
token that can be stale after a project config change).
Previously getHttpConfig attached x-vercel-trusted-oidc-idp-token on
both paths whenever getVercelOidcToken() resolved. That accidentally
forwarded the GitHub Actions OIDC token (when wired into
VERCEL_OIDC_TOKEN by the test runner) onto every SDK→proxy request,
which is harmless but wrong-by-design — the proxy is public, doesn't
look at that header on its inbound side, and the GHA token isn't its
intended audience.
Bearer auth source rules:
- Proxied: only config.token. (No fallback to OIDC; that auth
pathway doesn't go through the proxy's auth checks.)
- Direct: config.token (for tests / local dev), falling back to
getVercelOidcToken() (for Vercel-runtime calls).
* world-vercel: throw if proxied path is hit without a Vercel auth token
The api-workflow proxy authenticates the caller with a regular Vercel
auth token (not OIDC), so reaching the proxied path with no
config.token is always wrong: the proxy will reject the request and
the SDK caller would see an opaque 401 with no actionable hint.
Throw at config-resolution time with a clear message that points to
the WORKFLOW_VERCEL_AUTH_TOKEN env var the SDK reads from. Adds tests
covering both the no-token-throws case and the with-token-attaches-
bearer-and-skips-trusted-sources case.
* test(e2e): include x-vercel-id in startWorkflowViaHttp error message
When the trusted-sources bypass returns 401, the error message now
surfaces the response's x-vercel-id header so we can identify which
edge node served the failure. Helps distinguish proxy-rollout
incompleteness from actual config errors during incremental
rollouts of edge-side changes.
* ci: mint GHA OIDC tokens on demand to survive 5-minute expiry
GitHub Actions OIDC tokens have a hard 5-minute lifetime that cannot be
extended (no API to ask for a longer TTL — exp is always iat + ~300s).
Pre-minting once at the start of the job and shipping the result down
to the test runner via env var means tests that run late in the suite
hit an expired token and 401 on /api/trigger-pages (and any other
trusted-sources protected endpoint).
Move minting into scripts/trusted-sources-headers.mjs:
- getTrustedSourcesHeaders() is now async.
- It calls the runner's ACTIONS_ID_TOKEN_REQUEST_URL endpoint directly
(the env vars GHA exposes when permissions: id-token: write is on)
and re-mints 60s before the cached token's exp.
- Falls back to process.env.VERCEL_OIDC_TOKEN for non-GHA contexts
(Vercel runtime, local dev).
Workflow files drop the now-redundant 'Mint OIDC token' step and the
VERCEL_OIDC_TOKEN env-var passthrough on the test step. The runner env
vars propagate to subsequent steps automatically.
Updates all 17 callers in e2e.test.ts / bench.bench.ts / utils.ts /
docs/scripts/check-docs-smoke.mjs to await the now-async call.
* address PR #1882 code review
- Drop `statuses: read` from the three workflow permission blocks (the
wait-for-vercel-project action works without it on a public repo).
- Revert the `x-vercel-id` debug logging in `startWorkflowViaHttp`.
- Delete `packages/world-vercel/src/jwt-claims.ts` (debug-only helper).
- Drop the JWT claims diagnostic logging from `getHttpConfig`.
- Tighten the auth-flow comment in `getHttpConfig` and remove the
historical 'no longer attaches' note from `getHeaders`/its test.
- Restore `.changeset/world-vercel-protection-bypass.md` (already
shipped in a beta release per .changeset/pre.json).
- Trim the `.changeset/world-vercel-trusted-sources.md` description to
one short paragraph.
* docs(AGENTS): document local VERCEL_OIDC_TOKEN via vercel env pull
Configured trustedSources.projects on all 11 workbench app projects so
each one accepts a Vercel-issued OIDC token from any of the others. A
developer running e2e locally can now do `vercel env pull` from any
workbench app's directory and use the resulting VERCEL_OIDC_TOKEN to
bypass Deployment Protection on any of the workbench preview/prod
deployments — no need to disable protection on the project just to run
the suite locally.
A recurring Turbopack-on-Windows bug causes the dev server to enter a
'MODULE_UNPARSABLE' state during HMR in dev.test.ts, after which every
request returns 500. The remaining e2e suite then polls stuck workflows
for 60s each, burning the full 30-minute job window before getting
cancelled (~50% of recent main runs).
Bail out of the Windows e2e job as soon as dev.test.ts fails, and
health-check the dev server before kicking off test:e2e so any other
silent breakage is surfaced quickly instead of via a 30-minute timeout.
The repo enforces "Commits must have verified signatures" via an
org/enterprise-level ruleset, which blocks unsigned commits pushed via
the Git CLI by GITHUB_TOKEN. Switching the changesets action to
commitMode: github-api makes commits GPG-signed by GitHub.
Pranay Prakash