Commit Graph

78 Commits

Author SHA1 Message Date
Shalabh Chaturvedi 264ddff67b Add WebSocket transport for step-execution event writes (opt-in) (#3084)
* sdk side for workflow server websockets

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* hardcoded workflow server

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* debug info

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* more debug

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* remove unnecessary debug

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* default on websockets, and override url

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* fix for missing funcs

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* make websockets opt outo

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* enable ws again

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* [revert later] reduce test to single test, test both http and ws at the same time

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* empty

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Run full suite with and without ws

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* empty

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* improve e2e test

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* minimize tests

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* fallback to http when proxy present

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* default to websockets, remove matrix

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* remove smoke test

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* update to new protocol

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* adjust for new protocol (runid in path)

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* fix ws transport error

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* blank

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* fix ws dep

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* fix ws external: only accelerators, not ws itself

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* add dedicated WS-transport e2e job; flip WS default back to opt-in

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* build all packages before local vercel build (needs workflow/nitro on disk)

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* force NITRO_PRESET=vercel for the local vercel build step

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* install vercel CLI once instead of npx-ing it per command

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* add changeset for WS events transport

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Harden the WS events transport and gate its e2e jobs

Follow-ups from review of the WS transport.

CI: `e2e-vercel-ws-transport` was wired into the `summary` job but not
into `e2e-required-check`, so all three WS jobs could fail while the
required check stayed green. Added to both branches of the status
validation — including the `workflow-server-test` label branch, where
the job runs under the same gating as `e2e-vercel-prod`.

Transport:

- `reqId` and the pending-reply map are now per connection rather than
  per transport. The protocol defines `reqId` as a per-connection
  counter, so a reconnected socket restarts at 1; with one shared map
  that collided with the previous socket's still-registered waiters.
  It also makes the superseded-socket guard structural instead of
  something the close path has to remember.
- Post-open socket errors are no longer silent. The only `'error'`
  listener closed over the connect promise's `reject`, already settled
  once `'open'` fired, so every broken pipe / 1009 / protocol fault was
  swallowed and its requests hung with no per-request timeout to save
  them. Now logged, and the connection is torn down.
- An unexpected close reconnects eagerly instead of waiting for the next
  write, since a socket breaking mid-run means more writes are coming.
  Bounded by exponential backoff, an attempt cap that falls back to
  lazy reconnect, a bail-out when a newer socket is already live, and an
  `unref()`ed timer so a backoff window can't delay handler exit.
- `ws.send()` failures reject their request. `send()` doesn't throw on a
  non-OPEN socket — it reports through a callback we weren't passing —
  so the request just sat in `pending` forever.
- The reserved `reqId: -1` malformed-frame reply and undecodable frames
  are logged loudly instead of dropped.
- Auth headers resolve once per socket via a thunk, not once per event.
  The bearer only rides the upgrade, so the old code awaited
  `getVercelOidcToken()` on every write and discarded all but the first.
  Re-resolving on reconnect also means a new socket gets a fresh token.

Adapter: a reply with no numeric status now fails closed. Defaulting to
200 reported a write as applied whenever the client met a frame it
didn't understand — and the protocol is explicitly designed to grow new
response variants.

Tests: 24 new unit tests over the paths the e2e suite can't reach on
demand (send failure mid-flight, error after open, late close from a
superseded socket, reconnect backoff and give-up, sentinel/undecodable
frame logging, one-token-per-socket) plus the adapter's fail-closed and
typed-error mapping.

Co-Authored-By: Shalabh Chaturvedi <shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* ci: re-trigger to confirm the prior e2e failures were flake

No code change. The 5 failures on 0bb21e7 clustered in a ~20s window
across HTTP-path jobs (example/nuxt on the same test, sveltekit on a
timeout) and one WS job (sleepingWorkflow's clock-skew assertion), which
points at the environment rather than the transport changes. Re-running
to confirm.

Co-Authored-By: Shalabh Chaturvedi <shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Add WS wire-contract conformance tests and pin the transport gate

Three gaps in the existing coverage.

**The HTTP path was already covered** — `events-v4.test.ts` has 22 tests,
including five directly on `createWorkflowRunEventV4` over HTTP (alias
URL, frame meta contents, response decoding, skipPreload/stateUpdatedAt
forwarding). Those run with the gate unset, so they do confirm the
two-branch refactor didn't disturb HTTP. No new tests needed there.

**But nothing pinned the gate itself.** Every HTTP assertion stays green
if the default flips to WS, because the transports are built to be
indistinguishable at the result layer — and an earlier revision of this
branch did flip the default deliberately, for benchmarking. Added tests
for `isWsEventsTransportEnabled()` across values, and one that drives a
real HTTP request through a MockAgent while asserting the WS transport is
never constructed.

**Nothing verified the bytes.** `ws-transport.test.ts` replies with
whatever the test hands it, which proves the client's lifecycle but not
that its frames are what workflow-server accepts. That's the drift the
spec doc exists to prevent, and it already happened once: event meta flat
on the frame where the server wanted it nested under `event`, with both
sides' tests passing.

`ws-protocol-conformance.test.ts` pairs the real client stack (through
`createWorkflowRunEventV4`) with a fixture mirroring the server route's
per-message handling: decode one frame, validate against a local copy of
`WsRequestFrameSchema`, dispatch, encode the reply the way `replyMeta`
does. `experimental_upgradeWebSocket` needs a real Vercel runtime, so the
socket is faked — everything above it is genuine.

Covers: the frame shape the server accepts (and that `reqId`/`type`/
`runId` don't leak into the event meta), payload passthrough, exactly one
frame per message, 409 → the same typed error HTTP raises, fail-closed on
an unknown reply variant, and reqId correlation across concurrent writes.

Plus golden byte fixtures, since the schema copy is the one thing here
that can silently drift. This is the "golden-frame interop test" the
server spec lists as an open gap; the matching half still needs to land
in workflow-server.

Verified the conformance suite is not vacuous: flattening the client's
frame meta fails 5 of its tests.

Co-Authored-By: Shalabh Chaturvedi <shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* match the HTTP RetryAgent's transient-failure policy on WS

HTTP event writes go through an undici RetryAgent (RETRY_AGENT_OPTIONS):
5xx and transient connection errors are retried in-process, honoring
Retry-After. The WS path never touches undici, so it shipped with no
transient-failure handling at all — a single 503 or a mid-write reset
surfaced straight to the step runtime and cost a whole step retry where
HTTP would have absorbed it in milliseconds.

That gap is invisible in a passing test run: writes still succeed, they
just cost far more. So copy the policy rather than reinvent it —
[500, 502, 503, 504] plus transport failures, undici's default backoff,
Retry-After honored, and 429 deliberately excluded for the same reason
RETRY_AGENT_OPTIONS excludes it (a firewall challenge this client cannot
solve, which in-process retries only amplify).

Adds WsTransportError so retryability is a typed property of the failure
rather than something the adapter infers by string-matching. Splits
resolveWsTransport()/wsReplyStatus() out of postEventFrameOverWs so the
retry loop stays readable.

The existing "fails closed on an error frame" test used status 500,
which is now absorbed by the retry — switched to 403 so it keeps
testing fail-closed rather than accidentally testing no-retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* lazy-load `ws` so the default HTTP path never evaluates it

events-v4.ts imports ws-transport.js unconditionally — the transport
gate is a runtime branch, not a build-time one — so a top-level
`import { WebSocket } from 'ws'` put `ws` and its optional native
accelerators on the module-init path of every deployment, including the
overwhelming majority that never opt in and never open a socket.

Defer it to the first connect, memoized as a promise so concurrent
first connects share one import. WebSocket.OPEN becomes an inlined
constant so the readyState check doesn't pull the module in just to
read it off the constructor.

This does NOT remove the need for the bufferutil/utf-8-validate
externals this branch also adds: webpack and Rollup both statically
follow a dynamic import(), so the build-time story is unchanged. What
it buys is that a deployment which never enables the transport never
*evaluates* `ws`, so a mis-bundled accelerator can't break it.

The test lives in its own file because vitest caches a vi.mock factory
result for the life of the module registry — once any test in a file
has connected, the factory never runs again and the counter can't
distinguish "loaded lazily" from "loaded at import".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* release idle WS transports instead of renewing them forever

The transports map was never pruned and WsEventsTransport had no way to
close. Combined with eager reconnect that made a connection immortal by
construction: the server drains at its own maxDuration and closes, the
client immediately reopens, and the server pins a fresh invocation — for
a run that finished long ago. A warm container ended up holding a live
socket, and a live server invocation, for every runId it had ever
served. workflow-server#683 already lists "one invocation stays resident
per run rather than per write" as a known gap; this made it "per run,
forever".

Add close() plus a 60s idle release. There is no "run complete" signal
to hang teardown off — the events adapter is a stateless per-write call
— so idleness is the available proxy. 60s sits well below the server's
~680s drain deadline, so the client releases rather than the server
reclaiming, and well above the gap between steps of an active run.

scheduleReconnect() now bails when closed: close() closes the socket,
which fires the same close handler an unexpected drop would, and without
the guard the transport would instantly reconnect what it just released.

request() revives an idle-closed transport rather than failing the
write, re-registering itself only if nothing newer has claimed the map
slot. Eviction therefore costs one handshake, not an error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* refresh the bearer on an auth_expiry drain

workflow-server#683 tags a drain frame with why it is closing:
max_duration means the socket aged out and a plain reconnect is right,
auth_expiry means the *bearer* ran out and reconnecting with the same
one just earns a 401. This client logged the drain and ignored the
reason, so against #683 an auth_expiry drain would burn all five
reconnect attempts against a token the server had already rejected, then
give up.

Parse the reason (absent reads as max_duration, so this stays correct
against the currently-deployed server) and thread forceRefresh through
the getHeaders thunk, which triggers @vercel/oidc's refresh path via a
wide expirationBufferMs.

Worth being precise about when that can actually help. getVercelOidcToken
resolves getContext().headers['x-vercel-oidc-token'] ?? env, and
refreshToken() only writes the env var — the request-context header
wins. So inside a deployed function there is genuinely no fresher token
mid-invocation and the refresh is a no-op; outside one (CLI, local dev,
a long-lived server) it works.

That makes the guard the load-bearing half: if the re-resolved bearer is
byte-identical, decline to reconnect, say so, and wait for the next
write — which usually arrives on a new invocation carrying a new token.
That failure is marked non-retryable so the retry loop doesn't spin on
it either.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* document the WS path's instrumentation gap

The HTTP branch goes through fetchV4 -> instrumentedFetch, which is not
just a fetch wrapper: it opens the OTEL CLIENT span, injects trace
context, sets the cache-bust header, emits the DEBUG logs, and routes
through the global fetch that Vercel's observability "outgoing requests"
view instruments. The comment on fetchV4 records why that matters —
bypassing it via undici.request() is exactly what once made v4 event
traffic disappear from the log viewer.

The WS branch bypasses all of it. With the flag on, per-event writes
have no client span, propagate no trace context to workflow-server, and
don't appear in the outgoing-requests view; the server's own
transport-tagged request metrics are the only remaining signal.

That's acceptable for an opt-in POC behind a flag and unacceptable as a
default, so write it down where someone deciding to flip the default
will read it: instrumenting the transport is a prerequisite for that,
not a follow-up nicety.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* ship the ws-accelerator externals instead of documenting a workaround

`bufferutil` and `utf-8-validate` are optional native accelerators for
`ws`, and neither is installed by default. Every bundler has to be told
to leave them alone, for two different reasons: Rollup/Vite/Nitro fail
the build outright (`Could not resolve "bufferutil" imported by "ws"`),
while webpack bundles the JS wrapper without its native `.node` binding
and throws `bufferUtil.mask is not a function` at runtime.

The webpack half shipped in `@workflow/next`. The Rollup half only
existed in `workbench/vite` and `workbench/tanstack-start` as
`nitro.rollupConfig.external` — app configs, not shipped code. So a real
user of `@workflow/vite`, `@workflow/nitro`, `@workflow/nuxt`,
`@workflow/sveltekit` or `@workflow/astro` hit the same build failure the
workbench had already worked around, and had to rediscover the fix.

Fix it where it propagates: `workflowTransformPlugin` in
`@workflow/rollup`, which all of those integrations already install. It
is already the home of exactly this pattern for the optional
`@opentelemetry/api` peer, so this sits next to its closest precedent.

Note the treatment is deliberately the inverse of the OTEL one, which is
externalized only when it *can't* be resolved. The OTEL API must load
for tracing to work, so a self-contained output has to bundle it when
present. These accelerators must specifically NOT load — they are a
performance nicety with a correct try/catch fallback in `ws` — so
unconditional external is both simpler and safer than risking a
half-bundled native module.

The two workbench configs drop their local copies, which is what proves
the shipped fix actually works rather than being masked by them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* one retry policy for both transports, and no unanswerable waiters

Two review findings on the WS events transport.

**Retry belongs to `event-retry.ts`, not the adapter.** The WS path had its
own retry loop, justified as mirroring undici's `RetryAgent`. That
justification was wrong: `RetryHandler` defaults `methods` to GET/HEAD/
OPTIONS/PUT/DELETE/TRACE and nothing overrides it, so the `RetryAgent` never
retried an event POST on either transport — which is precisely why
`event-retry.ts` exists.

Worse, that loop sat *inside* `withEventPostRetry`, so it defeated a
compile-checked safety gate: `EVENT_RETRY_ELIGIBILITY` marks `step_started`,
`step_retrying` and `hook_received` non-retryable (a replayed `step_started`
double-increments `attempt`), and those frames were re-sent up to five times
before the gate ever saw a failure. For eligible types the two loops
multiplied: 3 outer attempts x 6 inner, with an inner backoff reaching 30s
against an outer base deliberately set to 100ms.

`postEventFrameOverWs` now makes one attempt and translates failures into the
vocabulary that policy already speaks — a transport failure becomes a
`WorkflowWorldError` with `code: 'TRANSPORT'`, exactly as `utils.ts` does for
a failed `fetch`, and `isRetryableEventPostError` gains one clause keyed on
that code. `WsTransportError` loses its `retryable` flag; its only consumer
was the deleted loop.

Two deliberate consequences. The code-keyed clause broadens HTTP in-process
retry to `UND_ERR_CONNECT`, `UND_ERR_CLOSED` and `EAI_AGAIN`, which were in
utils.ts's transient set but missing from event-retry.ts's — two
hand-maintained lists collapsed into one semantic code. And the stale-token
case (drain for auth expiry, refresh yields the same bearer) now gets two
in-process attempts that cannot succeed, ~300ms before it falls through to
queue redelivery; that is cheaper than keeping a WS-specific policy alive for
one call site. `TIMEOUT` is deliberately not in the clause: utils.ts maps a
caller-supplied `AbortError` onto it, and a cancelled write must not be
re-issued.

A status-less reply also stops being a bare `Error` — as one it failed
`WorkflowWorldError.is()` and surfaced a protocol version skew as a
USER_ERROR. It is now `code: 'PARSE_ERROR'`, the same code utils.ts uses for
an unreadable HTTP body, and for the same reason: the write may or may not
have landed.

**No waiter is left unanswerable.** An undecodable frame, the server's
malformed-frame sentinel (`reqId: -1`) and a non-numeric `reqId` were logged
and dropped. None can be correlated by construction, so the request that
provoked them stayed in `pending` with nothing in existence able to settle it
— freed only by the server's own drain (~680s from connect), typically past
the invocation's `maxDuration`. Each now fails the connection: every waiter
learns why, and the socket is replaced. A reply for an id nobody is waiting
on stays log-and-drop, deliberately — that request already settled, so
nothing is orphaned, and failing the socket would punish healthy in-flight
writes.

A per-request deadline backs that up for whatever is left, including a server
that accepts a frame and never answers it. Same knob as the HTTP path
(`WORKFLOW_REQUEST_TIMEOUT_MS`, 60s), whose doc comment already describes
this exact hang-to-SIGTERM pathology.

One existing idle-teardown test needed the deadline raised: the idle window
and the default deadline are both 60s, so a request could not outlive the
former without also outliving the latter. The test is about `inFlight > 0`
suppressing the teardown, so it now sets the deadline out of the way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Open the ws socket when the invocation starts, not on its first write

Lazily connecting bills the whole handshake — an upgrade round-trip plus the
OIDC token mint that rides it — to whichever event a fresh invocation writes
first. When that is a `step_started` issued as the step body is already
running, the event's server-recorded timestamp lands later than the work it
describes: the step looks shorter than it was. That is the shape of the e2e
timing failure on this branch, where a 9s step measured 6.5s from
`getStepMetadata().stepStartedAt`.

The queue handler is the earliest point that knows the run id, and a message
delivered for a run means writes are coming, so `warmWsEventsTransport` starts
the handshake there. By the first write it is done or in flight, and the write
just uses it.

Nothing about it is load-bearing:

- It doesn't await, and can't fail the handler. A warm that fails logs and
  stops — a never-opened first connect is precisely the case `connect`'s close
  handler already declines to retry, so no backoff loop starts for a run that
  may never write. The first real write connects as it would have anyway,
  carrying the shared retry policy.

- No-op unless `WORKFLOW_EVENTS_TRANSPORT=ws`, and no-op for the api-workflow
  proxy World, which can't serve an upgrade at all — the same fallback the
  write path takes.

- Warming arms the idle timer as if a request had settled, so an invocation
  that warms and never writes (a health probe carrying the run id it is about
  to create) releases its socket on the usual 60s rather than stranding it.
  The socket is not `unref`'d, so a stranded one would hold this process and a
  server invocation open.

Also closes a race that warming makes reachable: `close()` can only drop the
connection it can see, so a release landing mid-handshake left the socket to
install itself afterwards onto a transport already evicted from the cache,
which nothing would then ever close. The `open` handler now declines to adopt
a socket whose transport was released while it was connecting. This was
already reachable via the eager reconnect path, just much harder to hit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* changeset: just the env var

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* inline the ws-accelerator predicate at its only call site

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* refactor(world-vercel): trim ws-transport comments

Comments were 47% of the file. Cut the historical narration, the
restatements of adjacent code, and the repeated rationale (the `unref`
reasoning appeared four times, per-connection reqId three), keeping the
non-obvious facts: `ws.send()` reports failure via callback instead of
throwing, reqId is per-connection so `pending` must be too, the
unknown-reqId case is deliberately non-fatal, the auth_expiry same-token
bail-out, and why the idle timeout exists at all.

No code changes.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* own transport selection in the transport module

`events-v4.ts` was assembling the WS transport itself: reading the opt-in
flag, resolving the URL, deciding which Worlds can use a socket, minting the
per-connection header thunk, and holding the two once-per-process log latches.
None of that is about turning an event into a frame, which is what the rest of
that file does. Move it next to the socket it configures — `events-v4.ts` now
consumes one seam (`resolveWsTransport`) plus the gate, and `queue.ts` gets
`warmWsEventsTransport` from the module that owns the warm.

`headersToRecord` now lives in `http-core.ts` because both callers need it and
neither may import the other: `events-v4` already depends on the transport, so
the reverse edge would be a cycle.

Test fallout, and the reason the move is worth it: `events-v4-ws.test.ts`
mocked `getWsEventsTransport` to observe the resolve step, which no longer
intercepts anything now that the call is intra-module — an ESM mock replaces a
module's exports, not its own call sites. That mock's tests were only ever
about selection, so they move to `ws-transport.test.ts`, where the real
selection code runs against the existing fake-socket harness instead of a
stub. `resetWsEventsTransportsForTest` clears the log latches so the
once-per-process assertions don't depend on test order. What stays behind
mocks `resolveWsTransport` and covers what that file is actually for: reply
frame in, `Response`-shaped result out — including the null-resolve fallback to
HTTP, which nothing covered before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* import `ws` statically

The lazy `import('ws')` was there to keep the package off the module-init path
of deployments that never opt in — `events-v4.ts` imports this module
unconditionally, since the transport gate is a runtime branch. Measured, that
buys ~17ms: `require('ws')` is 16.5-18.0ms cold, 13 modules, and neither
`bufferutil` nor `utf-8-validate` loads (optional peers, absent by default).
Bundle size is identical either way — webpack and Rollup both statically follow
a dynamic `import()`, which is why the externals in `@workflow/builders` are
unaffected by this change.

For 17ms it cost a memoized promise, an inlined `WS_READY_STATE_OPEN` (so a
readyState check wouldn't force the module to load just to read a constant off
the constructor), and a whole test file — `ws-transport-lazy.test.ts` had to
live alone, because vitest caches a `vi.mock` factory result for the lifetime
of a module registry, so only a file that connects exactly once can observe
the laziness at all.

It also skewed the thing this branch exists to measure. The import lands inside
the first connect, so on a warm container it is billed to whichever event write
opens the socket, inflating the timestamp of the step it labels — the same
distortion the queue pre-warm was added to remove.

Also drops `WS_READY_STATE_OPEN` in favour of `WebSocket.OPEN`, now that
reading it is free.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* tighten the comments on the ws transport

Comments only — no code changes in this commit.

Cuts ~150 lines of prose across the WS additions. The rule applied: keep the
design factors a future reader needs (why the connection is scoped to a run,
why a bad reply takes the socket down, why the accelerators are externalized
unconditionally, why `TIMEOUT` is excluded from the `TRANSPORT` classification)
and drop the narrative of how the code got here — which revision did what, what
an earlier attempt got wrong, what was measured on the way. That history lives
in the PR and the git log, where it doesn't have to be re-read on every visit
to the file.

Biggest reductions: the retry essay above `postEventFrameOverWs` (30 lines to
11), the flag's OTEL-gap note (34 to 13), the OIDC refresh explainer (26 to
14), the accelerator rationale in `@workflow/builders` (26 to 14), and the
conformance suite's header (28 to 17).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* inject W3C trace context on the ws upgrade

Frames carry no headers, so the upgrade is the only place this transport can
propagate context; the server parents a run's event spans to whichever
invocation opened the socket. Covered in trace-propagation.test.ts, both with
and without an active span.

Splits the opt-in gate into an import-free ws-transport-enabled.ts so callers
can answer it without loading this module (used by the next commit).

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* load the ws transport module only when it is enabled

Both call sites read the gate from the import-free module and dynamically
import ws-transport.js behind a true result, so a deployment on the HTTP
default never pays ws's ~17ms of module init. The queue pre-warm absorbs it
for one that opted in, keeping it off the first event write.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* document WORKFLOW_EVENTS_TRANSPORT as experimental

Names the instrumentation gap (no client span per write) and the proxy path
where the variable is ignored.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* correct why the ws accelerators are externalized

No bundler fails the build on the unresolvable require — verified against
Rollup 4.62. webpack half-bundles the native module and Vite substitutes a stub
that makes the require succeed; both leave bufferUtil.mask undefined and throw
only once a frame reaches the native masker at 48 bytes, which every CBOR event
frame does. Same claim was repeated in the rollup plugin and its test.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* trim the WORKFLOW_EVENTS_TRANSPORT docs to user level

Mirrors the other Vercel World env vars: same facts on both pages, each in its
page's format. The instrumentation and socket-lifetime detail belongs in the
code, not in a user-facing reference.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* cover the vite bundler in the ws transport lane

Vite substitutes a stub for ws's absent native accelerators rather than failing
the require, so nothing catches it until a masked frame reaches 48 bytes — and
this job's three existing lanes are esbuild, turbopack and nitro.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* claim only what is measured about rollup and the ws accelerators

The rationale asserted plain Rollup was "safe by accident" via a mechanism
only ever observed in a minimal repro. Nitro traces and externalizes `ws` in
a production build, so the bundled path is not reached there at all.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Give the events socket an explicit lifetime instead of an idle timer

`openWsChannel` / `closeWsChannel` bracket one invocation of the flow
route, and are the only calls anywhere that create a channel. Writes ask
`resolveWsTransport` whether one is open — a lookup now, never a create —
and take pooled HTTP when it says no.

That removes the reason the idle timeout existed. A lazily-created socket
has no owner, so a timer was the only thing able to end it, and the socket
is not `unref`'d: the process could not exit, and a server invocation
stayed pinned, for the full window past the last write.

It also settles `run_created`. The trigger path opens no channel, so a
lone write no longer pays for a handshake it cannot amortize — `start()`
runs in an arbitrary request handler with no boundary the SDK can see.

Refcounted rather than a flag: inline step executions ride the flow topic
on per-step topics, so a run's steps can be concurrent invocations in one
instance sharing the channel, and the first to finish must not cut the
others short. A failed connect closes the channel so the invocation's
writes fall back to HTTP instead of each paying its own doomed handshake.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Name the one reply header the WS path does not map

The server copies six headers into an `event_ack`'s meta and this record
maps five. The sixth, `X-API-Deprecated`, is inert today — the v4 route's
middleware chain has no deprecation middleware to set it — but the record
is the only header source a WS reply has, so an unmapped key is gone
rather than merely unread, which is not true of the `Response` the HTTP
path returns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* docs: note that WORKFLOW_EVENTS_TRANSPORT=ws is ignored on the proxy path

The api-workflow proxy is an HTTP-only REST gateway and does not forward
a WebSocket upgrade, so a World configured with projectConfig keeps
writing events over HTTP regardless of the setting.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* ci: gate the ws-transport e2e lanes on a label

Three real `vercel deploy`s per run is too much to charge every
unrelated PR in the repo for a transport that is off by default. PRs opt
in with `ws-transport-test` (or `workflow-server-test`, which already
exists to test the half of this the protocol lives in); main keeps the
signal on every commit.

The required aggregate has to allow the lane to be skipped in that case,
so its status is asserted only when the lane was actually supposed to
run.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* chore: regenerate pnpm-lock against current main

main resolved `ws` to 8.20.0 as a transitive peer; this branch adds it
as a direct dependency of world-vercel and floats it forward, which
rewrites every `openai@x(ws@y)` peer key in the lockfile. Merging main
textually combined the two, leaving those keys pointing at a `ws` entry
the merged file no longer had — `--frozen-lockfile` then failed with
ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY on the PR's merge ref.

Regenerated from main's lockfile so ours is a minimal delta on top of it.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* fix(world-vercel): align ws on the version main already resolves

The lockfile broke on the PR's merge ref, not on this branch's head: main
resolves ws@8.20.0 as a transitive peer, and a `^8.21.1` direct dep here
floated it forward, rewriting all 73 `(ws@8.20.0)` peer keys. Git merged
the two lockfiles without a conflict but left main-side keys pointing at
a ws entry the merged file no longer had, so `--frozen-lockfile` failed
with ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY.

`^8.20.0` resolves to the copy main already has, so the lockfile delta is
the two importer entries instead of a repo-wide rewrite that re-breaks
every time main moves. Also keeps one ws in the store rather than two.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Bind the channel release to the instance it claimed

closeWsChannel resolved the transport by URL, but the refcount lives on
the instance. A channel is evicted from the map as soon as it closes — a
refused upgrade does that on the connect path — so the next opener for
the same run registers a different instance under the same URL, and the
first invocation's close then decremented that one instead. It dropped a
socket a live invocation was still writing over, and for the event types
EVENT_RETRY_ELIGIBILITY marks non-retryable there is no second attempt to
carry the in-flight write over HTTP.

openWsChannel now returns an idempotent release closed over the transport
it incremented, and queue.ts holds that instead of re-resolving the run.
The close awaits the open's own promise, so it also can no longer land
ahead of the claim it releases.

Also names the scope of the connect-failure de-opt: it covers the
handshake only, so a channel that connects and then fails every write
keeps taking the WS path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Decode a transport result, not a Response

Main extracted the v4 POST decode into a helper typed `Response` while this
branch narrowed the POST result to `FrameResponseLike`, because the WS branch
synthesizes its result rather than holding a real `Response`. The two merge
without a textual conflict and then fail to typecheck.

Widen the helper: it reads only the two members `FrameResponseLike` declares,
and a `Response` still satisfies them, so the HTTP call sites are unchanged.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Re-run CI

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Re-run CI

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Reconcile the WS transport with main's v4 POST rework

main moved the materialized POST result off the `x-wf-*` response headers
and onto a typed CBOR body, and added a second response shape: two callers
now POST with `Accept: application/vnd.workflow.v4-frames` and read back a
sentinel-terminated sequence of frames.

A frame stream has no representation in a protocol that pairs one reply
frame with one request frame, so the WS switch moves off the shared poster
and onto `createWorkflowRunEventV4` alone — the materialized write, which is
the hot per-step path this branch exists to shorten. `run_started` and the
`hook_received` preload stay on HTTP.

`decodeCreateEventResponse` takes `FrameResponseLike` rather than `Response`
because the WS branch has none to hand over; a real `Response` satisfies the
interface, so the HTTP callers are unchanged. The ids now come out of the
CBOR body, so `replyMetaToHeaderRecord` no longer maps any `x-wf-*` name —
only the two headers `errorFromV4Response` reads.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Re-run CI

Resample the WS-arm sleepingWorkflow failure: it has now recurred on a second
axis (nextjs-turbopack, 7709ms; previously vite, 7570ms), so the arm needs
more samples before the skew can be called WS-specific or repo-wide flake.

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Re-run CI

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* Re-run CI

Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

* blank

* blank

---------

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
2026-08-09 17:34:44 -07:00
Karthik Kalyan e084e08ac0 Reduce Vercel E2E polling load (#3316)
* Reduce Vercel E2E polling load

* Keep Vercel E2E matrix concurrency
2026-08-03 19:29:59 -07:00
Nathan Rajlich f8f6e17aeb Add opt-in QuickJS WASM VM engine (WORKFLOW_VM=quickjs) (#3048)
* Add opt-in QuickJS WASM VM engine (WORKFLOW_VM=quickjs) with full event replay

* QuickJS engine: AbortController, setAttributes, terminal drain, turbo-safe requeue, stable PRNG seed

* QuickJS engine: hook.getConflict support, cross-run writable forwarding symbols

* QuickJS engine: stream framing round-trip, bound step proxies, webhook fidelity

* Apply biome fixes to QuickJS engine files

* Address review feedback: anchor source-map strip to end-of-input, use getWorkflowQueueName for conflict requeue, Buffer-free asset decoding, function replacers for payload injection, maxEventsLimit guard

* CI: include generated QuickJS source assets in shared e2e build artifacts

* Fix same-token hook ordering and conflicted-hook disposal in the QuickJS engine

* CI: run both VM engines across all frameworks and worlds; label jobs with the engine

* Fix stack overflow stripping inline source maps from webpack dev bundles; harden step-listing e2e assertions against eventually-consistent reads

* e2e: poll step listings until analytics rows include attempt (optional column can lag terminal status)

* e2e: use --withData to force storage-backed step listings for attempt assertions (analytics listing can omit attempt entirely)

* Sort imports in QuickJS serialization files (biome organizeImports)

* QuickJS engine: resolve the run's full payload-key capability so sealed (encp) hook payloads open

Main's sealed-box work (#3096) makes cross-deployment resumeHook() seal
hook payloads to the target run's published X25519 public key. The shared
start() path publishes that key regardless of engine, so QuickJS runs
receive sealed payloads too — but the QuickJS entrypoint resolved only the
bare symmetric key via importKey(), which cannot open encp envelopes. The
first sealed hook payload wedged the run right after hook_received, timing
out every hook/webhook e2e on Vercel prod (node:vm legs were fine — the
node engine resolves the full capability via memoizeEncryptionKey).

Resolve deriveRunPayloadKeys() in the entrypoint instead and widen the
runtime's key types from CryptoKey to DecryptionKey. Writes stay symmetric
(encrypt() with RunPayloadKeys takes the encr path). Regression test seals
a payload exactly as resumeHook does and round-trips it through the VM.

* Address review: crypto/process parity, loud Intl guards, lazy engine import, VM-leak guard, telemetry namespace, eval-string escaping

- Deterministic crypto.getRandomValues/randomUUID in the VM bootstrap,
  drawing from the seeded Math.random (identical sequences to the node
  engine's vm/index.ts implementations); all crypto.subtle methods throw
  with step-function guidance. process.env exposed as a frozen copy,
  matching node.
- Intl: throwing constructors (no ICU in QuickJS), and toLocale*-family
  methods (incl. localeCompare) throw when given an explicit locale so
  cross-engine divergence is loud instead of silently writing different
  values into the event log. No-argument forms keep working.
- runtime.ts lazy-imports the QuickJS entrypoint at dispatch, keeping the
  ~1.3MB embedded WASM assets out of node-engine deployments.
- runQuickJSWorkflow wraps the per-run phase so an exceptional exit
  disposes the VM instead of leaking it in a reused compute instance;
  corrected the misleading fail-loud comment (run_failed, not retry);
  warn when the event drain loop exhausts its iteration bound.
- Telemetry attributes renamed quickjs.* → workflow.vm.* to stay in the
  file's workflow.* namespace.
- Eval-string correlation-id interpolation uses JSON.stringify instead of
  quote-only escaping.
- common-vm.test.ts pins the reducer/reviver superset invariant against
  common.ts so the duplicated sets can't silently drift.
- Docs enumerate the remaining global-surface differences (subtle.digest,
  Intl, WebAssembly, Atomics); quickjs-entrypoint documents the known
  precondition-guard gap.

* QuickJS engine: implement resilient resumeHook (hookInput materialization + resumeId dedup)

#1834 made resumeHook() fall back to enqueueing the run with a hookInput
payload when the direct hook_received write fails transiently, with the
runtime materializing the missing event on delivery. Only the node:vm
path implemented it — the QuickJS dispatch returned before the node
block, so the resilient payload was silently dropped and the new e2e
timed out on every quickjs leg.

- runtime.ts threads hookInput into runWorkflowWithQuickJS; the
  entrypoint materializes the missing hook_received after loading the
  event log (resumeId-keyed dedup, occurredAt from the resumeId ULID,
  local eventData substitution for lazy/ref responses, EntityConflict /
  HookNotFound handling) — mirroring the node block.
- processEvents drops duplicate hook_received rows sharing a resumeId
  (first-in-log wins), matching the node engine's EventsConsumer dedup;
  the seen-set lives in the VM heap so it is deterministic per replay.

Verified against the dev server with WORKFLOW_VM=quickjs: the resilient
resume e2e passes and the materialization is observable in the logs; all
27 hook e2e tests green.

* Rerun CI

* QuickJS engine: split VM-local class/step-function reducers off the hardened host codec

The hardened host-side serialization (#3257) made the shared
reducers/class.ts and reducers/step-function.ts depend on
serialization/hardened.ts, which imports node:util and captures host
intrinsics — unbundleable and meaningless inside the QuickJS guest,
where the codec already runs in the guest realm. Point the VM codec at
pre-hardening copies with identical wire format; the host/guest
boundary hardening for this engine arrives with the host-side serde
that retires the VM bundle.

* QuickJS engine: enqueue explicit wait continuations instead of same-message redelivery

Scheduling sleep wakeups by returning { timeoutSeconds } redelivers the
CURRENT queue message. When that message is a hook-resume delivery
(carrying hookInput), its redelivery re-runs the lazy-resume re-ensure
in the handler prologue; if the workflow disposed the hook during the
first delivery (dispose -> sleep), the re-ensure gets HookNotFound, the
prologue acks the message as 'nothing left to resume', and the wait
timer it carried is silently lost — the run wedges (caught by the
hookDisposeTestWorkflow e2e).

Enqueue fresh continuation messages instead, matching the node engine's
suspension handler: getWaitContinuationDispatch for pending waits
(gaining delay clamping/hop chaining and pending-wait dedup keys) and a
plain immediate message for elapsed-wait / attr_set / getConflict
requeues. A fresh message carries only runId, so its delivery always
reaches replay.

Also: read hook_received resumeId from the canonical top-level event
field (eventData.resumeId is the deprecated legacy fallback), and stop
passing hookInput into the entrypoint — the shared prologue in
runtime.ts materializes the event for both engines. Adds a VM replay
test for the hook -> dispose -> sleep shape.

* Sort imports in quickjs-entrypoint (biome organizeImports)

* Address review: dispatch inside run-level try/catch, queue namespace + run-origin trace carrier threading, configurable interrupt budget

- Move the QuickJS engine dispatch inside the replay loop's try so
  escaping engine failures (MaxEventsExceededError, WASM OOM,
  bundle-eval errors) reach the catch that classifies and records
  run_failed, instead of nacking the message and burning all 48 queue
  redeliveries into MAX_DELIVERIES_EXCEEDED. Transient world errors
  still rethrow for redelivery. Updated the two comments that describe
  the propagation.
- Thread the queue namespace from runtime.ts through
  runWorkflowWithQuickJS into every message publish (step dispatch,
  hook_conflict requeue, immediate requeue, wait continuation) —
  without it, publishes on a namespaced deployment land on
  __wkf_workflow_* while consumers listen on __<ns>_wkf_workflow_*.
- Thread the run-origin nextTraceCarrier accessor through instead of
  capturing the current invocation context, so linked-mode invocations
  form a star around workflow.start rather than chaining; the
  hook_conflict requeue now carries a traceCarrier and requestedAt.
- Replace the hardcoded 30s VM interrupt budget with the configurable
  replay budget (getReplayTimeoutMs, default 240s), matching the node
  engine.

* Sort imports in quickjs-runtime (biome organizeImports)
2026-08-03 16:38:58 -07:00
Pranay Prakash 11dc036854 ci: stop deploying changeset-release/main, run its e2e against production (#3243)
* ci: stop deploying changeset-release/main, run its e2e against production

The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.

Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.

Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

* ci: resolve changeset-release e2e deployments with the wait action, tokenless

Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.

Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

---------

Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
2026-07-31 10:09:36 -07:00
Nathan Colosimo 62d570ed4b Remove retired v1 step route plumbing (#3061) 2026-07-24 23:50:55 +00:00
Peter Wielander 599250771d [benchmarks/ci] SO payload variants + restructured E2E Test Results comment (#3080) 2026-07-23 19:52:41 -07:00
Pranay Prakash 9a2770ab34 test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
* test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident)

Reproduces the o2flow v5 upgrade failure (workflow@5.0.0-beta.26, fixed by
#2752 in beta.28): a plain API route importing defineHook() from the root
`workflow` entry and calling .resume() failed with Turbopack's
"Cannot find module as expression is too dynamic" stub, because the world
registration was tree-shaken out of the route bundle and getWorldLazy()'s
dynamic-import fallback got stubbed.

The bug only manifests when a route bundle loads in isolation (a Vercel
lambda): local `next dev`/`next start` evaluates next.config.ts, whose
workflow/next import chain registers the world process-wide and masks it —
which is why no existing server-driven suite caught it.

- route-bundle-isolation.test.ts: production Turbopack build of the
  nextjs-turbopack workbench, then loads ONLY the compiled route bundle in a
  bare Node subprocess (cold-lambda simulation) and invokes its POST handler.
  Fails with the exact incident error on regressed code; passes on main.
  Wired into the build-error-messages CI job.
- e2e: plainModuleDoneHook round-trip through a plain API route on the two
  Next workbenches (deployed matrix covers real lambda isolation).
- Workbench fixtures mirroring o2flow: a directive-less defineHook module
  shared by a workflow (create) and a plain route (resume). The webpack
  workbench gets a real route file because `next dev` (webpack) does not
  serve directory-symlinked app routes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

* test: authenticate plain hook resume request

* test: address review — marker-based harness output parsing, changeset summary

- route-bundle-isolation: prefix the harness result line with a unique
  marker and locate it explicitly instead of JSON.parse()ing the last
  stdout line, so stray logging from the route bundle or the world can't
  break parsing; failures now include the full subprocess stdout.
- changeset: add a human-readable summary to the (release-less) changeset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

---------

Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Karthik Kalyanaraman <karthik.kalyanaraman@vercel.com>
Co-authored-by: Karthik Kalyan <105607645+karthikscale3@users.noreply.github.com>
2026-07-21 13:24:17 +07:00
Peter Wielander 96719d8220 [ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011) 2026-07-20 14:21:17 -07:00
Nathan Rajlich 9da2d76260 [core][world][world-vercel] Add World.createRunId() and region-aware queue routing (#1981)
* [world-vercel] Add /run-id sub-export with tagged ULID encode/decode

Encodes a tag bit, 5-bit version, and 6-bit Vercel region ID into a
ULID-shaped string used for workflow run IDs. Tagged values remain
valid 26-char Crockford-Base32 ULIDs so they still sort and round-trip
through any system that accepts ULIDs.

* [world-vercel] Add string-value assertions to run-id tests

Add exact-string expectations for encoded outputs at known inputs,
covering the default region/version pair, numeric region IDs, version
overrides, boundary values (all-zero, all-max), the dirty-input
overwrite case, and the lexicographic-order checks. Also adds an
explicit byte-array expectation for the canonical ULID-spec example
string and an additional first-char-range coverage test for isTagged.

* [world-vercel] Remove internal-repo reference from regions doc comment

* [world-vercel] Address PR review feedback on run-id sub-export

- isTaggedString now fully validates the input as a 26-char Crockford
  Base32 ULID (delegating to ulidToBytes) instead of only inspecting
  the first character. This fixes false positives on inputs like
  '4UUUU...' that have a valid tag-bit position but invalid chars
  later in the string.
- isTagged() now accepts `unknown` to match its documented behavior
  of safely rejecting non-string inputs without requiring callers to
  cast.
- Introduce `RegionKey` for the full set of keys including 'unknown',
  and narrow `RegionCode` to `Exclude<RegionKey, 'unknown'>` so the
  return type of `lookupRegion` and the `DecodedRunId.region` field
  accurately reflect that 'unknown' is never produced. Updates
  `encode` to reject 'unknown' as a region code string at runtime
  (callers wanting the unknown sentinel should pass numeric 0).

* [world] [core] [world-vercel] Add World.createRunId() and region-aware queue routing

- @workflow/world: add optional createRunId(input?) to the World
  interface so worlds can mint run IDs with embedded metadata, and
  add an optional 'region' field to QueueOptions for per-message
  routing hints.

- @workflow/core: start() now delegates run ID generation to
  world.createRunId() when defined (falling back to a monotonic
  ULID otherwise), and accepts a new 'runIdInput' option that is
  forwarded verbatim to createRunId. When runIdInput.region is a
  string, it is also threaded onto the queue options so the initial
  workflow message is dispatched to the matching region.

- @workflow/world-vercel: implement createRunId() to mint
  region-tagged ULIDs, preferring an explicit runIdInput.region and
  falling back to the VERCEL_REGION env var. The queue now resolves
  its destination region from (in order): an explicit opts.region,
  the region embedded in the payload's tagged run ID, the
  VERCEL_REGION env var, and finally a hardcoded 'iad1' default.
  This replaces the previous unconditional 'iad1' region passed to
  the @vercel/queue client.

Monotonicity within a process is preserved by tracking the last
emitted run ID and bumping the bit immediately above the 11-bit
metadata window when a same-ms collision would otherwise occur,
then re-stamping the requested region/version on top so metadata
remains stable.

* [core] [world] [world-vercel] Pass full StartOptions to World.createRunId

Drop the dedicated 'runIdInput' field on StartOptions and forward the
entire options bag to world.createRunId() instead. This keeps the
public API surface smaller and lets each World pick the fields it
recognises (e.g. world-vercel reads 'region'). The top-level 'region'
option remains on StartOptionsBase and is also threaded onto the
queue's per-call region opt when set.

* Address review feedback: doc fixes and deterministic same-ms tests

- Document the final iad1 fallback in QueueOptions.region (world)
- Correct the World.createRunId doc: start() always passes an object
- Fix the clientOptions comment: the handler client omits region and
  relies on SDK auto-detection + the ce-vqsregion header for acks
- Fix a misleading QueueClient-construction comment in queue.test.ts
- Freeze time in the same-ms monotonicity test so it deterministically
  exercises the intended path, and add a test covering the
  bump-above-metadata fallback when the region changes mid-millisecond

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: keep workflow-server override rewrite-compatible

Export WORKFLOW_SERVER_URL_OVERRIDE while keeping the one-line const shape
that workflow-server's cross-repo e2e test automation rewrites. Update
world-vercel tests to import that exported value for mock origins and URL
expectations instead of duplicating the temporary preview URL.

* fix(world): clear region tag bit before ULID timestamp validation

Region-tagged run IDs set the high bit of the ULID timestamp byte. The
shared world timestamp validator used raw decodeTime(), so current tagged
run IDs appeared thousands of years in the future and were rejected before
reaching workflow-server. Clear the tag bit before decoding, matching the
workflow-server behavior, and cover tagged IDs in tests.

* fix(world-vercel): validate tagged runId timestamps via run-id decode

Keep @workflow/world's ULID helpers generic; they should not know about
world-vercel's region-tagged run ID layout. Instead, world-vercel decodes
its tagged runId to the original ULID before using the shared timestamp
validator for run_created events. Add a world-vercel regression test that a
current sfo1-tagged runId passes validation.

* fix(world-vercel): default run ID region to iad1 instead of unknown

When neither an explicit region option nor VERCEL_REGION is available,
createRunId minted a tagged ULID with the unknown (0) region sentinel,
producing the tagged: true, region: null state. The server already
resolves unknown/untagged runs to DEFAULT_VERCEL_REGION (iad1), so mint
a concrete iad1 tag instead, keeping every run ID self-describing and
routable.

* test(e2e): use verbose reporter + per-test start heartbeat

The default vitest reporter buffers per-file output, so a stalling e2e
test produces no output until its timeout — making CI look like a silent
30-minute hang. Switch the e2e CI invocations to the verbose reporter
(prints each test result as it completes) and emit a '[e2e] ▶ start:'
heartbeat to stdout at the start of every test (bypassing vitest's console
buffering) so a stuck test is immediately identifiable in the live CI log.

* test(world-vercel): point WORKFLOW_SERVER_URL_OVERRIDE at combined 527+529 preview

Temporarily target the workflow-server combined-527-529-preview deployment,
which bundles platform-directed multi-region routing (vercel/workflow-server#527,
incl. the iad1 hook pin) and durable stream state (vercel/workflow-server#529),
so e2e can validate the full multi-region path end-to-end. Revert to empty on main.

* fix(core): region-tag the health-check correlationId

The health-check response is delivered over a Redis stream whose name (and
synthetic run ID) embed the correlationId. Under platform-directed routing
the responding endpoint and the polling reader can be served from different
physical regions; Redis is physical-region-local, so the correlationId must
carry the region for both sides to resolve the same backend.

Generate the correlationId via world.createRunId() (a region-tagged ULID)
when the world provides it, falling back to a plain ULID for worlds that
don't tag IDs (e.g. local, single-region). The synthetic wrun_hc_<id> run ID
then carries the region; workflow-server's region middleware decodes it.

* Address review feedback: validate region overrides, reset server override

- queue: validate opts.region and VERCEL_REGION against the known region
  table before routing, ignoring unrecognised codes so a bad override
  can't clobber the payload-derived region (Copilot)
- add isKnownRegionCode() runtime guard to run-id/regions
- reset WORKFLOW_SERVER_URL_OVERRIDE to '' (must be empty on main)
- fold the within-PR iad1-default changeset into the main world-vercel
  changeset and delete it (review)
- start.test: declare specVersion on createRunId mock worlds now that
  the merged world-compatibility check requires it
- cover the new region-validation fall-through paths in queue.test

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(world-vercel): point WORKFLOW_SERVER_URL_OVERRIDE at wave-1 multi-region preview

BRANCH-ONLY — revert the override to '' before merge (lint enforces).

Points this PR's e2e/benchmark runs at the wave-1 multi-region
workflow-server preview (vercel/workflow-server#590: iad1+sfo1+fra1
serving, staging data backends) so region-tagged runs are validated
against real multi-region serving end-to-end.

Also makes the unit-test mock origins in events-v4.test.ts and
trace-propagation.test.ts override-aware (same pattern the rest of
the file and utils.test.ts already use), so the suite passes whether
or not the override is set — these two files were the only spots
hardcoding https://vercel-workflow.com.

* test(e2e): Vercel multi-region suite for start()'s region option

Adds a dedicated e2e suite validating @workflow/world-vercel region
routing end to end, run as its own CI job (e2e-vercel-multi-region)
against the nextjs-turbopack workbench only — deliberately separate
from e2e.test.ts, which runs as a matrix across all worlds/frameworks
where Vercel-specific multi-region behavior doesn't apply.

- workbench/nextjs-turbopack/vercel.json: deploy to iad1+sfo1+fra1 so
  region-routed flow messages have a function to land on in each region.
- workflows/99_e2e.ts: regionProbeWorkflow returns the VERCEL_REGION
  observed by both the workflow and a step, so tests can assert the run
  EXECUTED in the intended region (not just that it was tagged).
- packages/core/e2e/e2e-region.test.ts: per-region cases assert
  1) start(..., { region }) mints a region-tagged run ID (decoded via
     @workflow/world-vercel/run-id),
  2) the workflow + step both observed VERCEL_REGION === region,
  3) the server reports the run completed;
  plus a concurrent all-regions case guarding against cross-region
  misrouting under simultaneous multi-region traffic. Skips on local
  deployments.
- .github/workflows/tests.yml: new e2e-vercel-multi-region job
  mirroring e2e-vercel-prod's env/deployment-wait, running only the
  new suite.

* test(e2e): start region probes in-function; fix getWorld await

The first multi-region CI run surfaced two issues:

1. sfo1/fra1-tagged runs executed in iad1. The suite started runs from
   the external test process, which uses the api.vercel.com token proxy
   — and the proxy's queues path forwards every send to the region-less
   VQS host (the world's proxy-mode resolveBaseUrl ignores the region
   argument, and the proxy's x-vercel-vqs-api-url escape hatch only
   allowlists vqs-server-*.vercel.sh preview hosts). Production traffic
   publishes IN-FUNCTION (direct regional queue routing), so the suite
   now triggers start() through a new workbench route
   (/api/e2e-region-start) and rehydrates the run with getRun() —
   testing the path production actually takes. Proxy-mode regional
   queue routing is a known gap to address separately in api-workflow.

2. TypeError on world.runs.get: getWorld() is async and was called
   without await.

* test(e2e): cover explicit and implicit region starts in the multi-region suite

With regional VQS routing now working through the api.vercel.com proxy
(vercel/api#79056 + #2789 + this branch's per-send region resolution),
the suite covers both start configurations, asserting the same three
properties for each (region-tagged run ID, execution in the intended
region via VERCEL_REGION echoed in the return value, server-side
completion):

1. EXPLICIT: start(..., { region }) called directly in the vitest
   runner — publishes through the token proxy, per-send region carried
   by x-vercel-queue-region. Restores the direct-start shape the suite
   had originally, plus the concurrent all-regions case.

2. IMPLICIT: dedicated per-region workbench routes
   (/api/e2e-region-implicit/{iad1,sfo1,fra1}), each pinned to a single
   region via a per-function 'regions' entry in the workbench
   vercel.json, calling start() with NO region option — createRunId
   derives the tag from the minting function's VERCEL_REGION. The test
   also asserts the route reported executing in its pinned region, so
   the implicit-tagging assertion can't pass vacuously.

Replaces the interim /api/e2e-region-start route (explicit region via
request body), which existed to work around the pre-#79056 proxy gap.

* Revert WORKFLOW_SERVER_URL_OVERRIDE to '' — wave-1 multi-region serving is in production

workflow-server#590 (iad1+sfo1+fra1 serving) merged and deployed to
production and the e2e backend, so this branch's e2e/benchmark runs no
longer need to target the wave-1 preview. Restores the empty override
the No Test Overrides lint job enforces for merge.

The override-aware unit-test origins (events-v4/trace-propagation)
stay — they are correct under any override value.

* test(e2e): cross-region stream visibility (iad1 writer, sfo1 reader)

Regression coverage for a backend bug that made cross-region stream
reads report zero chunks on IN-PROGRESS streams (completed streams were
unaffected), which forced the multi-region serving rollback.

The new case exercises exactly that geometry:
- crossRegionStreamWorkflow (99_e2e.ts) writes N chunks to the default
  output stream, then holds the stream OPEN for 45s before closing —
  the in-progress window is the point, since completed streams are the
  easy case.
- The e2e starts it with region iad1, waits (same-region, via the
  api.vercel.com proxy) until all chunks are written, asserts the run
  is still 'running', then reads through a new sfo1-pinned workbench
  route (/api/e2e-stream-read/sfo1) that returns getTailIndex() plus
  its VERCEL_REGION. The reader's region served none of the stream's
  writes, so the reported chunk count must come from the backend's
  cross-region stream metadata. The test fails loudly if the route
  isn't actually executing in sfo1.

Also bumps the explicit-region test timeout to 120s: the first case in
the file absorbs every cold start at once (fresh workbench instances
in up to three regions plus a cold backend preview) and was observed
just over the 60s default.

BRANCH-ONLY (revert before merge, lint enforces):
WORKFLOW_SERVER_URL_OVERRIDE points at a multi-region backend preview
that includes the fix, so this validates cross-region stream
visibility end-to-end before multi-region serving is re-enabled.

* test(e2e): extend multi-region suite to all 19 provisioned regions

Points the suite at an all-regions backend preview and widens coverage
from the wave-1 trio to every provisioned region:

- Explicit path: a single concurrent all-regions case starts one
  tagged run per region (one shared cold-start window instead of 19
  sequential ones) and aggregates per-region failures so a single
  region's breakage reports alongside the full picture. The trio keeps
  its detailed per-region cases and the 9-way concurrent-isolation
  case.
- Implicit path: workbench gains a region-pinned
  /api/e2e-region-implicit/<region> route per provisioned region (19
  total, shared handler), the workbench itself now deploys to all of
  them, and the test.each covers the full set with per-case timeouts
  for regional cold starts.
- Multi-region CI job timeout 20m -> 35m for the sequential implicit
  cases.

BRANCH-ONLY (revert before merge, lint enforces):
WORKFLOW_SERVER_URL_OVERRIDE now targets the all-regions backend
preview instead of the previous (stale, since-merged) fix preview.

* test(e2e): tolerate geo-adjacent execution of queue callbacks

The first all-regions run surfaced a subtle execution-locality
behavior: queue delivery is guaranteed to the tagged region's
dataplane and the delivery callback egresses from that region, but the
consumer invocation's execution region is chosen by where that
callback enters Vercel's edge — and adjacent regions can geo-resolve
to each other's functions. Observed live: kix1-tagged runs (callback
egressing from Osaka) deterministically executing in hnd1/Tokyo on
both the explicit and implicit paths, with tagging, data placement,
and completion all still strictly kix1.

expectRunInRegion now asserts execution lands in the tagged region OR
one of its geographic neighbors (EXECUTION_ADJACENCY), while run-ID
tagging and server-side completion remain strictly the requested
region. Gross misrouting (e.g. kix1 -> iad1) still fails.

* Revert WORKFLOW_SERVER_URL_OVERRIDE to '' — all-regions serving is in production

The all-regions workflow-server rollout is deployed and serving
production traffic from every Vercel region, so this branch's e2e no
longer needs to target a branch preview. Restores the empty override
the No Test Overrides lint enforces for merge.

With this the PR is complete: region-tagged run IDs, region-aware
queue routing, and the multi-region e2e suite (explicit + implicit +
all-regions + cross-region streams) all validate against the
production-default backends.

* docs: fix three stale comments flagged in review

- start.ts: StartOptionsBase.region fallback is iad1, not the unknown
  sentinel (createRunId always mints a concrete routable region)
- queue.ts: example used a nonexistent start({ runIdInput }) API; the
  real option is start({ region })
- events.ts: decode() clears only the tag bit (top bit of the 48-bit
  timestamp field) — it does not restore the original untagged ULID;
  reword to say what actually matters for timestamp validation

* test(e2e): cover hook resolve/resume for runs owned by non-iad1 regions

Hooks are resolved by opaque token, which carries no region hint, so
lookup and resume must work regardless of which region owns the run's
data. Exercises the full follow-up-message path on sfo1- and
fra1-tagged runs: create inside the workflow, resolve by token from
the test process, resume twice sequentially, and assert payload order
and completion.

Regression coverage for the failure mode where the first message to a
hook-driven app on a non-iad1 run worked but every follow-up failed
with 'Hook not found'.

---------

Co-authored-by: Peter Wielander <peter.wielander@vercel.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:03:56 +00:00
JJ Kasper f6772d95c8 Optimize Next dev HMR rebuilds (#2678)
* Optimize Next dev HMR rebuilds

* Fix Next dev HMR CI coverage

* Gate dev HMR logs behind opt-in flag

* Match workflow dev build logs to Next style

* Fix Next dev HMR changed-file classification

* Fix Windows port detection

* Relax HMR log wait in dev e2e

* Avoid canary workflow execution cache flakes

* Allow slower Turbopack HMR propagation in e2e

* Scope canary HMR fuzz execution assertions
2026-06-29 20:58:38 +00:00
JJ Kasper 24f370773d Fix Workflow loader source map warnings (#2693) 2026-06-29 20:16:46 +00:00
Marco 2bb4164c9c Add Platformatic World to worlds-manifest.json (#1450)
* Add Platformatic World to worlds-manifest.json

Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>

* ci fixup

Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>

* ci: pin platformatic world image to 0.8.1 and harden community-world runner

Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>

* platforamtic-world version

Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>

* ci: wire generic docker service-type into community benchmark workflow

The shared community-worlds matrix now emits service-type "docker" for any
world with non-builtin or multiple services (e.g. Platformatic, which needs
postgres + the platformatic/workflow image). tests.yml's e2e-community path
already handles it, but benchmarks.yml's benchmark-community path
(label-gated, non-blocking) did not — so a "community-benchmarks" run would
start no services and fail.

Mirror the e2e "Start Docker services" step, package-version pin, and docker
cleanup into benchmark-community-world.yml, and pass `services`/`version`
through from benchmarks.yml.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Signed-off-by: marcopiraccini <marco.piraccini@gmail.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 14:39:11 -07:00
JJ Kasper 57cccaf373 Remove lazy discovery from workflow/next (#2545) 2026-06-22 13:14:35 -05:00
Peter Wielander c000462502 Capture Vercel runtime logs when e2e Vercel Prod lanes fail (#2356) 2026-06-11 12:36:39 +02:00
Karthik Kalyan 5bf2c167a5 Add serializable reviver compatibility check (#2250) 2026-06-04 12:50:32 -07:00
Pranay Prakash 3867270be8 Reduce unnecessary CI runtime (#2151)
* Reduce unnecessary CI runtime

* Fix shared E2E artifact extraction path

* Stabilize getWorkflowPort timeout test on Windows

* Preserve UI unit coverage on CI fast path
2026-06-02 02:49:20 +00:00
Nathan Rajlich 3c50f8c77b ci: extract wait-for-vercel-project to vercel/wait-for-deployment-action (#2065)
* ci: extract wait-for-vercel-project to vercel/wait-for-deployment-action

The action's logic was duplicated between this repo and
vercel/workflow-server, which is annoying to keep in sync. Move it to
a standalone repository so both can consume the same pinned build.

Changes:

- Delete .github/actions/wait-for-vercel-project entirely.
- Replace all five `uses: ./.github/actions/wait-for-vercel-project`
  references with `uses: vercel/wait-for-deployment-action@<sha>` in:
    benchmarks.yml, dispatch-front-workflow-release-pr.yml,
    docs-checks.yml, tarballs-checks.yml, tests.yml
- All `with:` inputs (project-slug, environment, timeout,
  check-interval, github-token) are unchanged — the new action's
  input contract is backwards-compatible.

The new action is ESM-only, targets Node 24, ships a ~12KB bundle
(down from ~830KB in the old in-repo version) by dropping
@actions/core and its transitive undici dependency, and is
unit-tested. See https://github.com/vercel/wait-for-deployment-action.

* ci: bump wait-for-deployment-action to fix/status-context-auto for verification

Repinning to vercel/wait-for-deployment-action#fix/status-context-auto
(SHA 04d46ef) which fixes the broken 'opt-out' heuristic that made
status-context resolution silently disabled for every consumer.

Reproduced in this repo's E2E logs:

  Looking for GitHub deployment in environment "Preview – example-workflow"
  Deployment ID resolution disabled (status-context is empty)
  Deployment ready: https://example-workflow-...labs.vercel.dev
  Run E2E Tests: VERCEL_DEPLOYMENT_ID=         <-- empty

Will repin to the post-merge main SHA once CI is green.

* ci: bump wait-for-deployment-action pin to merged main SHA

Repinning from the fix/status-context-auto branch (04d46ef) to the
post-merge main SHA (0e2b0c5, vercel/wait-for-deployment-action#4).
The deployment-id resolution fix verified against the prior fix-branch
pin (E2E tests now read VERCEL_DEPLOYMENT_ID=dpl_... correctly across
the matrix; only flaky/unrelated Vercel deployment failures remain).

* ci: grant statuses:read alongside deployments:read

The wait-for-deployment-action also reads the 'Vercel – <slug>'
combined commit status to resolve the dpl_xxx ID. The official
permissions table lists statuses:read for
GET /repos/{owner}/{repo}/commits/{ref}/status.
2026-05-21 17:28:13 -07:00
Karthik Kalyan ee61817865 ci: pin third-party GitHub Actions to commit SHAs (#2050)
Major-version refs like `@v2`/`@v5` resolve to mutable refs on the
upstream repos — sometimes a tag, sometimes a branch (e.g. marocchino
keeps `v1`/`v2`/`v3` as branches), and dawidd6 force-pushes the bare
`v6` tag forward outside of releases. A compromised maintainer account
could push new code that our CI picks up on the next run with
GITHUB_TOKEN (or, for changesets/action, NPM_TOKEN) in hand.

Pin all third-party `uses:` references to full commit SHAs with a
trailing version comment so the upstream release is still visible to
reviewers. Dependabot/Renovate can keep these fresh going forward.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 00:13:40 +00:00
Karthik Kalyan 94572d741f ci: publish gh-pages updates via signed GraphQL commits (#2047)
The repo's enterprise `~ALL` required-signatures ruleset rejects the
unsigned commits produced by `peaceiris/actions-gh-pages@v4`, breaking
the benchmark and E2E result publishing jobs on `main`. Replace those
steps with a shared composite action that uses the GraphQL
`createCommitOnBranch` mutation — same pattern already used by
`backport.yml` — so commits are signed automatically by GitHub and
satisfy the rule.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 16:25:54 -07:00
Pranay Prakash 4708a77a35 CI: drop setup-command input from reusable community-world workflows (#1828)
* drop setup-command input from reusable community-world workflows

The community-world matrix is produced by running
scripts/create-community-worlds-matrix.mjs in the fork PR's checkout,
so any field on it is attacker-controlled. Forwarding
matrix.world.setup-command into the reusable workflow and eval-ing it
let a malicious fork PR execute arbitrary shell on the runner.

Replace the pass-through with a hardcoded per-world-id case in the
reusable workflows (only turso currently needs a setup step) and drop
the setup field from the matrix generator.

* rename step to "Per-world setup"

Addresses Copilot review feedback: the step no longer executes an
arbitrary command, so the old name was misleading.
2026-05-14 17:14:49 -07:00
JJ Kasper 00a011dee4 Add stable Next.js eager and lazy test coverage (#1747)
* Add stable Next.js eager and lazy test coverage

* Address PR review feedback

* Fix eager Next step route builds

* Fix eager Next manifest refreshes

* Fix eager Next e2e stack assertions

* Externalize native step bundle bindings

* Lazy load Vercel world runtime

* Fix Next dev step sourcemap assertions

* Consolidate eager build changesets

* Fix Vercel world tracing in Next deployments

* Externalize Vercel world in Next builds

* Fix webpack tracing for Vercel world deps

* Fix eager workflow route bundling

* Rely on Next server externals
2026-05-04 21:09:20 +00:00
Peter Wielander 26de71b9f8 [ci] Enable Vercel-prod e2e for tanstack-start (#1904) 2026-05-04 10:20:28 +00:00
Peter Wielander 8ea1532e48 [core] Combine flow+step bundle and process steps eagerly (#1338) 2026-05-04 09:53:02 +00:00
Nathan Rajlich 059821cb39 ci: pass stale-banner via path: to sticky-pull-request-comment in tests + benchmarks workflows (#1887)
* Pass stale-banner via path: to sticky-pull-request-comment instead of message:

The 'Update existing test comment with stale warning' step inlined the
previous comment body via ${{ steps.get-comment.outputs.previous-results }}
into the action's `message:` input. As the test matrix grows, the
resulting argv can exceed ARG_MAX and the action fails with
'Argument list too long' — observed on a feature branch where the
matrix doubled.

Write the rendered stale-banner message to
$RUNNER_TEMP/stale-comment.md in the github-script step and pass the
path to sticky-pull-request-comment via its `path:` input instead.
This is robust to any future matrix size.

* Apply same fix to benchmarks.yml

Same ARG_MAX hazard exists in the benchmark workflow's stale-warning
step. Apply the identical `path:`-instead-of-`message:` refactor:

- The github-script step now writes the rendered stale-banner to
  $RUNNER_TEMP/stale-comment.md and exposes the path as a step output.
- The sticky-pull-request-comment 'Update existing benchmark comment
  with stale warning' step uses `path:` instead of inlining
  ${{ steps.get-comment.outputs.previous-results }} via `message:`.

The final 'Update PR comment with results' step in this workflow
already used `path: benchmark-summary.md`; only the stale-banner
update was inlined.

* Use `github.run_started_at` for stale-comment timestamps

The 'Started at:' label was sourced from `github.event.pull_request.updated_at`,
which is the PR metadata-update timestamp — not the workflow run start
time. That made the displayed timestamp:
- coupled to PR edits (label changes, description edits, etc.) rather
  than to the actual CI run, and
- stale on workflow re-runs (an empty re-run would still show the
  original PR-update time).

Switch all six occurrences across `tests.yml` and `benchmarks.yml` to
`github.run_started_at`, the canonical "this CI run started at"
timestamp.
2026-05-04 04:09:37 +00:00
Peter Wielander 8202663857 [workbench] Add TanStack Start workbench and tests (#1875) 2026-05-04 00:42:44 +00:00
Nathan Rajlich cd50618d1f ci: switch Vercel deployment-protection bypass to OIDC Trusted Sources (#1882)
* ci: switch Vercel deployment-protection bypass to OIDC Trusted Sources

The e2e, benchmark, and docs-smoke CI jobs previously used the static
`VERCEL_AUTOMATION_BYPASS_SECRET` deployment-protection bypass token
to reach protected Vercel deployments. Switch them over to the new OIDC
Trusted Sources flow: the GitHub Actions runner mints a short-lived
OIDC token via `core.getIDToken()` and forwards it on requests in the
`x-vercel-trusted-oidc-idp-token` header.

Each workbench project (and `workflow-docs`) has been configured with a
matching trusted-source rule:
  aud=https://github.com/vercel, repository=vercel/workflow

The shared header helper now lives at `scripts/trusted-sources-headers.mjs`
and is imported by both the e2e/bench tests and the docs smoke script,
removing the previous duplication.

* rename to VERCEL_OIDC_TOKEN and wire through world-vercel

- Rename the env var from VERCEL_TRUSTED_OIDC_TOKEN to VERCEL_OIDC_TOKEN
  to match Vercel's convention (also read by @vercel/oidc's
  getVercelOidcToken()).
- In @workflow/world-vercel, replace the legacy
  VERCEL_WORKFLOW_SERVER_PROTECTION_BYPASS / x-vercel-protection-bypass
  flow with VERCEL_OIDC_TOKEN / x-vercel-trusted-oidc-idp-token. The
  trusted-source header is attached on every outbound workflow-server
  request (both proxied through api.vercel.com and direct).
- Drop the bypass header from the encryption-key and
  resolve-latest-deployment fetches: those go to api.vercel.com which
  is public.
- Drop VERCEL_WORKFLOW_SERVER_PROTECTION_BYPASS plumbing from tests.yml.
- Update the pending world-vercel changeset to describe the final
  trusted-sources flow.

* .

* .

* ci: add statuses:read permission for wait-for-vercel-project action

The action queries /commits/{sha}/status (Commit Statuses API) in addition
to the Deployments API, in order to extract the Vercel `dpl_...` ID. With
an explicit permissions block in place, GITHUB_TOKEN now needs
`statuses: read` or the action 403s when resolving the deployment ID.

Reported by Copilot review on #1882.

* ci(docs): log status code and body when waitForServer times out

Helps diagnose deployment-protection / OIDC-trusted-source bypass
failures (e.g. SSO redirects) on the workflow-docs preview.

* ci(docs): log OIDC token claims (aud, repository, etc.) for diagnostics

Helps determine whether the bypass is failing because of missing
trusted-source config, claim mismatch, or audience mismatch.

* ci(docs): add curl debug step to verify OIDC header reaches Vercel

* .

* ci: remove debug logging now that trusted-sources config is correct

The fetch-failure root cause was the trusted-sources rule format: the
labs workbench projects had been PATCHed with just `to.slugs` (no
`preset`), but Vercel's edge requires the dashboard-form-style
`to.preset: 'all-custom'` field plus `development` in the slug list to
match incoming requests. After re-PATCHing all projects with the
correct format, the bypass works end-to-end.

* ci(docs): debug — test trusted-sources bypass against docs and labs deployments

Trying repository_owner claim added to one labs project to see if that
fixes the bypass.

* ci(docs): revert curl debug step

The GitHub Actions OIDC trusted-sources bypass returns 401 on all tested
projects regardless of claim configuration (including workflow-docs which
was set up via the dashboard). This is not a per-project config issue.
Need to investigate with Vercel team before continuing.

* ci(docs): probe trusted-sources bypass and surface x-vercel-id

Adds a debug step that does two HEAD requests against the docs preview
deployment (with and without the OIDC trusted-sources header) and prints
the response status line plus `x-vercel-id` for each. The proxy-side
trusted-sources changes for GitHub Actions OIDC tokens are rolling out
gradually (~12+ hours), so the edge-node identifier in `x-vercel-id`
helps explain why a request might succeed or fail during the rollout
window.

Also includes `x-vercel-id` in the `waitForServer` timeout error so
post-mortem analysis of failing runs has the same edge-node info.

* ci(docs): drop trusted-sources curl probe — bypass works once proxy fix reaches the serving edge node

The probe served its purpose: confirmed the bypass is functional once
the request lands on a region that has the proxy-side trusted-sources
fix rolled out. The waitForServer error message still surfaces
x-vercel-id for any future rollout-window debugging.

* .

* world-vercel: log outbound OIDC token claims once per process

Adds a one-shot diagnostic that prints the non-sensitive claims of the
OIDC token (`iss`, `aud`, `owner_id`, `project_id`, `environment`,
`sub`, `scope`, `exp`) on the first request that uses bearer auth.

This is invaluable for debugging Vercel deployment-protection
trusted-source rule mismatches: a 401 from the edge tells you nothing
about why the rule didn't match, and the token's claims are the only
thing that determines that. The signature is never logged.

Gated to once per process — Vercel-issued tokens are process-stable for
the lambda's lifetime so further log lines would just be redundant
spam.

* world-vercel: route trusted-sources header through getVercelOidcToken()

The Authorization bearer correctly preferred config.token (a static
Vercel auth token from CLI / Actions runner) and fell back to
getVercelOidcToken() inside a Vercel function. But the trusted-sources
bypass header (x-vercel-trusted-oidc-idp-token) was being read directly
from process.env.VERCEL_OIDC_TOKEN inside getHeaders(). That env var is
the bake-time token, frozen at deployment-creation time — on a project
that has been redeployed after a settings change, it carries stale
claims (e.g. an iss from when the project was briefly in 'global' mode)
that no longer match the workflow-server's trusted-sources rule.

Move trusted-sources header attachment from getHeaders() (sync) to
getHttpConfig() (async) and source it from getVercelOidcToken(). That
function reads getContext().headers['x-vercel-oidc-token'] first — a
freshly minted per-request token that always reflects current project
settings — and only falls back to the env var when that header is
missing.

Bearer auth source remains config.token-first.

Also expand the diagnostic to log claims from BOTH the per-request OIDC
token AND the bake-time env var so the divergence is visible in logs
when debugging future trusted-source mismatches.

Removes the now-misleading getProtectionBypassHeader() helper (its
'read env var directly' semantics were exactly the bug).

* world-vercel: skip OIDC trusted-sources header on proxied path

The two outbound flows have different auth requirements:

  1. Proxied (usingProxy=true) — calls api.vercel.com/v1/workflow.
     Public endpoint, authenticated with a static Vercel auth token via
     config.token. The api-workflow proxy mints its own OIDC token
     before forwarding to workflow-server, so the trusted-sources
     bypass header on the SDK→proxy hop is meaningless. CLI, GitHub
     Actions, and other API-client callers take this path.

  2. Direct (usingProxy=false) — runs inside a Vercel deployment
     talking straight to workflow-server. workflow-server validates a
     Vercel OIDC bearer; Vercel's edge validates the trusted-sources
     header. Both must come from getVercelOidcToken() (the per-request
     fresh token), not process.env.VERCEL_OIDC_TOKEN (the bake-time
     token that can be stale after a project config change).

Previously getHttpConfig attached x-vercel-trusted-oidc-idp-token on
both paths whenever getVercelOidcToken() resolved. That accidentally
forwarded the GitHub Actions OIDC token (when wired into
VERCEL_OIDC_TOKEN by the test runner) onto every SDK→proxy request,
which is harmless but wrong-by-design — the proxy is public, doesn't
look at that header on its inbound side, and the GHA token isn't its
intended audience.

Bearer auth source rules:
  - Proxied: only config.token. (No fallback to OIDC; that auth
    pathway doesn't go through the proxy's auth checks.)
  - Direct: config.token (for tests / local dev), falling back to
    getVercelOidcToken() (for Vercel-runtime calls).

* world-vercel: throw if proxied path is hit without a Vercel auth token

The api-workflow proxy authenticates the caller with a regular Vercel
auth token (not OIDC), so reaching the proxied path with no
config.token is always wrong: the proxy will reject the request and
the SDK caller would see an opaque 401 with no actionable hint.

Throw at config-resolution time with a clear message that points to
the WORKFLOW_VERCEL_AUTH_TOKEN env var the SDK reads from. Adds tests
covering both the no-token-throws case and the with-token-attaches-
bearer-and-skips-trusted-sources case.

* test(e2e): include x-vercel-id in startWorkflowViaHttp error message

When the trusted-sources bypass returns 401, the error message now
surfaces the response's x-vercel-id header so we can identify which
edge node served the failure. Helps distinguish proxy-rollout
incompleteness from actual config errors during incremental
rollouts of edge-side changes.

* ci: mint GHA OIDC tokens on demand to survive 5-minute expiry

GitHub Actions OIDC tokens have a hard 5-minute lifetime that cannot be
extended (no API to ask for a longer TTL — exp is always iat + ~300s).
Pre-minting once at the start of the job and shipping the result down
to the test runner via env var means tests that run late in the suite
hit an expired token and 401 on /api/trigger-pages (and any other
trusted-sources protected endpoint).

Move minting into scripts/trusted-sources-headers.mjs:
  - getTrustedSourcesHeaders() is now async.
  - It calls the runner's ACTIONS_ID_TOKEN_REQUEST_URL endpoint directly
    (the env vars GHA exposes when permissions: id-token: write is on)
    and re-mints 60s before the cached token's exp.
  - Falls back to process.env.VERCEL_OIDC_TOKEN for non-GHA contexts
    (Vercel runtime, local dev).

Workflow files drop the now-redundant 'Mint OIDC token' step and the
VERCEL_OIDC_TOKEN env-var passthrough on the test step. The runner env
vars propagate to subsequent steps automatically.

Updates all 17 callers in e2e.test.ts / bench.bench.ts / utils.ts /
docs/scripts/check-docs-smoke.mjs to await the now-async call.

* address PR #1882 code review

- Drop `statuses: read` from the three workflow permission blocks (the
  wait-for-vercel-project action works without it on a public repo).
- Revert the `x-vercel-id` debug logging in `startWorkflowViaHttp`.
- Delete `packages/world-vercel/src/jwt-claims.ts` (debug-only helper).
- Drop the JWT claims diagnostic logging from `getHttpConfig`.
- Tighten the auth-flow comment in `getHttpConfig` and remove the
  historical 'no longer attaches' note from `getHeaders`/its test.
- Restore `.changeset/world-vercel-protection-bypass.md` (already
  shipped in a beta release per .changeset/pre.json).
- Trim the `.changeset/world-vercel-trusted-sources.md` description to
  one short paragraph.

* docs(AGENTS): document local VERCEL_OIDC_TOKEN via vercel env pull

Configured trustedSources.projects on all 11 workbench app projects so
each one accepts a Vercel-issued OIDC token from any of the others. A
developer running e2e locally can now do `vercel env pull` from any
workbench app's directory and use the resulting VERCEL_OIDC_TOKEN to
bypass Deployment Protection on any of the workbench preview/prod
deployments — no need to disable protection on the project just to run
the suite locally.
2026-05-02 19:21:52 +09:00
Nathan Rajlich 2d66c75ad0 ci: fail fast when Next.js dev server is wedged on Windows (#1871)
A recurring Turbopack-on-Windows bug causes the dev server to enter a
'MODULE_UNPARSABLE' state during HMR in dev.test.ts, after which every
request returns 500. The remaining e2e suite then polls stuck workflows
for 60s each, burning the full 30-minute job window before getting
cancelled (~50% of recent main runs).

Bail out of the Windows e2e job as soon as dev.test.ts fails, and
health-check the dev server before kicking off test:e2e so any other
silent breakage is surfaced quickly instead of via a 30-minute timeout.
2026-04-30 00:42:09 -07:00
Nathan Rajlich 3a08eaa0a1 ci: refactor wait-for-vercel-project to use GitHub Deployments API (#1861)
* ci: refactor wait-for-vercel-project to use GitHub Deployments API

Replaces the Vercel SDK / Vercel API token-based implementation with one
that resolves the deployment URL via the GitHub Deployments API:

- Find the GitHub Deployment for (target SHA, environment) where
  environment matches the Vercel-app-created "Preview \u2013 <slug>" or
  "Production \u2013 <slug>" naming pattern.
- Wait for the latest deployment status to be `success` (or `inactive`
  when Vercel skips a duplicate build, in which case its environment_url
  still points at the live deployment).
- Probe the URL to confirm the edge can route to it (any non-5xx
  response counts as live, including 401/403 from Deployment Protection
  and 404/405 from the app). Manual redirect handling treats redirects
  to vercel.com as "still building".
- Resolve the dpl_xxx deployment ID from the matching commit status
  (Vercel posts `Vercel \u2013 <slug>` statuses where target_url's last
  path segment is the inspector ID == deployment ID without the prefix).

Inputs change: project-slug + bypass-secret + github-token (with
GITHUB_TOKEN default) replace team-id + project-id + vercel-token.

Removes the @vercel/sdk dependency, shrinking the bundled dist from
5.4MB to 829KB. The VERCEL_DOCS_TOKEN secret is no longer referenced
anywhere in the repo and can be deleted from GH after this lands.

* ci(wait-for-vercel-project): drop URL probe and bypass-secret input

The GitHub Deployment status transitions to `success` only after the
Vercel app finishes building and routing is live, so an extra HTTP
liveness probe of the deployment URL was redundant. Removing it lets
us also drop the bypass-secret input \u2014 protected deployments don't
need a workaround anymore because we never make the request.

Reduces the action surface area and eliminates a runtime fetch.

* ci(wait-for-vercel-project): address PR review

- Fail loudly when the dpl_xxx deployment ID can't be resolved instead
  of returning an empty string. Consumers wire this into
  VERCEL_DEPLOYMENT_ID, which world-target uses to pick between the
  vercel and local worlds (packages/utils/src/world-target.ts), so an
  empty value would silently flip execution mode.
- Pass the GitHub App token to wait-for-vercel-project in the dispatch
  release workflow. The job sets `permissions: contents: read`, which
  blocks the default GITHUB_TOKEN from reading the Deployments API.
  The App token (already generated for workflow,front) has the
  necessary scopes.
2026-04-28 20:20:47 +00:00
Nathan Rajlich 28dc089eef ci: fix VERCEL_WORKFLOW_SERVER_* env var ternary on main (#1859)
* ci: fix VERCEL_WORKFLOW_SERVER_* ternary so main actually unsets them

In GitHub Actions expressions, '' is falsy, so the original
`cond && '' || secrets.X` pattern always fell through to the secret
regardless of branch. The result was that pushes to main were sending
preview workflow-server values to production, causing 'invalid_url'
errors on `x-vercel-workflow-api-url` across all e2e jobs.

Flip the condition so the secret sits in the truthy branch and ||
correctly selects '' on main.

* ci: skip pnpm cache in matrix-generation jobs

The Get Test Matrix and Get Community Worlds Matrix jobs only run a
small Node script to emit a JSON matrix; they never run `pnpm install`.
With `cache: 'pnpm'` set on actions/setup-node, the post-job cache save
step fails with 'Path Validation Error' because the pnpm store path was
never created, marking the whole job as failed.

Add a cache-pnpm input to setup-workflow-dev (default true) and opt out
in the two matrix-generation jobs.
2026-04-28 10:26:04 -07:00
JJ Kasper 393ffd86fc Lock next@canary test version (#1860) 2026-04-28 10:25:34 -07:00
Pranay Prakash e2ef3568a5 CI script improvements (#1826)
* CI script improvements

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* address codex feedback: harden remaining workflows

- add allowlist regex in prepare-workbench-path to block path traversal
- move matrix/input values to env vars across e2e-vercel-prod,
  benchmarks (local/postgres/vercel), and the reusable community-world
  workflows
- validate app-name/world-id/world-package inputs in the reusable
  community-world workflows
- pipe getCommunityWorldsMatrix script output through jq -c to prevent
  \$GITHUB_OUTPUT injection

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 23:00:27 -07:00
Nathan Rajlich 354840e93b feat(world-vercel): support new env vars for Vercel Deployment Protection (#1824)
* feat(world-vercel): support WORKFLOW_VERCEL_PROTECTION_BYPASS env var

Allows sending a Vercel Deployment Protection bypass secret via the
`x-vercel-protection-bypass` header on all outbound requests made by
the Vercel world, enabling use against protected deployments (e.g.
previews, or workflow-server once protection is enabled).

* feat(world-vercel): support VERCEL_WORKFLOW_SERVER_URL env var

Replace hard-coded WORKFLOW_SERVER_URL_OVERRIDE constant with a function
that reads from the VERCEL_WORKFLOW_SERVER_URL env var. Allows configuring
the workflow-server URL per-deployment (e.g. workbench Preview envs
pointing to a branch deployment) without editing source.

* fix(world-vercel): preserve inline WORKFLOW_SERVER_URL_OVERRIDE const

Keep the inline const as an empty-string literal so external CI rewrite
tooling continues to work unmodified; the env var is a fallback when the
inline value is empty.

* refactor(world-vercel): rename to VERCEL_WORKFLOW_SERVER_PROTECTION_BYPASS

Align env var naming with VERCEL_WORKFLOW_SERVER_URL.

* ci: expose workflow-server protection bypass env vars to e2e-vercel-prod

Set VERCEL_WORKFLOW_SERVER_URL and VERCEL_WORKFLOW_SERVER_PROTECTION_BYPASS
on PR runs so e2e tests hit the protected workflow-server preview; leave
unset on main so production runs use the public default URL.

* refactor(world-vercel): address PR review comments

- Consolidate bypass header logic in getHeaders() to reuse
  getProtectionBypassHeader() instead of duplicating env lookup.
- Use consistent 'Authorization' casing in direct fetch() calls.
- Add unit tests for getProtectionBypassHeader, getHttpUrl, and getHeaders
  covering env var toggling and proxy/override combinations.
2026-04-21 17:43:30 -07:00
Peter Wielander 71d594c5dd [ci] Skip community world E2E tests on main (#1783) 2026-04-16 18:34:18 -07:00
Nathan Rajlich 69af0c1374 ci: upgrade pnpm/action-setup to v5 and read version from package.json (#1785)
* ci: upgrade pnpm/action-setup to v6 and read version from package.json

Removes hardcoded pnpm version (10.14.0) from all workflows and instead
reads the version from the packageManager field in package.json, so CI
stays in sync with the version used locally.

* ci: update setup-workflow-dev composite action to use pnpm/action-setup@v6

Also removes the pnpm-version input since the action now reads the
version from package.json#packageManager.

* ci: downgrade pnpm/action-setup to v5

v6 installs pnpm 11 RC/beta, which has a regression
(pnpm/pnpm#11264, pnpm/action-setup#225/#227/#228) that causes
'ERR_PNPM_BROKEN_LOCKFILE: expected a single document in the stream'
when the project's packageManager pins a 10.x pnpm version. v5 is the
latest stable release before v6 and supports reading the version from
package.json#packageManager.
2026-04-17 01:17:27 +00:00
Pranay Prakash cd4abd80fe test: improve e2e test failure diagnostics (#1426)
* test: improve e2e test failure diagnostics with run context and GitHub annotations

When e2e tests fail, automatically dump workflow run diagnostics (status,
input/output, error details, event timeline, dashboard link) to the CI
logs. Emit GitHub Actions annotations that surface on PR file diffs.
Fix collectedRunIds which was declared but never populated, enabling
observability links in the PR comment. Enrich the aggregation script
to include run IDs and dashboard URLs for failed tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: increase diagnostics hook timeout and fix flaky Vercel Prod tests

- Increase onTestFailed hook timeout to 30s (default was 10s) so
  diagnostics can fetch run data even after slow test timeouts
- parallelSleepWorkflow: increase elapsed threshold from 10s to 25s to
  accommodate Vercel cold start latency
- webhookWorkflow: increase hook polling deadline from 30s to 60s and
  test timeout from 60s to 120s for slow Vercel webhook registration
- readableStreamWorkflow: stop reading once expected content is received
  instead of waiting for stream close (which can hang on Vercel), and
  increase test timeout to 120s

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: emit ::error annotations via process.stdout.write to bypass vitest ANSI prefix

Vitest's console interceptor prepends ANSI escape codes to console.log
output, which prevents GitHub Actions from parsing ::error workflow
commands. Use process.stdout.write() directly to ensure clean output.

Also enhance the custom reporter to emit annotations in onFinished
(which runs after vitest output is complete) as a reliable fallback,
and enrich failure data from the diagnostics sidecar.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: only show observability links for vercel-prod test failures

Community world and local tests don't run on Vercel's backend, so
dashboard links are meaningless for those categories. Previously,
test name collisions across sidecar files could cause community
test failures to show Vercel dashboard URLs from vercel-prod runs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: link annotations to test files instead of symlinked workflow sources

The workflow source files in workbench/ are symlinks that GitHub can't
resolve, causing annotations to show raw paths like #L0 instead of
linking to code. Now:
- utils.ts: omit file= from onTestFailed annotations (just show title)
- github-reporter.ts: use the actual test file path (e.g.
  packages/core/e2e/e2e.test.ts) which GitHub can resolve

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 19:47:26 -07:00
Peter Wielander 40c2d95a3a Enable DurableAgent tests (#1411) 2026-03-17 00:09:02 +00:00
Peter Wielander 456c1aa455 Add vitest plugin for testing full workflows without setting up a server (#1237) 2026-03-05 16:08:48 -08:00
JJ Kasper a9fea9132e Update workbench tests to build and run outside of monorepo (#1230)
* Setup fixes

* ci: run local e2e against staged tarball workbenches

* ci: update staged workbench tarball setup script

* chore: set nextjs workbenches back to next 16.1.6

* update lock

* test(e2e): resolve workbench path from WORKBENCH_APP_PATH

* fix: address deferred builder issues outside monorepo

* ci: stage tarball workbenches only for nextjs local e2e

* fix(next): discover deferred steps imported via workflows

* test(core): gate deferred step-discovery dev test to canary

* test(e2e): cover cross-file imported step in build/start lanes

* fix(e2e): use local manifest in local runs and relax dev rebuild timeout

* fix(workbench): add imported-step workflow symlink for sveltekit/astro

* test(e2e): scope imported-step workflow test to nextjs lanes

* fix(next): rebuild deferred entries on discovered file updates

* fix(next): watch transitive deferred step deps for dev rebuilds

* fix(next): restore socket-driven deferred step rebuilds

* add changeset

* chore: address review feedback on deferred e2e updates

* fix(cli): guard stream flush against closed write streams
2026-03-03 11:17:39 -08:00
Nathan Rajlich 81a883bc9b ci: don't cancel in-progress CI runs on main branch (#1166)
* fix: use types.isNativeError() for cross-VM Error serialization

FatalError was not properly serialized when passed from workflow code into a step function because the Error reducer checked `value instanceof global.Error` where `global` is the VM's globalThis. Errors created in the host context (like FatalError from @workflow/errors) have a different Error prototype than the VM context, so the instanceof check returned false and the error was silently dropped.

Replaced with `types.isNativeError()` from `node:util` which uses V8's internal type tag and works across VM context boundaries.

* ci: don't cancel in-progress CI runs on main branch
2026-02-24 06:50:59 +00:00
JJ Kasper b733e2a173 Unlock canary Next.js version in tests and bump min deferred version (#1088)
* Unlock canary Next.js version

* bump min version
2026-02-16 17:27:16 -08:00
JJ Kasper 550997659d Update tests.yml triggers for release PR (#1048) 2026-02-14 02:31:22 +00:00
JJ Kasper 565e5ebd56 Lock Next.js canary version (#1054) 2026-02-13 23:13:44 +00:00
JJ Kasper 26399c107a ci: add 30-minute timeouts to test jobs (#1052) 2026-02-13 14:45:23 -08:00
Peter Wielander b2ba6ff70e [ci] Add flag that can disable non-vercel e2e tests (#1006) 2026-02-11 16:53:55 -08:00
JJ Kasper 8cb7be5f48 Ensure unit tests are included in required check (#1001) 2026-02-10 23:25:48 -08:00
JJ Kasper d8a9ee9c29 ci: add final E2E required-check job (#996)
* ci: add final e2e required check job

* bump
2026-02-10 15:30:42 -08:00
JJ Kasper da64519db8 ci: use PR head SHA for tests workflow checkout (#988)
Signed-off-by: JJ Kasper <jj@jjsweb.site>
2026-02-09 16:51:16 -08:00
Nathan Rajlich 86f62f2779 Refactor e2e tests to no longer use "trigger" endpoint (#958)
## Summary

Refactors the E2E tests to call `start()` from `workflow/api` directly instead of going through the `/api/trigger` HTTP endpoint in each workbench app. This removes a layer of indirection — the tests now use the same API that users would use to start workflows programmatically.

### Before

```ts
const run = await triggerWorkflow('addTenWorkflow', [123]);
const returnValue = await getWorkflowReturnValue(run.runId);
```

- `triggerWorkflow()` sent an HTTP POST to `/api/trigger` on the workbench app
- The workbench app looked up the workflow function, called `start()`, and returned the run ID
- `getWorkflowReturnValue()` polled `GET /api/trigger?runId=...` until the workflow completed

### After

```ts
const run = await start(await e2e('addTenWorkflow'), [123]);
const returnValue = await run.returnValue;
```

- `e2e()` / `getWorkflowMetadata()` fetches the manifest from `/.well-known/workflow/v1/manifest.json` to look up the correct `workflowId`
- `start()` is called directly from the test process via the configured World
- `run.returnValue` polls for completion via the World (no HTTP polling endpoint needed)

### Changes

**`packages/core/e2e/e2e.test.ts`**
- Removed `triggerWorkflow()` and `getWorkflowReturnValue()` helpers
- Added `fetchManifest()` to fetch and cache the workflow manifest from the deployment
- Added `getWorkflowMetadata(file, fn)` to look up `{ workflowId }` from the manifest
- Added `e2e(fn)` shorthand for the common case of `workflows/99_e2e.ts`
- All tests call `start()` and `run.returnValue` directly
- Error tests use `.catch()` to inspect `WorkflowRunFailedError`
- Output stream tests use `run.getReadable()` directly (skipped on local world where cross-process streaming isn't supported)
- `beforeAll` configures the local World with the correct data directory and base URL
- Pages Router tests use `startWorkflowViaHttp()` to specifically validate the HTTP trigger path

**Workbench apps (hono, express, fastify, nest)**
- Removed `/api/trigger` route handlers
- Kept `/api/hook`, `/api/test-direct-step-call`, `/api/test-health-check` endpoints
- Re-added `_workflows.js` side-effect import for hono/express/fastify to maintain Nitro's HMR dependency graph

**Deleted trigger-only route files** from: nextjs-turbopack, nextjs-webpack, vite, sveltekit, astro, nuxt, nitro-v2, nitro-v3, example

**`.github/workflows/tests.yml`**
- Added `WORKFLOW_PUBLIC_MANIFEST: '1'` to all E2E test jobs

### Dependencies

Stacked on #963 which adds `WORKFLOW_PUBLIC_MANIFEST` support to all framework builders.
2026-02-06 16:25:47 -08:00
Nathan Rajlich c28d6ca660 Upload Next.js server logs as artifact for Windows E2E (#946) 2026-02-05 11:10:22 -08:00
Nathan Rajlich bd8116d40b Add WORKFLOW_SERVER_URL_OVERRIDE var to "world-vercel" for testing (#833)
Added `WORKFLOW_SERVER_URL_OVERRIDE` configuration to the Vercel world adapter and removed the deprecated `WORKFLOW_VERCEL_SKIP_PROXY` and `WORKFLOW_VERCEL_BACKEND_URL` environment variables.

### What changed?

- Added a changeset for a patch release across multiple packages
- Removed `WORKFLOW_VERCEL_SKIP_PROXY` environment variable from GitHub workflow tests
- Removed `WORKFLOW_VERCEL_BACKEND_URL` from environment variables in CLI and core packages
- Simplified the URL resolution logic in the Vercel world adapter
- Added support for a `WORKFLOW_SERVER_URL_OVERRIDE` constant for testing against different workflow-server versions
- Added the `x-vercel-workflow-api-url` header when the URL override is set

### How to test?

1. Verify that Vercel deployments continue to work without the removed environment variables
2. Test with a custom workflow server URL by setting the `WORKFLOW_SERVER_URL_OVERRIDE` constant in the world-vercel package

### Why make this change?

This change simplifies the configuration for the Vercel world adapter by removing deprecated environment variables and standardizing on a cleaner approach for specifying the workflow API URL. The new implementation automatically determines whether to use the proxy based on project configuration, making it more intuitive and reducing the need for explicit configuration.
2026-01-23 11:01:45 -08:00