* fix(nitro): use workspaceDir for monorepos
* test: stabilize Next canary HMR e2e
Signed-off-by: Nathan Colosimo <110621881+NathanColosimo@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Propagate trace context to vercel-workflow.com in workbench instrumentation
@vercel/otel only propagates W3C trace context to Vercel deployment URLs
by default, so outgoing requests to the workflow-server
(vercel-workflow.com) got a client span with no `traceparent` header —
breaking the APM trace link to workflow-server's spans. Add
`instrumentationConfig.fetch.propagateContextUrls` for the workflow-server
domain in every workbench that uses @vercel/otel: example,
nextjs-turbopack, nextjs-webpack, and sveltekit. The Next.js and SvelteKit
apps already declared @vercel/otel but weren't registering it at all; they
now do.
* Also propagate trace context to the Vercel Queue Service (vercel-queue.com)
The workflow-server queue path (@vercel/queue) sends to regional
vercel-queue.com subdomains (e.g. iad1.vercel-queue.com) when not using the
queues proxy, which were missing a `traceparent` header for the same reason
as vercel-workflow.com. Add `/vercel-queue\.com/` to propagateContextUrls in
all four workbench instrumentation configs.
---------
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
* Redrive on transient workflow-server transport failures instead of failing the run
A firewall in front of workflow-server shedding load with sustained 429/503
makes undici's shared RetryAgent exhaust its retries and throw
UND_ERR_REQ_RETRY. That raw error was rethrown unwrapped, so it was
classified as USER_ERROR and the replay terminal branch wrote run_failed —
permanently failing a run on a transient blip (or, in an outage, falling back
to the ~5min queue visibility-timeout redrive).
- world-vercel: map exhausted-retry / socket / connect / DNS / timeout
failures to a typed WorkflowWorldError (code TRANSPORT/TIMEOUT) by walking
the fetch() cause chain.
- core: add isRetryableWorldError (429 / 5xx / TRANSPORT / TIMEOUT) and
rethrow such errors from the replay terminal branch so the queue redrives
quickly (1s->60s backoff) instead of failing the run. Reuse it in start()
and step_started handling.
- world-vercel: surface the Vercel firewall x-vercel-mitigated
(challenge/deny) header alongside x-vercel-id in error diagnostics and logs.
* Address review: back off + cap on every retry path; fix mock; refine scope
Revises the transport-error handling per PR review (VaguelySerious,
karthikscale3).
Blocking fix — step_started no longer self-enqueues a throttled defer for
transient world errors. Returning `{ type: 'throttled', timeoutSeconds: 1 }`
acked the delivery and enqueued a fresh message, resetting the delivery count
so the path never backed off and never reached MAX_QUEUE_DELIVERIES — an
unbounded flat-1s loop if step_started kept failing. It now throws, so the
error flows through the replay loop's retryable-world-error rethrow and earns
both the delivery-count backoff and the max-delivery cap. Throwing is safe on
step_started (the body hasn't run; a write that landed dedupes to skipped).
Also in this revision:
- Backoff that lasts: raise the queue handler-error retry ceiling 60s -> 900s.
VQS clamps each redelivery to its 900s SQS limit and adds its own post-32
exponential, so ramping our base toward 900s stretches survival from ~3.7h to
most of the 24h message-visibility window. Corrected the stale
MAX_QUEUE_DELIVERIES comment to match the real VQS schedule.
- Stop amplifying firewall challenges: the undici RetryAgent no longer retries
429 in-process (a challenge is a 429 the client can't solve). 429s surface
immediately as ThrottleError carrying x-vercel-mitigated / x-vercel-id, so
the diagnostic header now reaches us for the challenge case too.
- Track world faults as WORLD_CONTRACT_ERROR (not USER_ERROR) in
classifyRunError so an outage isn't attributed to user code.
- Fix queue.test.ts mock that `biome check --write` had rewritten from a
newable `function` into an arrow (broke `new QueueClient`); pin with a
biome-ignore.
All Vercel-specific logic stays in @workflow/world-vercel; @workflow/core
operates only on the generic WorkflowWorldError abstraction.
* Update .changeset/transport-error-redrive.md
* Trim changeset to a single sentence per review
* Route firewall-challenge 429s to the retryable transport path, not ThrottleError
A 429 carrying `x-vercel-mitigated: challenge` is a firewall challenge our
server-to-server client cannot solve, so it recurs for the life of the
incident. Mapping it to `ThrottleError` meant the `step_started` write deferred
it as `{ type: 'throttled' }`, which self-enqueues a FRESH queue message and
resets the delivery count — so it never backed off past `retryAfter` and never
reached `MAX_QUEUE_DELIVERIES`, hot-looping against an already-overloaded
firewall (the exact amplification this PR set out to remove, and contrary to
the "step_started can't loop unbounded" invariant, which only held for 5xx).
Map a challenge to a retryable transport `WorkflowWorldError` (`code:
'TRANSPORT'`) in both the v3 `makeRequest` and v4 `throwForErrorResponse`
(the hot event-write path) error mappings, via a shared `isFirewallChallenge429`
helper. It then propagates through the V1/V2 step paths and the replay loop's
retryable-world-error rethrow, earning the delivery-count backoff AND the
delivery cap. A genuine application-level 429 (no `challenge` mitigation) stays
a `ThrottleError` and keeps its `Retry-After`-paced defer.
Also correct the survival-window comments: with the 900s ceiling,
MAX_QUEUE_DELIVERIES=48 spans ~9-10h (~35,000s), not "the better part of 24h";
reaching 24h would need a higher delivery cap, not a higher per-hop ceiling.
---------
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Co-authored-by: Peter Wielander <peter.wielander@vercel.com>
Move cn to its own lib/cn module that teaches tailwind-merge about the
design-system text-heading/text-label/text-copy/text-button/material
utilities, and import it directly from lib/cn.
Signed-off-by: Mitul Shah <mitulxshah@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* Make world own zod dependency
* Relax zod catalog range
Signed-off-by: Karthik Kalyan <105607645+karthikscale3@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* docs(core): document scheduleWhenIdle macrotask is load-bearing
Revert the synchronous consume-loop drain optimization: it caused a
replay divergence (ReplayDivergenceError on step_started →
CorruptedEventLogError) in the world-testing inline-batches parallel
workflow on the Windows CI runner. The per-event `process.nextTick` in
the consume loop is load-bearing — it guarantees at most one event is
consumed per macrotask, letting the cross-VM `resolve → workflow VM body
→ subscribe()` chain register the next operation's consumer before the
drain advances. A synchronous drain races ahead of that registration.
What remains is a documentation comment on `scheduleWhenIdle` capturing
why its initial `setTimeout(0)` must not be downgraded to a microtask
(empirically: queueMicrotask breaks hook/sleep Promise.race ordering →
CorruptedEventLogError). No behavior change.
* docs(core): use empty changeset for comment-only macrotask doc
The scheduleWhenIdle change is a pure code comment with no consumer-facing
effect, so it does not warrant a patch bump / changelog entry. Replace the
patch changeset with an empty one to satisfy the changeset-bot convention
without claiming a release. Per the PR template's `pnpm changeset --empty`
guidance for non-releasing changes.
* perf(core): drain consumable replay events synchronously
The EventsConsumer rescheduled `process.nextTick(this.consume)` after every
consumed event, so replaying N already-consumable events (structural
lifecycle events, step_created/step_started, completed deliveries) cost N
macrotask hops — O(N) per consume wave across a sequential replay.
Drain consecutively consumable events within a single synchronous pass
instead. This is safe because callbacks only ever consume events with a
consumer that is already registered; new consumers are registered by
workflow VM body code that runs asynchronously off ctx.promiseQueue after a
delivery resolve(). When the next event's consumer is not yet registered,
no callback consumes it and we fall through to the existing cross-VM-safe
deferred unconsumed-event check, exactly as before. A null end-of-events
sentinel never continues the drain, so it cannot spin past end-of-log.
scheduleWhenIdle is intentionally left unchanged: its initial setTimeout(0)
is load-bearing for cross-VM propagation (pendingDeliveries is already 0
between a delivery resolve() and the VM body registering its next
subscriber). Replacing it with queueMicrotask empirically breaks hook/sleep
Promise.race ordering (CorruptedEventLogError); a comment now records this.
Re-validated after a premature revert: the windows-unit flake that prompted
the revert reproduces on unmodified main at the same rate (local 8-way
harness: opt 4/80 vs main 7/80; main historical windows-unit ~13%), so it is
a pre-existing flake, not a regression from this change.
---------
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* perf(core): cache compiled workflow-bundle vm.Script across replays
The inline replay loop calls runWorkflow on every iteration, and each call
re-parsed the entire workflow bundle string via vm.runInContext. For a bundle
containing many workflow definitions (the production shape: one workflow called
per replay), this re-scans every definition on every replay.
Cache the compiled vm.Script per process, keyed by (workflowCode, filename),
and run it against the fresh context instead of recompiling. Compilation is a
pure function of (code, filename), so the result is byte-identical to the
previous re-parse-every-time behaviour — determinism is preserved. filename is
part of the key because it drives source attribution in stack traces (consumed
by remapErrorStack).
Measured per-replay savings scale with bundle size (and multiply by replay
count): ~34% for a 50-workflow app, ~59% for 155 workflows, ~80% for 400.
* perf(core): bound script cache with LRU; soften determinism claim; add tests
Addresses review on #2471:
- Bound `scriptCache` to a small LRU (cap 8 bundle versions). Production
serves one bundle per process so the bound is never reached; it exists for
dev/watch mode, where each edit produces a new bundle string that would
otherwise be pinned forever (~0.8MB/edit, monotonic). Touch-on-access keeps
the latest bundle hot; evicting a `code` entry drops its per-filename scripts
together, restoring pre-cache GC behaviour.
- Document precisely why keying includes `filename` (intentional: drives
stack-trace attribution via `remapErrorStack`; NOT a dedupe key), and that
the whole bundle is compiled once per distinct filename.
- Soften the "byte-identical including thrown errors" claim to
same-workflow-function + same-`filename`-attribution, noting the one caveat:
a lookup-expression error's line number shifts to line 1 of the separate
lookup Script. Updated in both the code comment and the PR description.
- Add tests: cache-is-bounded regression (eviction past the cap), LRU recency
(hot bundle survives churn), and a realistic multi-workflow collision test
(distinct code/filename never returns the wrong Script, results carry their
own bundle marker).
---------
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
github-actions[bot]