On a fan-out that runs some steps inline, the orchestrator invocation
publishes the queued siblings' step-execution messages, runs its inline
steps, falls back into the replay loop, reloads the log, and its
pending-step dispatch pass publishes every one of those messages again
(measured 19-28 re-sends per pass on a 32-branch fan-out, ~400 ms after
the originals). The queue dedupes them by idempotency key, but the sends
still cost round-trips on the shared connection pool and hold the
invocation open past its useful work.
Track, per delivery, the correlation ids this invocation has already
published a step message for (the suspension handler's resilient
publishes plus the dispatch pass's own immediate enqueues) and skip the
immediate re-enqueue for those on later passes, unless a step_retrying
has been observed since (a new schedule). The set is invocation-scoped
and never derived from the log: a step_created does not prove the message
was ever sent, so a different delivery still re-enqueues unconditionally.
Reported as a debug log and the `workflow.dispatch.republish_skipped`
span attribute. The QuickJS engine already keeps the equivalent
invocation-scoped `queuedStepIds` set, so it is unchanged.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The only thing a World does with `eventCount` is bump-and-report: when the
write lands above the position named, it reads the events in between and
returns them so the writer can merge them without a second round-trip. The
replay loop and the suspension handler merge that page into their loaded
log. The step executor has no log to merge into, so it took the page's
highest position and discarded the rest.
In production that discarded read fell on a third of all `step_started`
writes (10.8M of 13.4M skipped-slot report reads per day were on executor
event types), each a strongly consistent DynamoDB query on the run
partition with resolved refs, on the response path. This removes the
executor's `knownSlot` / `observeSlot` machinery, the `slotSnapshot`
executor param, and the `batchCommittedSlotCeiling` the suspension handler
computed only to seed it. The loop's and the suspension handler's own
snapshots are unchanged; they consume their reports.
The World contract already describes omitting the count for a caller with
no loaded log to be stale against; the executor now matches it.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* perf(world-vercel): batch a fan-out's step-execution queue publishes
A `Promise.all` fan-out dispatched one queue message per branch. Those
publishes ride the shared default undici agent (8 connections, HTTP/1.1,
`pipelining: 1` — see `getQueueDispatcher`), and `handleSuspension` is
awaited in full before the first inline step body runs, so an N-branch
fan-out paid ~N/8 serialized round trips straight onto time-to-first-step.
The `step_created` writes were already batched and HTTP/2-multiplexed; the
publishes were the remaining per-branch round trip.
Adds an optional `Queue.queueBatch`, implemented on `@vercel/queue`'s
`experimental_sendBatch` (0.5.1), and uses it for the batched fan-out fold's
publishes. Each commit chunk now publishes in one request instead of up to
32.
`queueBatch` reports per-entry outcomes rather than throwing, because a
batch can partially fail. `queueMessages` in core keeps the previous
all-or-nothing behavior for this call site: it rejects if any entry failed,
so the delivery is redelivered and republishes the set, deduped by the
per-step `idempotencyKey` the caller already passed. Worlds without
`queueBatch` fall back to concurrent single sends.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(core): reject a short queueBatch result set instead of reading it as success
`queueMessages` only inspected `error`, so a World whose `queueBatch`
returned fewer results than it was given messages reported success for the
whole batch. The omitted entries were never published and nothing raised:
`handleSuspension` resolved, the delivery was acked, and those steps were
never dispatched, so the run stalls with no error recorded anywhere.
Reproduced at 64 branches against a World returning half its results: 32 of
63 steps silently lost.
world-vercel guards this internally and `@vercel/queue` length-checks its
own response, so it was not reachable through the world added here. It is
reachable through the interface `building-a-world` opens to third-party
worlds, which is where the check belongs. Documented on the interface and
in the guide alongside it.
Also notes that the batch grouping degenerates to one request per message
under WORKFLOW_SEQUENTIAL_REPLAYS=1 (per-step physical topics are one of
the routing dimensions groups split on), and corrects the comment claiming
the error's `retryable` flag is consumed downstream: nothing reads it yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(world-vercel): carry trace context on each batched queue message
`experimental_sendBatch` injects the active trace context into the multipart
REQUEST headers, and the per-part headers it builds never see it. VQS stores
headers per message and re-emits a stored `traceparent` at delivery as
`x-vercel-queue-traceparent`, which is what lets a consumer attach a span link
back to its producer, so a batched message arrived with no producer context
and its `vqs.process` span got no link. `send()` is unaffected: for a single
message the request headers ARE that message's headers.
At 64 branches that was 63 of 64 step dispatches losing the transport-level
producer link. The run's own step tracing was never affected: that carrier
travels in the message payload (`WorkflowInvokePayload.traceCarrier`), which
is what the consumer builds its trace context from, not a header.
Injects the active context into each entry's headers in `queueBatch` — last,
so it wins over caller-supplied `opts.headers` exactly as the SDK's own
injection does — and honors VERCEL_QUEUE_TRACE_PROPAGATION so that kill
switch still covers both paths. `getTraceContextHeaders()` is factored out of
`injectTraceContextIntoHeaders` so the two share one source.
Verified on the wire against a stub VQS speaking the real batch endpoint:
`traceparent` carrying the producer's traceId/spanId lands on all 64
multipart parts through the real SDK, with the per-message idempotency keys
still alongside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Karthik Kalyanaraman <karthik.kalyanaraman@vercel.com>
## Summary & Motivation
- Manifest coverage now declares an entry for every matrix app, uses real Vitest skips instead of silent early returns, and fails when a targeted app's manifest is missing, unknown, or unparseable.
- Retries are scoped to deployment e2e runs (`DEPLOYMENT_URL` set), so a flaky unit or integration test can no longer be hidden by a second attempt.
- The stop-workflow cookbook parks on a sleep between iterations, giving the hook an observable barrier to race instead of a fixed delay, and the AbortController hook test waits on queue state rather than a 10ms timer.
- The world-postgres direct-storage fixture drives its run to a terminal state so the conformance worker doesn't recover and replay an unregistered workflow.
- Generated e2e result sidecars are ignored and the committed copies removed; they're CI artifacts, not fixtures.
## Test Plan
Existing coverage runs in CI. With retries disabled: the cookbook agent suite passed 8/8, the two stop-workflow tests passed 10/10, and the AbortController hook replay test passed 25/25 under `CI=1`. The manifest suite skips 52 apps explicitly when nothing is built, and fails on unknown or missing targeted apps. The Docker-backed Postgres spec could not run locally (Testcontainers found no container runtime); `@workflow/world-postgres` typechecks.
## Summary & Motivation
### Situation
- Workflow stream writers are expected to call `releaseLock()` when a step finishes writing so another step can acquire the stream.
- The runtime observes that release and drains the server sink, but `step_completed` currently races the overall stream operation against 500ms.
- A slow PUT can therefore continue under `waitUntil` while the next step starts and reads a stale tail.
- Release can also happen while native `writer.write()` promises remain unsettled, leaving frames upstream of the server sink when a naive drain runs.
- Writers intentionally kept locked must remain non-blocking so producer and consumer steps can overlap.
### Fix
- Treat a writer released before step return as an implicit durable handoff boundary.
- At step end, acquire the unlocked stream with a temporary writer and enqueue an internal checkpoint behind all writes queued by the released writer.
- Once the checkpoint crosses serialization, wait for those frames to reach the server sink and drain the group-commit PUT before `step_completed`.
- If the writer remains locked, do not wait for durability; preserve the existing 500ms inline-loop heuristic and background `waitUntil` lifecycle.
- Drain failures or the 30-second safety timeout fail/retry the step. Client disconnect errors remain non-fatal.
## Test Plan
- Covers released and held locks, release with unsettled writes, delayed first writer acquisition, forwarded writable arguments, drain timeout/failure, and multiple streams.
- `pnpm --filter @workflow/core build`
- `pnpm --filter @workflow/core typecheck`
- `pnpm --filter @workflow/core test` — 2,405 passed, 3 expected failures, 1 skipped
## Summary & Motivation
Adds three bounded trace attributes so a sampled trace says how a `step_started` claim was made: `workflow.step_start.strategy` on the step span (`awaited` / `optimistic` / `batch_preclaimed`, set before the write so a losing claim keeps it after its 409 reconciles to `skipped`), and `workflow.step_start.mode` plus `workflow.step_start.owner_stamped` on the world-vercel write spans across the http, batch, and ws paths. Purely additive telemetry — no change to execution behavior.
## Test Plan
Test added covering a stamped lazy claim's attributes on the per-write span; existing coverage runs in CI. Local run of 81 focused core and world-vercel tests passed; full core typecheck is blocked by unrelated workspace resolution issues.
* [core] Make `hook.metadata` a lazy Promise getter
Hydrating a hook's metadata is a decrypting READ: it needs the owning
run's payload keys, and resolving those costs a run fetch plus a
`run-key` API round trip (~350ms). `getHookByToken()` did that work
eagerly on every lookup that found a metadata-bearing hook, so callers
that only wanted `runId`/`token` — and hook resumption, which never
reads metadata at all — paid for it anyway.
`metadata` is now a getter returning a memoized Promise, the same shape
as `run.returnValue`. The lookup is one read again; hydration and the
key resolution behind it happen on first access, or never.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Pranay Prakash <1797812+pranaygp@users.noreply.github.com>
* docs: surface the lazy hook.metadata change in What's new, the migration skill, and the resumeHook reference
Adds the breaking-change row to the v5 What's new page and puts that page
in the sidebar as the first visible entry (the /v5/docs redirect to
getting-started is unchanged). Teaches the migrating-workflow-v4-to-v5
skill the `await hook.metadata` rewrite and bumps its version. Points the
resumeHook reference at HookWithLazyMetadata, and notes on the World
storage page that world.hooks.getByToken() returns raw serialized
metadata.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* [core] Export the lazy-metadata hook type as `Hook` from `workflow/api`
`getHookByToken()` and `resumeHook()` return `Hook`, not a separate
`HookWithLazyMetadata`: one public hook type whose `metadata` is a lazy
Promise, mirroring `Run` for runs. The World-level record from
`@workflow/world` is unchanged and is referenced as `WorldHook` inside the
runtime.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* [core] Define the lazy `metadata` getter in place; tighten changeset and docs wording
Review feedback: the hook record a World returns is a fresh object per
lookup and the eager path mutated it anyway, so define the getter on it
directly instead of copying it with Object.create(). The changeset is one
sentence, and the docs describe hydration as extra network round trips
rather than decryption, since not every World encrypts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <1797812+pranaygp@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
## Summary & Motivation
Gives one in-memory stream writer a stable identity and its own sequence space, so a transport can preserve chunk ordering across a mid-stream HTTP/WebSocket transition. `Streamer.streams.createWriteSession` is optional — Worlds that don't implement it keep using `write`/`writeMulti`/`close` unchanged.
Abort disposes the session rather than closing it, since a producer failure is transport cleanup, not stream completion.
## Test Plan
Tests added, plus the full `@workflow/world-vercel` suite and package builds/typecheck pass. Root build/typecheck is blocked locally by a missing Rust toolchain for the unrelated `@workflow/swc-plugin`.
## Summary & Motivation
Lets a long-lived run own a shared stream while independent runs append through `run.writable` or `run.getWritable()` using only the owner's run ID. The handle seals to the owner's public key when it has one, so it grants append access without read capability, and carries the existing forwarding symbols so passing it through `start()` and into a step keeps the owner's identity.
## Test Plan
Tests added
## Summary & Motivation
Stream infrastructure failures (HTTP/2 session wedges, transport timeouts, non-2xx stream responses) surfaced as plain `Error`, so terminal classification attributed them to customer code as `USER_ERROR`. They now carry a catchable `StreamError` with a `STREAM_ERROR` run error code, attributed to the SDK and retried when transport-level or 5xx.
The v4 events response body is wrapped so a post-header stream failure is classified and reported to the dispatcher recycler — a response header arriving is not yet a successful streamed request.
## Test Plan
Unit tests added across classification, serialization round-trip, the streamer, and the v4 transport; 331 `@workflow/core` and 123 `@workflow/world-vercel` focused tests pass.
* [core] Settle a hook's awaiter in-process instead of re-invoking, on creation and on conflict
* [core] Address review: deterministic hook signal tests, split changesets, document the boundary
- hook.test.ts: drive the idle poll with explicit macrotask turns instead of a
fixed 20ms sleep (Copilot)
- Split the changeset so each package's entry says only what changed in it
- runtime-tuning docs: hook-only suspensions no longer always park; the hook
write continuation is the one exception
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
## Summary & Motivation
Stream reads now start the run-metadata and encryption-key lookup concurrently with the stream GET, instead of waiting for the first encrypted frame to arrive before touching either, so a reader no longer stalls on that round trip. The key resolver is memoized per readable session and its promise is observed eagerly, so a lookup that fails while nothing consumes it can't surface as an unhandled rejection; the original error still reaches the consumer through the deserialize transform. `getReadable()` builds its underlying stream lazily so an unread stream does no work. The key-resolution phase gets its own `workflow.stream.read.resolve_key` span, keeping `workflow.stream.read`'s time-to-first-chunk measuring the raw stream alone.
## Test Plan
Unit tests added for the concurrent start, the single resolver across reconnects and sessions, and the cancellation/failure paths; `pnpm --filter @workflow/core test` and `typecheck` pass locally.
## Durabench evidence
A completed matched Workflow stream sweep on the final-refactor implementation showed the intended first-chunk improvement. The cell used the canonical Eve cadence, speed 1, in-step reader, five executions per variant, and concurrency 1.
| Metric | Key prefetch | Control |
| --- | ---: | ---: |
| CTT-first p75 | **107 ms** | 167 ms |
| CTT-first p99 | **137 ms** | 189 ms |
| All-chunk CTT p75 | 160 ms | 159 ms |
| All-chunk CTT p99 | 1509 ms | 2260 ms |
| E2E | 54.024 s | 54.046 s |
The feature and control cells both completed successfully. The first-frame improvement does not come with an all-chunk CTT or end-to-end regression in this sample.
Source sweep: `psweep-1788293149845-37d61d53-477c-476c-bd19-a0b7cfa7a2ca`.
- Feature run: `prod-1788378070863-f2435fc6-3aa2-4fd3-9608-71c67d273c9a` at `36a17405`
- Control run: `prod-1788378070866-ecc76dbd-8cfd-4b23-9fd6-386c0fae9007` at `3c087789`
The current head (`d63263ee3`) adds only the reviewed byte-branch completion-state consistency guard; it does not alter the object-stream key-prefetch path measured above.
### Description
Cancelling a reconnecting framed stream could be mistaken for a clean, incomplete EOF. The pending pull then reopened the World stream after cancellation, leaving the new reader unowned; local World consumers retained emitter listeners and polling intervals, eventually producing `MaxListenersExceededWarning` after repeated reads of the same durable stream.
The reader now latches cancellation across pending reads, completion checks, and reconnect acquisition. Reconnect work stops after cancellation, and a World stream that finishes opening after cancellation is immediately cancelled instead of being installed as the active reader.
### How did you test your changes?
Added focused regressions for cancellation while completion metadata is pending and while a reconnect acquisition is pending. The complete `@workflow/core` suite passes: 107 test files passed, 1 skipped; 2,285 tests passed, 3 expected failures, and 1 skipped. `@workflow/core` also typechecks. Before applying the fix, a 20-turn local eve session deterministically retained one World reader per turn and warned on listener 11; with this change, the same run peaked at one reader and ended with zero.
* test(core): reproduce lazy resume disposal race
* Fix durable hook resume race
* Fail closed on unknown hook wakes
* Improve unsupported hook wake diagnostics
* Address durable hook resume review feedback
* Harden producer-committed wake handling
* Serialize durable hook resume: write, then wake
resumeHook() now dispatches strictly serially: the hook_received event
is made durable first, and the workflow wake is published only after
the write is acknowledged. The wake is a plain runId message (the shape
the sequential path always published), so the producer-committed wake
barrier, its queue-message field, and the HOOK_RESUME_INPUT_VERSION
bump are all removed — no consumer or backend coordination is needed,
and either side rolls back independently to today's behavior.
The pre-write ops flush now partitions serialization ops: producer-push
uploads are awaited before the event commits (the payload must not
point at bytes still in flight), while consumer-settled reader ops — a
dehydrated WritableStream, e.g. a manual webhook's responseWritable —
are backgrounded. Awaiting those deadlocked the resume against its own
wake (webhookWorkflow failing across the whole e2e matrix).
Also: wake retries stop on definitive 4xx errors instead of burning the
retry budget; WORKFLOW_DISABLE_LAZY_HOOK_RESUME no longer gates
anything and is ignored; the internal resumeHookDurable alias is
removed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Address review: retry classification, wake dedup, 409 passthrough
- Wake retry classification now actually fires against @vercel/queue:
its errors carry no status field, so classify by the World's
deployment-unavailable hook, then numeric status, then the queue
client's definitive-4xx error names.
- The wake publish carries idempotencyKey `hook-<resumeId>` on the
claim path, so a retried publish whose response was lost dedups
instead of costing a duplicate full replay.
- EntityConflictError (HTTP 409) from the durable write is no longer
re-keyed to HookNotFoundError: every 409 the backend emits on this
write today is transient (slot conflict past the server's retry
budget, claim race) and committed nothing, so it surfaces retryable
instead of presenting as a permanent 404.
- Stamp workflow.hook.resume_committed / wake_published span
attributes after each leg resolves, making stranded resumes
(committed event, no wake) queryable from traces.
- Document on the public resumeHook signature that passing the token
(not a cached Hook) is what makes the write idempotent-on-retry.
- Changeset/changelog: note the ended-run behavior change (late
webhook deliveries to finished runs now 404 instead of 202) and the
409 passthrough.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [core] Make a duplicate attr_set inert instead of terminal
A workflow-body attribute write draws a correlation id that resolves exactly
once: the dispatcher's consumer takes the matching event and deregisters. A
second event under that id therefore has no callback left and never will.
`attr_set` had no entry in ENTITY_EVENT_CLASS_BY_TYPE, so the duplicate skip
could not take it, and `PARKABLE_EVENT_TYPES` does list the type, so it was
parked for a consumer that could never come. Parking is settled by the workflow
function returning, and a survivor there is reported through `strandedEvent` as
a replay divergence. So the run did all of its work, every step succeeded, and
the final replay failed it, deterministically enough to burn the whole
replay-divergence recovery budget and terminate with CORRUPTED_EVENT_LOG.
Give `attr_set` a class so the straggler is skipped like every other one:
committed but inert. Parking still covers the first arrival, for a replay that
walks past an attribute event before the body reaches the call that claims it.
An attribute write from a step body carries no correlation id and is consumed by
the structural lifecycle consumer, so it is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [core] Release a parked duplicate, and agree with the UI about one
The class map alone decides a straggler only where the walk meets it after a
consumption recorded the class. When neither copy has a consumer yet both
park — the walk steps over the first and re-enters in the same tick, with
nothing consumed and so no class recorded — and the drain then claims one and
holds the other for a callback that will never be registered. That survivor is
`strandedEvent`, which is the CORRUPTED_EVENT_LOG this branch set out to stop,
reached by the other road. `dropParkedDuplicates` releases it on the same terms
the walk skips one. Not an `attr_set` property: `wait_completed` parks in pairs
too, and `ONE_SHOT_EVENT_TYPES` only sees the order where the consumption came
first.
Giving `attr_set` a class also moved the observability UI, which reads the same
`entityEventClass` to grey out events a run passed over. It kept treating the
straggler as live, because its terminal-class set had no `attr_set` while the
dispatcher's consumer does deregister on the first event under an id. The two
now share `classifyEntityEvent` and `TERMINAL_EVENT_CLASSES` rather than each
keeping a copy of the rule.
That sharing needs the entity rule to be exact, because a step-written
`attr_set` carries no correlation id: keyed on the run it would collapse every
attribute write a run made into one class, and a captured production log in
`__fixtures__` holds forty. `classifyEntityEvent` gives such an event no class
at all, so neither side can read the second as a repeat of the first.
The shared fixture corpus had nothing for `attr_set`, which is why the drift
between the two halves went unseen. It has four now, and each of them fails on
both sides without the fix above it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
* fix(world-vercel): fail the run on a lost event payload instead of retrying
A frame stream that dies mid-body reaches us as a truncated response, which
is exactly what a dropped socket looks like. So an event whose stored payload
is permanently gone was indistinguishable from a transient blip, and the
runtime kept redelivering a replay that could never succeed: one run re-read
a single missing payload 12,932 times in 26 minutes, and the backend query
behind each attempt throttled its table.
The World now sends a terminal `{_error: 1, code}` frame for failures that a
retry cannot fix. Handle it:
- `payload-missing` raises `CorruptedEventLogError`, so the run fails with
`CORRUPTED_EVENT_LOG` rather than looping. The log does reference a payload
nothing can produce.
- An unknown code raises a `WorkflowWorldError` with no retryable code and no
status, which is also terminal. A future code stays safe without needing a
client release first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Revert the world-vercel URL override to empty
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(world-vercel): classify terminal stream errors
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex Langenfeld <alex.langenfeld@vercel.com>
* fix(world-vercel,world-local): hold process-wide state on globalThis
Both packages are bundled into the host application's server build, and a
bundler keys module identity on (resource, layer) — Next.js alone builds
`instrument`, app-route, `ssr` and `edge` layers, so one process holds one
copy of each of these modules per layer. Every module-scope `const`/`let` in
them was therefore per-copy state wearing the costume of a process singleton.
vercel/workflow#3493 made `@workflow/world-vercel` bundled rather than
external and the events WebSocket transport regressed to HTTP for exactly
this reason: the queue consumer registered its channel in the `instrument`
copy's `Map` and the write path looked it up in the route copy's empty one. A
deterministic miss, for the life of the process. `@workflow/world-local` had
the same exposure all along — including `runFileLocks`, where a duplicated
mutex simply stops mutually excluding.
Add `globalSingleton()` to `@workflow/utils` (the primitive `@workflow/core`
already hand-rolls for its World cache) and route every mutable module-scope
binding in both worlds through it.
Regression cover, in three layers:
- `global-singleton.test.ts` pins the primitive's semantics.
- `ws-transport-module-copies.test.ts` imports the module twice in one
process and asserts a transport registered by one copy is found by the
other — it fails on a plain module-scope `Map`, which is the shipped bug.
- `scripts/lint/module-scope-state.mjs` fails the class: an AST rule banning
mutable module-scope state in these packages, with `// per-copy-ok: <why>`
as the deliberate escape. Wired into both packages' `vitest run src`, with
fixture self-tests so it cannot rot into a no-op.
* test(world-postgres): pin the module-scope-state rule for the postgres world
It is deduped today only because `getRuntimeRequire()` loads it — a property
of how it is loaded, not how it is written, and exactly what changed for
world-vercel in #3493. The package is already clean; this keeps it that way.
* docs(worlds): codify "a world must not hold mutable module state"
A world package is loaded one of two ways, and only one of them gives it a
single module instance: a runtime `require()` (deduped by Node) or the host's
bundler (one copy per layer). Which one you get is a property of how the world
is loaded, not of how it is written, and it changed under `world-vercel` in
#3493 — so the rule has to be "never rely on module scope", not "rely on it
until someone flips a config".
Written down in the four places someone can meet it:
- `docs/content/worlds/{v4,v5}/building-a-world.mdx` — a "Process-wide state"
section for custom-world authors, with the loading modes spelled out and a
nudge to prefer World-instance state over a global.
- `packages/world/README.md` — the same constraint on the contract package.
- `CLAUDE.md` — so the next contributor working in these packages sees it.
- `packages/core/src/runtime/world.ts` — at the two static imports, which is
where the difference between a bundled world and a required one originates.
The rule's own error message now teaches it too, rather than naming a helper.
Consolidates the guard while here: `@workflow/utils` owns the rule and its
fixture self-tests, and sweeps every *published* `packages/world-*` discovered
at runtime, so a world package added later is covered without anyone
remembering. Each world keeps a one-assertion mirror for locality.
* style: drop prose em dashes from this branch's new text
#3704 landed a repo-wide writing pass hours after this branch was written and
took `world-vercel/src` from 406 em dashes to 130 (`ws-transport.ts` alone went
35 to 1). This branch's docs section, README, comments and lint messages were
written before that and would have put 36 of them straight back into the files
that were just cleaned.
Rewritten sentence by sentence rather than by substitution: an em dash becomes a
colon, a comma, a full stop or a parenthetical depending on what it was doing.
Also fixes a real defect the sweep surfaced: `world-postgres`'s guard test was
generated through a shell heredoc and had literal backslash-backticks in its
doc comment.
* Update .changeset/world-module-scope-state.md
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
* fix(core): build the entrypoint's queue handler from getWorld()
Adopted from #3666 by @MintedKenny, which implements #3665 and could not run
CI as a fork PR. One line of behavior: `workflowEntrypoint`'s lazy handler
init calls `getWorld()` rather than `getWorldHandlers()`.
`getWorldHandlers()` owns a second, build-time-safe cache, so calling it from
the runtime route built a *second* World in the same process. That costs a
stateful World duplicate resources on every instance — world-postgres eagerly
constructs a `pg.Pool` (default `max: 10`) and a nested world-local World in
`createWorld()`, so self-hosted users have been paying for two of each — and,
for a bundled world package, the two Worlds are built by two different module
copies, which is the mechanism behind the WS transport regression the rest of
this branch contains.
The public `getWorldHandlers()` and its separate build-time cache are
unchanged; only the runtime route stops using it.
Kept from the original: the regression test asserting the factory runs exactly
once, and the api-reference wording (re-applied over #3704's list punctuation).
Not taken: renaming the `workflow.route.get_world_handlers` span. It is a
distinct span from the per-request `workflow.route.get_world` at the top of the
flow route, and reusing that name would collide with it in traces and in
`runtime-trace-mode.test.ts`; a comment records why the name outlived the call.
Co-authored-by: Kenneth <kenneth@standardforensics.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: address AI review on the module-scope work
Two blocking findings, both real:
- **Cross-version state sharing** (`ws-transport.ts`). A process can hold two
*published versions* of `@workflow/world-vercel` (a transitive dependency
pinning an older `@workflow/core`, which depends on this package by exact
version). Both wrote to the same unversioned `Symbol.for` key, so one
version's write path could be handed a `WsEventsTransport` built by the
other's class and frame against a protocol it may not share — with no version
negotiation on the socket to catch it. `shapeVersion` cannot express this: the
container is stable, the hazard is its contents. The registry and the events
dispatcher recycler are now keyed by package version. The plain connection
pools stay unversioned; sharing those across copies is the point.
- **The documented pattern failed the rule this PR adds.** The custom-world docs
teach `store[StateKey] ??= …`, which the rule flagged as a field write. It now
recognizes state rooted at `globalThis`, following one alias hop, which is
also what `core/private.ts:23` and `next/src/index.ts:58` are already doing
correctly (core drops 26 findings to 22, next 7 to 6). The docs also now say
outright that `globalSingleton()` is the same thing, since AGENTS.md
prescribes it and the page did not mention it.
Rule precision, from the review's probes:
- `.mts`/`.cts` are scanned. `@workflow/world-testing` is authored in `.mts`, so
its entry in the sweep was passing vacuously — with the walk fixed it reports
a real finding, now annotated (it is a standalone `serve()` entry).
- Mutations in top-level statements no longer count. A table filled at module
evaluation is identical in every copy; divergence needs a later write.
- `static` class fields are collected, attributed to the class name.
- An *exported* binding initialized to an empty collection is a finding on its
own, which approximates the cross-file case the walk cannot resolve.
Six fixtures pin the new behavior. The rule's header now states what it does not
see, and AGENTS.md states where the sweep stops and why core is not gated yet.
Also tags `resetGlobalSingletonForTest` `@internal`.
* fix(lint): attribute a static-field write to the field, not the class
The static-field support added in the previous commit keyed `declared` on the
class name, so a class carrying more than one mutable static reported one
finding instead of one per field, and labelled the survivor with whichever
mutation was seen first. On a two-static fixture it reported
`static Registry.latch (`.set()`)`: the name of one field, the reason
belonging to the other, pointing the reader at the wrong line.
Key static fields `Class.field` and resolve a write to the same shape, via a
new `memberPath()` that takes the first two segments of a member chain and
tries that key before the bare root identifier. Two follow-ons fall out of
having the path:
- `this.field` inside a `static` member resolves to the class, which is the
ordinary way to write the mutation. `staticClassOf()` returns nothing for an
instance member, where `this` is an instance and the state is per-instance
rather than per-copy, and nothing inside a nested `function`, which rebinds
`this`.
- `state.count++` is now a finding, like the `state.count += 1` that
`assignment()` already reported.
Fixtures pin all four, including the instance-field case that must stay clean.
The four world packages still report zero, and the extracted `recordMutation()`
keeps the file at its previous two Biome complexity warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: make module duplication inert across every bundled package
`@workflow/core` is bundled into the host server build the same way the worlds
are, and always has been — the original repro measured three live copies in
every arm, including the pre-#3493 external one. One instance is not reachable:
layers cannot share a module, and core cannot be external because it *is*
workflow code (`runtime/start.ts:253` and nine methods in `runtime/run.ts` are
`'use step'`), so it must go through the SWC loader. The Next integration
already encodes that rule by removing workflow-bearing packages from
`serverExternalPackages`.
So the duplication stays and the hazard is removed instead, everywhere the
duplication can happen.
`@workflow/core` (22 findings to 0): warn-once latches in `constants.ts`,
`start.ts` and `telemetry.ts`; the source-map tracer cache; the VM script cache;
the QuickJS compiled-assets and baseline caches; the dev-server port cache (its
own comment already said "per process"); the text codecs; the zstd browser
decoder; and the `useStep` closure brand, where a function marked by one copy
was invisible to another.
The one with teeth was `step-single-flight.ts`: a per-copy map is not
single-flight. Two invocations reaching it through different layers would each
believe they were alone in the process and both run the step body, silently
degrading in-process dedup to the cross-process residual its own doc scopes out
to the ownership lease.
Also `@workflow/world` (a warn-once set, hand-rolled onto `globalThis` to keep
that package dependency-free), `@workflow/ai` (the lazy OTel API), and
`@workflow/nest` (bootstrap config in a module-level `let` and two static class
fields — configure one copy, read another, and the controller is unconfigured
for the life of the process).
Five sites are deliberately per-copy and now say why: state keyed on objects
that never cross copies (the barrier safety-net `WeakSet`, the QuickJS pending
byte `WeakMap`), the synchronously-scoped guest-code sink, and the OTel
diagnostic that reports what *this* copy sees.
The sweep now covers all of it. Packages with a single module graph stay out
(build-time code, the CLI, the o11y UI, the test runner) and AGENTS.md records
which and why.
Found while doing this: two static fields on one class collapsed into a single
entry in the rule, so `WorkflowModule.options` was invisible behind
`WorkflowModule.outDir`. Statics are now keyed `Class.field`.
* fix(world): suppress noAssignInExpressions on the globalThis idiom
The hand-rolled form trips Biome, as it does in `packages/core/src/private.ts`,
which carries the same suppression. Restructuring it into a helper function
instead would hide the state behind a call the module-scope rule cannot follow,
so the binding would stop being recognized as off-module and the package would
report a finding for correct code.
* fix: sweep every bundled package, and mark utils side-effect free
@shalabhc asked on review whether `@workflow/utils` needs this too. It does,
and so do three others: `utils`, `errors`, `serde` and `workflow` all end up in
the host application's server build and none were in the sweep. All four report
zero today, which is exactly the state `world-testing` appeared to be in before
the `.mts` walk was fixed and it turned out to have a real finding. Being clean
and being *checked* are different properties, and only the second one survives
the next contributor.
`sideEffects: false` on `@workflow/utils`: verified that every module in the
package only declares (no import-time work), so a bundler can now drop the
unused parts of the barrel instead of keeping all ~64 KB of it because three
packages import one 476-byte function.
---------
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Co-authored-by: Kenneth <kenneth@standardforensics.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Peter Wielander <peter.wielander@vercel.com>
* Run the test suites CI was silently skipping
`turbo test` runs a package's tests only if that package declares a `test`
script, so a suite can sit in the repo for months without ever running. Four
were in that state: @workflow/world (13 files, 160 tests), @workflow/cli (5 /
51), @workflow/nitro (1 / 30), and two files under packages/core/e2e that no
workflow named.
Wire each one up, and add scripts/check-test-suites-wired.mjs plus a lint job
so the next unwired suite fails CI instead of going unnoticed.
Fixes#3731
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop the changesets and rename the guard job
The PR only wires up existing suites and adds a CI check, so there is nothing
to release. Rename the job to match its `no-test-overrides` sibling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nathan Colosimo