* fix(nitro): support React Router Vite builds
* refactor(nitro): simplify React Router cleanup
* docs(react-router): specify cleanup version
* fix(nitro): close temporary Vite servers
The recoverActiveRuns factory option had no environment variable, so
disabling startup re-enqueueing of pending/running runs required a custom
world module via WORKFLOW_TARGET_WORLD. Wire an env fallback
(0/false disables, 1/true enables, explicit factory option wins) and
document it in the worlds configuration reference and local world guide.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(deps): dedupe @opentelemetry/api to a single workspace instance
The lockfile resolved both 1.9.0 and 1.9.1, so the copy that registers
the tracer provider (via @vercel/otel in the app) and the copy a package
imports could differ. The API's global-registration version check rejects
a consumer newer than the registered copy and silently hands back a noop
tracer — which is why world-vercel's spans (workflow.stream.write/
chunk_rtt, read.connect, its http spans) never reached Datadog from
deployed apps while core's spans flowed in the same process. Root-caused
via the DEBUG=workflow:* run on #2900: import succeeds, no warn, spans
dropped.
Pin a single version via a workspace override so every bundle shares one
API instance.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* debug: one-shot OTEL runtime diagnostic in core + world-vercel; DEBUG on turbopack workbench
The dedupe alone did not restore world-vercel span emission (verified on
this PR's own preview: stream traffic flowed, zero workflow.stream.write
spans). Under DEBUG=workflow:*, both packages now log once how their
module instance of @opentelemetry/api sees the world — global
registration version, provider/delegate/tracer/probe constructor names,
and whether a probe span is recording. Diffing the core line (spans work)
against the world-vercel line (spans dropped) in one deployment's logs
pinpoints the divergence.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* debug: log span identity for named world-vercel spans; namespace otel probes per package
Diag round 1 showed world-vercel's tracer records and instrumentedFetch
handles the stream PUTs, yet the named spans are unfindable in the
backend. Round 2: log traceId/spanId/isRecording for every named
instrumentedFetch span under DEBUG so export can be checked for a
specific span id, and split the probe span names (.core /
.world_vercel) so per-package export is attributable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* telemetry: emit stream RPC latencies from core (chunk_rtt, connect_ms, close span)
world-vercel's instrumentedFetch spans never export from deployed apps
(root cause still open — see PR discussion), so the operationally
needed client-side latency signals move one layer up to core, whose
spans are proven to export:
- workflow.stream.write.chunk_rtt on the workflow.stream.flush span:
the World write RPC duration, network included (same attribute key as
world-vercel's per-request span so queries are layer-agnostic).
- workflow.stream.read.connect_ms on the workflow.stream.read span:
the world.streams.get await (read dispatch -> stream handle).
- new workflow.stream.close span: the close RPC round trip.
Bonus: measured at the World interface, these cover world-local and
world-postgres too, not just Vercel deployments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* telemetry: emit read-completion span (total duration, chunks, bytes)
Completes the read-side picture: workflow.stream.read.complete is
back-dated to the read dispatch so its duration is the total read, with
chunk/byte counts for throughput. Cancelled reads emit nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: drop DEBUG from turbopack workbench; tighten changeset
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* telemetry: cover createReconnectingFramedStream in read telemetry
Ordinary serialized streams read through createReconnectingFramedStream
(which calls world.streams.get directly), so connect_ms / ttfc /
read.complete never fired for that path — only WorkflowServerReadableStream
was instrumented. Wire the same helpers into the framed reader: first-
connect duration, first-frame TTFC, and completion totals — plus
workflow.stream.read.reconnects, which only this path can know.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
- analytics: attribute the serving path to the Vercel observability data
pipeline instead of naming the ClickHouse store, drop the Vercel-only
"Plan-bounded lookback" bullet, and mark the pageInfo lookback ceiling
and upgradeAvailable comments as Vercel-specific.
- sidebar: folder rows without an index page render as <button>s, which
don't stretch to the row width like the <a> folder links, leaving their
ms-auto chevron hugging the label. Stretch those triggers to full width
so every chevron sits at the end of the row.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Audited every vercel.com link in docs/content (20 unique URLs,
HTTP-validated including anchor fragments):
- project-configuration#regions (2x): the #regions anchor no longer
exists on that page — content moved to the vercel-json subpage; now
links project-configuration/vercel-json#regions
- gateway/api-reference/overview (2x): hard 404; the AI Gateway docs
restructured — 'get an API key' context now points at
ai-gateway/authentication
- observability/otel-overview (1x): redirects to
tracing/instrumentation; link the final URL
- docs/workflow and docs/workflow/python (8x): redirect to the plural
docs/workflows paths; link the final URLs (also drops a redundant
?language=py param that the redirect discards)
All other links (queues, queues/pricing + anchors, plans/hobby,
limits, regions, sandbox, workflows/pricing#storage-retention,
cli/project-linking, audit-log, home, help, blog) verified 200 with
live anchors.
* Add attribute discovery and filtering to world.analytics
- analytics.attributes.list() — distinct attribute keys observed on runs
in the window, with run counts and first/last seen timestamps
(GET /v2/analytics/attributes).
- analytics.attributes.listValues({ key }) — distinct values for one key
with latest-write-wins run counts (GET /v2/analytics/attributes/values).
- analytics.runs.list({ attributes: { key: value } }) — restrict the runs
listing to runs whose latest attribute snapshot matches every provided
pair (JSON-encoded query param, up to 8 pairs; $-prefixed framework
keys allowed in read filters).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Shorten changeset; document world.analytics in the World SDK reference
The analytics namespace was previously undocumented. Adds a full
reference page (runs, attributes, steps/events/hooks/waits, lookback
windows and pageInfo), links it from the World SDK index and meta, and
replaces the stale 'in the future you'll be able to search by
attributes' line in the attributes guide with a filtering section.
Extends the docs-typecheck ambient world global with the analytics
namespace so reference snippets typecheck.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop analytics.attributes.listValues
Not a derived requirement: the agent-runs UI filters by known constant
values and reads per-run values via batch attribute fetches; it never
enumerates distinct values for a key.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Feature-detect world.analytics in the attributes guide example
The snippet dereferenced the optional analytics namespace without the
runtime check its skip-typecheck annotation claimed, and would throw on
local/Postgres/custom Worlds. Guard it and drop the annotation — the
block is now genuinely typechecked.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix stale 'querying attributes not available' bullet in the guide
The Behavior list still claimed a query API was only planned, directly
contradicting the Searching and filtering section above it. State what
is implemented: attributes are readable on run objects everywhere, and
key discovery / key=value run filtering are available through the
optional Analytics API.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Surface the analytics namespace on the API reference index pages
The Analytics page sits under workflow/runtime > World SDK, but neither
the API reference index card, the workflow/runtime World SDK card, nor
the World SDK overview mentioned analytics — making the new reference
effectively undiscoverable from /docs/api-reference. Mention it at each
level of the path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Bump changeset to minor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* docs: document multi-region support in the Vercel World
The Vercel World Limitations section still described the backend as
iad1-only, which is no longer true — it now runs in every Vercel
Function region, with each run pinned to a single region at creation.
- Replace the single-region / iad1 data-residency limitations with a
Multi-region section: automatic pinning from the creating function's
region (single- and multi-region deployments), explicit per-run
selection via start(..., { region }), routing semantics for readers,
and 4.x/pre-existing-run behavior (iad1, no migration).
- Version requirement called out as workflow 5.x beta with a TODO to
pin the exact minimum version once released; 4.x will not support
region pinning.
- Limitations now lists the one that remains: a run's region is fixed
at creation (no migration).
- /docs/deploying: add a Multi-region bullet to the Vercel World
feature list, deferring details to /worlds/vercel.
* docs: clarify region option scope — data + queue dispatch, not code placement
The start({ region }) option pins where the run's data is stored and
where its queue messages dispatch from; it does not deploy code.
Execution happens in the regions the application is deployed to, so
region-local execution requires deploying the app to the desired
region (vercel.json regions or the project's Function Regions
setting). Adds a warning callout to the explicit-selection section and
tightens the intro to only claim region-local execution for the
automatic case.
* docs(v4): point default-version pages at the v5 multi-region docs
The v4 docs are the default version, so readers landing on
/worlds/vercel (which renders v4 content) or /docs/deploying would
never discover multi-region exists behind the v5 switcher.
- v4 Vercel World Limitations: add a callout that multi-region ships
with workflow 5.x (deep link to the v5 Multi-region section), and
scope the iad1-only statements to the 4.x release line — they remain
true there; 4.x will not gain multi-region.
- v4 /docs/deploying: matching callout after the Vercel World blurb.
* docs(v5): surface the world pages in the Deploying sidebar
The v5 world pages (vercel-world, postgres-world, local-world) were
orphaned: the Deploying sidebar only listed 'Building a World', and
the dedicated /worlds/:id route renders v4 content — so there was no
navigation path to the v5 Vercel World page (and its new Multi-region
section) at all.
- v5/deploying/meta.json: add the world folder to the sidebar
(Worlds group between the section index and Building a World)
- world/meta.json: group title 'World' -> 'Worlds'
- v5 /docs/deploying callout: link to the v5 world page (with a
multi-region deep link) instead of /worlds/vercel, which silently
drops the reader into v4-rendered content
Verified via dev server: the Worlds group (all three pages) renders in
the v5 sidebar and the Multi-region section renders on the page.
Note for docs owners: other v5 content still links to the versionless
/worlds/* routes, which render v4 content — probably worth a broader
pass or making /worlds version-aware.
* docs(v4): surface the world pages in the Deploying sidebar
Same fix as v5: the v4 Deploying sidebar only listed 'Building a
World', leaving the world pages reachable only via the /worlds/:id
routes with no sidebar path. Adds the Worlds group (Vercel World,
Postgres World, Local World) between the section index and Building a
World, and pluralizes the group title.
Verified via dev server: the group renders, and the sidebar items land
on /worlds/:id through the existing permanent redirects (same v4
content).
* docs: pin multi-region minimum version to workflow 5.0.0-beta.33
vercel/workflow#1981 merged; the first release carrying multi-region
support is 5.0.0-beta.33. Replace the '5.x (currently in beta)'
placeholders (and the TODO marker) with the exact minimum version on
all four touched pages (v5 world page + deploying bullet, v4
breadcrumb callouts).
* docs: note hook-token data residency in the multi-region section
Hook tokens carry no region information, so the token-to-run mapping
behind getHookByToken()/resumeHook() is currently stored in iad1 for
every run regardless of its region. Hook payloads are unaffected —
a received payload lands on the run's event log in the run's region
like all other run data. Noted as potentially becoming a
project-level setting in the future.
* docs: address review — execution-locality wording + dedupe limitation
- /docs/deploying multi-region bullet: 'execution' -> 'queuing'; the
region option pins data/queue/streams, while execution follows the
app's deployed regions (matching the v4 callout and the world page's
own warning callout)
- v5 world page: drop 'region is fixed for its entire lifetime' from
the Good to know bullet — the Limitations section already owns that
statement; the bullet now covers only routing semantics
* telemetry: emit client-observed workflow.stream.write span per flush batch
Complements the existing workflow.stream.read TTFC span: each flushed
batch emits a back-dated CLIENT span covering the app-perceived write
latency (buffer dwell + RPC), with buffer_dwell_ms / chunks / bytes
attributes so client-side batching cost (flush timer, turbo run-ready
barrier) can be told apart from network/server time. Failed flushes
keep the batch's original t0 so a retried batch reports its full dwell.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: tighten changeset
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* telemetry: rename flush span to workflow.stream.flush; DEBUG-log world-vercel OTEL load failure
- workflow.stream.write is taken by world-vercel's per-request RPC span
(#2857, chunk_rtt); the per-batch flush span gets its own name so the
two stay distinguishable in trace queries. Attributes move to
workflow.stream.flush.{buffer_dwell_ms,chunks,bytes}, operation=flush.
- world-vercel's @opentelemetry/api load failure was silently latched as
null, which also swallows bundler/resolution failures in apps that DO
register a tracer (observed in production: workbench apps emit core
spans but none of world-vercel's). Log the reason under
DEBUG=workflow:* so the failure mode is diagnosable.
- Document workflow.stream.flush in the tracing docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: tighten changeset
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(world-vercel): name stream client spans + add stream attributes
Stream write/read requests already share the instrumented HTTP envelope
(a CLIENT span + W3C trace-context injection), but the spans were named
for the bare HTTP verb (`http PUT`/`http GET`) and carried only generic
HTTP attributes — so stream latency couldn't be sliced per run/stream.
Name these spans for their operation (`workflow.stream.write` /
`workflow.stream.read`) and tag them with `workflow.run.id`,
`workflow.stream.name`, `workflow.stream.operation`
(write | write_multi | close | read), and `workflow.stream.start_index`
(read). Implemented via new optional `spanName`/`attributes` fields on
`instrumentedFetch`, so other callers are unaffected.
Additive OTEL only: no behavior change when no OpenTelemetry SDK is
registered (the span is undefined and attributes are dropped).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Shorten changeset summary
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add client-observed end-to-end read TTFC span
The read GET can't report a client-measured latency back to the server (the
value only exists after the response starts streaming), so capture it purely
in the SDK's own OTEL: watch response.body for the first non-empty chunk and
emit a workflow.stream.read span back-dated to read dispatch, whose duration is
the end-to-end time-to-first-chunk (incl. the network hop) via
workflow.stream.read.ttfc_ms. Rename the fetch/connect span to
workflow.stream.read.connect. No-op without an OTEL SDK registered.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add client-observed e2e write latency attribute
The write PUT is request/response and the server acks only after capturing the
chunk, so the workflow.stream.write span duration already equals the
client->server write latency. Expose it as a named attribute
workflow.stream.write.e2e_ms (via a durationAttribute option on
instrumentedFetch) for direct querying, parallel to the read ttfc_ms.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: document stream spans and latency attributes
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Emit read TTFC span from the core reader instead of a world-vercel transform
Per review: move the client-observed time-to-first-chunk measurement out of a
TransformStream wrapper in world-vercel and into WorkflowServerReadableStream in
core, emitting workflow.stream.read on the first non-empty chunk reaching the
consumer. Removes the passthrough, measures at the reader abstraction, and is
backend-agnostic. world-vercel keeps the workflow.stream.read.connect HTTP span;
the recordElapsedSpan helper now lives in @workflow/core.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Apply suggestion from @VaguelySerious
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
* Update docs/content/docs/v5/observability/tracing.mdx
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Signed-off-by: Karthik Kalyan <105607645+karthikscale3@users.noreply.github.com>
* Address review: rename write.e2e_ms -> write.chunk_rtt; docs + changeset wording
Per review, rename the write attribute to workflow.stream.write.chunk_rtt (it's
a per-chunk client<->server round-trip, not a full e2e), update the docs row
wording for both write and read attributes, and shorten the changeset.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
Signed-off-by: Karthik Kalyan <105607645+karthikscale3@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
- Add deprecation banners (with migration-guide link) to the DurableAgent and
WorkflowChatTransport API references in v4 and v5; keep the full API surface intact
- Bring v4 headline guides to parity with v5's WorkflowAgent migration (ai/index,
foundations/streaming, the cookbook recipe + index)
- Convert standard agent examples (defining-tools, message-queueing) to WorkflowAgent
and reframe the streamText-vs comparison page
- Banner + repoint the deep recipes that stream custom UIMessageChunk data parts
(chat-session-modeling, human-in-the-loop, agent-cancellation, serializable-steps) —
that pattern doesn't map to WorkflowAgent's ModelCallStreamPart model, so their
legacy DurableAgent examples are kept behind a clear deprecation banner
- Point all WorkflowChatTransport examples at the @ai-sdk/workflow 1:1 port
- Rename the cookbook agent-patterns recipe to WorkflowAgent
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Swap the text-based eve placeholder for the real eve wordmark (hard-copied
SVG from @vercel/geistcn-assets, themed via currentColor) and drop Streamdown
so AI Elements is last.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
Section index card grids (e.g. foundations) were hand-written and drifted
from the sidebar (meta.json) and the actual pages. Make them derive from
the fumadocs page tree (single source of truth) and add CI lint so the
card grid and navigation can't fall out of sync again.
- resolveSectionChildren + <AutoCards/>, bound in both v4 and v5 docs
routes (correct /docs vs /v5/docs URL spaces)
- getLLMText expands <AutoCards/> so llms.txt/.md/copy-page keep child links
- manualCards frontmatter opt-out for curated pages (source.config.ts)
- checkSectionCards (card<->nav completeness) + checkMetaEntriesResolve
(dangling meta entries) in scripts/lint.ts
- convert foundations + errors (drift fixes) and v5 observability to AutoCards
- mark deploying + ai as manualCards (intentionally curated)
- remove dangling meta entries: v4 cancellation (x2), root introduction
(x2), v4/internal serializable-abort-controller
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
On the first delivery of a run's first invocation, background run_started,
skip the initial event-log load, and force optimistic inline start so the run
reaches its first steps with no preceding network round-trips. Safe because the
first delivery has no concurrent handler to race the step create-claim; turbo
exits the moment a suspension creates a hook or wait, and is a no-op for every
other invocation. On by default; disable with WORKFLOW_TURBO=0.
Wire the existing AI SDK logo into the OSS product switcher (above Flags
SDK) and add a new eve entry (text wordmark + Beta badge, linking to
eve.dev/docs) above it at the top of the list.
Co-authored-by: Ben Sabic <bensabic@users.noreply.github.com>
* Default source maps to dev-on / prod-off
Inline source maps are embedded in the step bundle and the intermediate
workflow VM bundle, which bloats production function bundles (a problem for
the Vercel 250MB limit) even though maps only help when reading a stack trace.
Make the default environment-aware in @workflow/builders: inline in
development (next dev / nitro dev / Vite-based dev servers, detected via
config.watch or NODE_ENV=development) and off in production. The `sourcemap`
config option and `WORKFLOW_SOURCEMAP` env var still override in either
environment. A production build with no override also drops the
source-map-support shim from the Vercel step function.
Keep runtime stack remapping graceful and fast when maps are absent
(@workflow/core): short-circuit when no frame references the workflow file
and memoize the parsed map (or its absence) per bundle, so production failures
don't rescan the bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(e2e): make source-map expectations match dev-on/prod-off default
The e2e error-stack tests gate source-map assertions on hasWorkflowSourceMaps()
and hasStepSourceMaps(). Now that source maps default to off in production
builds, update those helpers:
- hasWorkflowSourceMaps(): false for all production builds (local prod,
postgres, Vercel — keyed off DEV_TEST_CONFIG), and exclude nest in dev (the
Nest integration builds with watch:false / no NODE_ENV=development, so its
bundles have no maps).
- hasStepSourceMaps(): nest now resolves to a production build (maps off) in
both dev and prod.
Add unit cases for the dev-vs-prod and nest behavior.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add a `--url` flag to `inspect`/`web` that prints a run's observability
dashboard deep link to stdout and exits — no browser, no local server —
so scripts and agents can share a link instead of opening a UI.
Fix the Vercel dashboard URL to the current
`…/workflows/runs/<id>?environment=<env>` route (drop the legacy
`/observability` segment) and respect `--env`. Apply the same route fix
to the e2e helpers, CI aggregation scripts, and the nextjs-turbopack
workbench. Document deep-linking in the workflow skill and observability
docs.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(core,world): gzip-compress serialized payloads behind specVersion 5
Add a composable 'gzip' format prefix layer to the serialization
pipeline (compress before encrypt: encr(gzip(devl))), cutting stored
payload bytes by ~70-87% on real-world-style workloads. Compression is
gated on run specVersion 5 (new SPEC_VERSION_SUPPORTS_COMPRESSION) and
on target-deployment capabilities for cross-deployment writes; payloads
under 1KB or that don't compress meaningfully are stored unchanged.
Reads dispatch on the format prefix so both compressed and uncompressed
data are always readable. WORKFLOW_DISABLE_COMPRESSION=1 disables
writes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(core): add CPU/perf compression benchmark + shared workloads
Split the compression benchmark into reproducible size and CPU scripts
sharing deterministic workloads (lib/workloads.mjs). The CPU benchmark
measures serialize/deserialize overhead per payload, total CPU across
thousands of events, and compares gzip levels/brotli/deflate. Documents
how to run the size, CPU, and end-to-end (bench.bench.ts) benchmarks
against local and Vercel in scripts/README.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(world-vercel): advertise specVersion 5 to enable compression on Vercel
Now that workflow-server declares spec-5 support (vercel/workflow-server#520),
bump the Vercel world's advertised specVersion from 4 to 5 so new Vercel runs
are stamped spec 5 and become eligible for gzip payload compression. Payloads
stay opaque to the server (compression is client-side); spec 5 is a superset of
spec 4, so initial run attributes still work.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(core): emit OTel span attributes for compression impact
Track gzip payload compression on both the serialize (write) and
deserialize (read) paths via span attributes:
workflow.serialization.{operation,compressed,uncompressed_bytes,
stored_bytes,compression_ratio}. Sizes are measured at the compression
boundary (pre-encryption), so they reflect compression's effect rather
than the at-rest size.
The compression codec stays pure — compress/decompress optionally
populate a CompressionStats sink, threaded through CodecOptions to the
mode serializers and read by the dehydrate/hydrate wrappers, which set
attributes on the active span. Telemetry failures are swallowed so they
can never break the serialize/deserialize data path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(core,web-shared): prefer zstd compression codec (gzip fallback)
Switch the payload compression codec to zstd, which benchmarks 3–7×
faster than gzip at an equal-or-better ratio on representative workloads
(compression runs at every step boundary, so the write CPU is a per-step
tax). zstd uses node:zlib (>= 22.15); gzip via the portable
CompressionStream remains the fallback when zstd is unavailable, and
WORKFLOW_COMPRESSION_CODEC=gzip forces it. Reads dispatch on the format
prefix, so 'zstd' and 'gzip' payloads are both always decodable.
zstd is Node-only (Web CompressionStream has no zstd), so the browser
o11y read path registers a WASM-backed decoder (@tootallnate/zstd-wasm)
via a new registerZstdDecoder hook; node:zlib handles Node-side reads
(runtime replay, CLI, server o11y). A new workflow.serialization.codec
span attribute reports which codec applied. gzip and zstd read support
co-ship, so the existing specVersion-5 capability gate is unchanged.
Verified end-to-end: spec-5 runs store zstd-prefixed payloads on disk
and replay/complete correctly; the WASM decoder round-trips node:zlib
zstd output. Benchmarks updated to compare zstd vs gzip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(core): make deploymentId 'latest' a no-op in non-Vercel worlds
Previously, start({ deploymentId: 'latest' }) threw a WorkflowRuntimeError
in any World that doesn't implement resolveLatestDeploymentId() (local dev,
Postgres). That meant a workflow which opts into 'latest' on Vercel would
fail outright in local development.
Resolving 'latest' only means something in worlds with atomic, immutable
deployments. In other worlds there is nothing to resolve between, so instead
of throwing we now log a warning and fall back to the current deployment,
making 'latest' an effective no-op there.
- start.ts: warn + fall back to currentDeploymentId instead of throwing
- start.test.ts: replace the "should throw" test with a warn + fallback test
- e2e.test.ts: assert 'latest' completes (no-op) on non-Vercel worlds
- docs: note the no-op behavior in v4 + v5 start.mdx
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(core): warn once for deploymentId 'latest' no-op; harden test cleanup
Address PR review:
- Gate the 'latest'-has-no-effect warning behind a once-per-process guard
(mirrors the warnOnce pattern in constants.ts) so a workflow that hardcodes
'latest' for Vercel doesn't flood local/Postgres dev logs on every run.
Exposes _resetLatestNoOpWarnForTests() (@internal) for unit tests.
- start.test.ts: reset the guard in beforeEach and restore spies in afterEach
via vi.restoreAllMocks() so a throwing assertion can't leak the
runtimeLogger.warn spy into later tests; drop the manual mockRestore().
- Add a test asserting the warning fires exactly once across repeated
'latest' starts while every run still falls back to the current deployment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* otel: explicit traceparent injection + linked-trace mode for bounded per-invocation traces
- Add WORKFLOW_TRACE_MODE ('linked' default, 'continuous' legacy) to the
workflow and step queue handlers. In linked mode, WORKFLOW_V2/STEP spans
start a new trace root with span links to the incoming delivery context
and the run-origin context, and re-enqueued messages forward the
ORIGINAL run-origin trace carrier unchanged.
- world-vercel now explicitly injects W3C traceparent/tracestate/baggage
headers on outgoing workflow-server HTTP requests from inside the
client span (no-op without an OTEL SDK registered).
- New workflow.trace.mode span attribute; unit tests for both modes and
for header injection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* changeset: call out behavioral telemetry changes of the linked default
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: add v5 observability tracing page
Documents OTEL spans/attributes, linked trace mode and WORKFLOW_TRACE_MODE,
span links, context propagation, and the v4 behavior-change callout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* otel: human-friendly span names for workflow and step spans
WORKFLOW_V2/STEP prefixes with full machine names (workflow//./src/...//fn)
become workflow.execute / step.execute / workflow.start with the short
function name. New workflowDisplayName/stepDisplayName helpers in
@workflow/utils handle both raw and queue-sanitized name forms; full names
remain in the workflow.name/step.name attributes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* changeset: merge span-name and linked-trace notes into one changeset
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: update trace-shape prose to renamed span names
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: replace ascii trace diagram with mermaid
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* address review: empty carriers, shared trace helpers, mode warning, name edge cases, consumer span kind
- Treat an empty ({}) trace carrier as absent everywhere the trace-mode
logic branches, so linked mode falls back to a fresh origin instead of
forwarding a useless {} forever; workflow.trace.propagated now reports
whether a usable carrier arrived.
- Extract the duplicated linked-mode logic into shared telemetry helpers
getNextTraceCarrier() and buildInvocationSpanLinks(), used by both the
workflow and step queue handlers; resume-hook now uses
linkToTraceCarrier (gaining the isSpanContextValid guard).
- Warn once per distinct unrecognized WORKFLOW_TRACE_MODE value instead
of silently selecting linked.
- shortNameFromSanitized: map default/__default to the module short name
(mirroring parseName) and document the `$`-sanitization limitation.
- Queue-delivered workflow.execute spans now use the CONSUMER span kind,
matching queue-delivered step.execute spans; docs span table and
changeset updated accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document run idempotency
* docs: address idempotency review feedback
* docs: make hook tokens the idempotency pattern
* docs: address toolbar idempotency feedback
* docs: clarify idempotency page description
* docs: scope idempotency descriptions
* docs: move step idempotency example under section
* docs: simplify idempotency guidance
* docs: simplify idempotency cookbook
* docs: add empty changeset
Signed-off-by: Nathan Rajlich <n@n8.io>
* docs: address idempotency review feedback
* feat: add hook ready promise
* docs: mention conflicting hook run id
* test: cover hook ready continuation scheduling
* feat: replace hook.ready with hook.hasConflict (Promise<boolean>)
- hook.hasConflict resolves true when the token is owned by another
active hook, false once registration is committed — no throw, so
workflows can branch on conflicts early. Awaiting it suspends the
workflow to commit the hook registration (createHook alone does not).
- Chain the already-created fast-path through promiseQueue so
resolution order matches event-log order (review feedback).
- Skip inline step execution when a suspension has an awaited hook
creation so the hasConflict continuation can advance independently
of step execution (review feedback).
- Update unit tests, e2e tests, workbench workflows, and v4/v5 docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: fix inconsistent hasConflict bullet in create-webhook reference
State both resolution values explicitly (true = token already owned,
false = registered) instead of a parenthetical that only described the
false case.
* docs: require docs preview links in PR descriptions for docs changes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: restore SWC Plugin heading in AGENTS.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: adopt hook.hasConflict in run idempotency docs
- Primary claim pattern is now `if (await hook.hasConflict)` instead of
try/catch on HookConflictError; payload awaits still reject with
HookConflictError (with conflictingRunId) when the owner's run ID is
needed.
- Route example returns the active owner via resumeHook()'s runId
instead of threading conflictingRunId through the workflow result.
- Update claim-pattern prose across start(), getHookByToken(), world
storage, scheduling, workflow composition, and cookbook idempotency
pages (v4 + v5).
- Add @skip-typecheck marker to the cross-block route sample, fixing a
pre-existing docs typecheck failure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: move resume-or-start guidance into a dedicated resumeHook example
The early callout was too vague and out of place at the top of the API
reference. Replace it with a 'Resume or Start' example section that
explains the flow, shows the resume-first/start-then-retry route, and
links to the run idempotency pattern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: detect the concurrent-start race via runId comparison instead of awaiting returnValue
The 'Resume or Start' example returned the just-started run's runId with
reused: false even when a concurrent request's run won the token race —
the payload had reached the actual owner, so the response pointed callers
at a run that exits as a duplicate. The foundations route handled the
race correctly but by awaiting run.returnValue, blocking the HTTP
response on full workflow completion.
resumeHook() always resolves against the actual active owner, so
comparing the resumed hook's runId with the started run's runId detects
the race in both examples — race-correct and non-blocking.
* feat: replace hook.hasConflict with hook.getConflict (Promise<Run | null>)
hasConflict's boolean didn't expose WHICH run owns the token, so the
duplicate run couldn't act on the conflict. getConflict resolves with
null once registration commits, or with a Run handle for the conflicting
run — letting the workflow return/log the owner's runId, inspect its
status, await its result, or cancel it and continue, all in code.
The workflow-mode create-hook module exposes the bundle's compiled Run
class (durable step-proxy methods) on a well-known symbol so the host-
side hook consumer can construct the conflicting run inside the VM.
Contexts without the class (plain unit tests) fall back to a { runId }
object, which is also the documented v4 shape (no native Run
serialization in v4).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: adopt hook.getConflict and add conflict-handling strategy guide
Run idempotency docs now use getConflict (resolves with the conflicting
Run in v5, { runId } in v4) and document code-driven conflict strategies
in place of static ID-reuse policies: reject the duplicate, adopt the
owner's result, inspect before deciding, signal the owner via
resumeHook, and supersede via cancel-and-reclaim.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: never resolve getConflict with a non-Run fallback shape
getConflict's contract is Promise<Run | null>. In the degenerate cases
where a real Run cannot be constructed — a hook_conflict event persisted
by an old world without conflictingRunId, or a context that never loaded
the workflow-mode create-hook module — reject with HookConflictError
instead of resolving with a { runId }-shaped impostor.
Test harnesses now register the Run class on the (VM) globalThis like
real bundles do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: make getConflict a method — hook.getConflict()
A property getter that triggers registration/suspension reads as passive
state; a method makes the side effect explicit. Update implementation,
types, tests, e2e workflows, docs, and changeset.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: getConflict is a method — hook.getConflict()
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: typecheck every sample — drop skip-typecheck escape hatches
Route examples typecheck as-is since the runId-comparison rewrite;
strategy fragments are now complete self-contained workflows; the
publishing-libraries cross-block dependency uses the declare @setup
convention. 934 samples typechecked, none skipped by this PR.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: guard Run class registration, fix anchors, clarify changeset
- Only register WORKFLOW_RUN_CLASS when the workflow runtime is present
(WORKFLOW_CREATE_HOOK installed on globalThis), so host imports of the
workflow-mode module neither mutate the host global nor expose the
non-step-proxy host Run.
- Drop #run-idempotency link fragments — that section lands in the
stacked docs PR (#2011), which restores the anchored links.
- Note in docs that getConflict() rejects with HookConflictError for
legacy hook_conflict events lacking the owner's run ID.
- Changeset now calls out the hasConflict -> getConflict() replacement.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: restore run-idempotency anchors now that the section exists here
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: describe fixed conflict policies generically, without naming other systems
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Signed-off-by: Nathan Rajlich <n@n8.io>
Co-authored-by: Nathan Rajlich <n@n8.io>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
- Teal diamond markers for attr_set events on the trace timeline with
time tooltips (new trace viewer)
- attr_set payloads render changed/removed keys and the writer
(workflow vs step + attempt) in the run sidebar and Events tab
- Run root span selection now shows run-level events (run lifecycle +
attr_set) in the sidebar
- Attributes card on run details renders key-value rows with reserved
$-prefixed keys badged and sorted after user keys
- attr_set added to MARKER_EVENT_TYPES, BOUNDARY_LABELS, event colors
(teal), and the flat events list run-level grouping
- Docs: screenshots on the attributes page, served from docs/public
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(docs): repair broken links and make the docs link linter actually validate
The docs link linter (docs/scripts/lint.ts) had been silently passing
everything since the app moved under app/[lang]/ (#552): the
next-validate-link populate key 'docs/[[...slug]]' no longer matched the
real route, and the unpopulated [lang] homepage route produced a fallback
regex (^\/(.+)$) that matched every href. It also only scanned v4 content.
- Rewrite lint.ts to build explicit v4/v5 URL spaces from both fumadocs
sources (including cookbook URL variants, app routes, worlds pages,
public/ assets, and next.config.ts redirects) and validate each version's
content against version-correct render semantics. Also validate
frontmatter related/prerequisites references (version-relative) and
heading fragments.
- Rewrite Card hrefs on v5 pages: the v5 routes rewrote inline markdown
links from /docs/... to /v5/docs/... but Card renders its own Link, so
Card hrefs escaped to the v4 routes and 404'd for v5-only pages (e.g.
/v5/docs/observability linking to /docs/observability/attributes).
- Fix all dead content links surfaced by the working linter (56 across
v4+v5): nonexistent use-workflow/use-step/start API pages now point at
foundations/workflows-and-steps and workflow-api/start, getStepMetadata
path corrected, /docs/worlds/local → /worlds/local, dead changelog/
internal references removed or unlinked, retired common-patterns links
point at the cookbook, and a dead #returnvalue anchor now targets
#returns.
- Add an index page for api-reference/workflow-errors (both versions),
which was linked from the API reference landing page but had no page.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(docs): add version prefix to 'Edit this page on GitHub' links
All "Edit this page on GitHub" links 404'd since the v4/v5 content split
(#1948): page.path is relative to the per-version content dir, but
EditSource built URLs against docs/content/docs/ without the v4/ or v5/
segment. Add a required version prop, passed from each page route.
Incorporates #2120 by Luke Howard (@gldkhoward), rebased onto the v5
route changes from this branch. Fixes#2119.
Co-authored-by: Luke Howard <dev@lukehoward.com.au>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add allowReservedAttributes option to start()
experimental_setAttributes already exposes allowReservedAttributes for
framework-level callers that own a $-prefixed sub-namespace, and the
run_created / run_started event schemas plus the local and Postgres
worlds already accept and validate the flag. start() was the one gap:
it always validated initial attributes with the reserved prefix
disallowed and had no way to opt out, so framework code could not seed
reserved attributes at run creation.
Thread the option through start():
- StartOptions.allowReservedAttributes, passed to client-side
validation and forwarded on the run_created eventData
- carried in the queue runInput (new RunInputSchema field) and
forwarded to run_started so the resilient/lazy run creation path
validates identically
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add e2e coverage for reserved initial attributes via allowReservedAttributes
Verified locally against the nextjs-turbopack dev server: the reserved
key passes client and server validation, lands on the run at creation,
and survives the workflow's own attr_set writes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
SidebarFolderTrigger renders a <button>, which shrink-to-fits its
content, so the ms-auto chevron sat directly next to the folder name
for folders without an index link (e.g. How it works, AI Agents,
Testing). SidebarFolderLink renders an <a> that spans the full sidebar
width, so its chevron was pushed to the right edge. Add w-full to both
so every folder caret is right-aligned.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat: replace hook.hasConflict with hook.getConflict (Promise<Run | null>)
hasConflict's boolean didn't expose WHICH run owns the token, so the
duplicate run couldn't act on the conflict. getConflict resolves with
null once registration commits, or with a Run handle for the conflicting
run — letting the workflow return/log the owner's runId, inspect its
status, await its result, or cancel it and continue, all in code.
The workflow-mode create-hook module exposes the bundle's compiled Run
class (durable step-proxy methods) on a well-known symbol so the host-
side hook consumer can construct the conflicting run inside the VM.
Contexts without the class (plain unit tests) fall back to a { runId }
object, which is also the documented v4 shape (no native Run
serialization in v4).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: never resolve getConflict with a non-Run fallback shape
getConflict's contract is Promise<Run | null>. In the degenerate cases
where a real Run cannot be constructed — a hook_conflict event persisted
by an old world without conflictingRunId, or a context that never loaded
the workflow-mode create-hook module — reject with HookConflictError
instead of resolving with a { runId }-shaped impostor.
Test harnesses now register the Run class on the (VM) globalThis like
real bundles do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: make getConflict a method — hook.getConflict()
A property getter that triggers registration/suspension reads as passive
state; a method makes the side effect explicit. Update implementation,
types, tests, e2e workflows, docs, and changeset.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: guard Run class registration, fix anchors, clarify changeset
- Only register WORKFLOW_RUN_CLASS when the workflow runtime is present
(WORKFLOW_CREATE_HOOK installed on globalThis), so host imports of the
workflow-mode module neither mutate the host global nor expose the
non-step-proxy host Run.
- Drop #run-idempotency link fragments — that section lands in the
stacked docs PR (#2011), which restores the anchored links.
- Note in docs that getConflict() rejects with HookConflictError for
legacy hook_conflict events lacking the owner's run ID.
- Changeset now calls out the hasConflict -> getConflict() replacement.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: resolve the conflicting Run through the serialization class registry
Replace the bespoke WORKFLOW_RUN_CLASS global with the registry the
serialization pipeline already uses to revive Run instances:
- The SWC plugin already auto-registers the workflow bundle's compiled
Run in globalThis[workflow-class-registry], but under a path-derived
classId the host cannot know statically. The workflow-mode create-hook
module now aliases it under a stable id (class//workflow//Run) via a
new aliasSerializationClass() helper (a plain registry entry —
registerSerializationClass cannot be reused since the plugin's IIFE
already defined the non-configurable classId property).
- createConflictingRun() looks the class up with
getSerializationClass(RUN_CLASS_ID, ctx.globalThis) and constructs
through its WORKFLOW_DESERIALIZE hook, exactly as the Instance reviver
would for a serialized Run crossing from a step into the workflow.
- Because the registry is keyed per-global, no environment guard is
needed: a stray host-side import registers the host Run on the host
registry, which is the correct class for that context. The
WORKFLOW_CREATE_HOOK guard, the ??=, and the WORKFLOW_RUN_CLASS symbol
are all deleted.
Verified: 1156 core unit tests; compiled workbench bundle contains the
stable alias alongside the plugin's path-derived registration with zero
WORKFLOW_RUN_CLASS references; all 5 hookGetConflict e2e tests pass
against a local nextjs-turbopack dev server, including conflict
resolution reading conflict.status through a durable step.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Nathan Rajlich <n@n8.io>
* feat: add hook ready promise
* test: cover hook ready continuation scheduling
* feat: replace hook.ready with hook.hasConflict (Promise<boolean>)
- hook.hasConflict resolves true when the token is owned by another
active hook, false once registration is committed — no throw, so
workflows can branch on conflicts early. Awaiting it suspends the
workflow to commit the hook registration (createHook alone does not).
- Chain the already-created fast-path through promiseQueue so
resolution order matches event-log order (review feedback).
- Skip inline step execution when a suspension has an awaited hook
creation so the hasConflict continuation can advance independently
of step execution (review feedback).
- Update unit tests, e2e tests, workbench workflows, and v4/v5 docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: fix inconsistent hasConflict bullet in create-webhook reference
State both resolution values explicitly (true = token already owned,
false = registered) instead of a parenthetical that only described the
false case.
* docs: require docs preview links in PR descriptions for docs changes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: restore SWC Plugin heading in AGENTS.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Nathan Rajlich <n@n8.io>
* [core] V2: pre-schedule the wait timer before inline-executing a step
Fix `Promise.race(step, sleep)` semantics in V2 mixed suspensions
without losing inline step execution.
Inline `await executeStep(...)` blocks the V2 handler for the full
step duration, but `wait_completed` events are only created on the
*next* loop iteration's "complete elapsed waits" pass. So if the
sleep is shorter than the step, replay always picked the step
because the wait_completed event hadn't been written yet —
`sleepWinsRaceWorkflow` returned `'step'` instead of `'sleep'`.
Fix: when a suspension contains both an owned inline step and at
least one pending wait, queue a delayed self-message with
`delaySeconds = suspensionResult.timeoutSeconds` *before* starting
inline execution. The queued continuation fires in a separate
function invocation while the step is still running. That parallel
invocation's "complete elapsed waits" pass writes wait_completed,
replay observes the elapsed wait, and `Promise.race` resolves with
the sleep correctly. The original (still-running) inline invocation
finishes its step, sees `run_completed` on the next loop iteration,
and exits.
This preserves inline-step execution speed for the step-wins case:
the step finishes inline and the workflow returns directly. The
eagerly-queued wait continuation fires after the step has won and
just no-ops on the terminal run.
Test plan:
- New e2e tests `sleepWinsRaceWorkflow` and `stepWinsRaceWorkflow`
exercising `Promise.race` between a step function and `sleep()`,
in both directions.
- Verified locally against `nextjs-turbopack` workbench: both pass.
Event log confirms `wait_completed` is created at t≈1s after
`wait_created` (1s sleep) instead of at t≈11s after the inline
step finishes.
Eager-processing changelog updated with a "Mixed Suspensions"
section describing the pre-scheduled wait approach and its
rationale.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [world-local] Honor delaySeconds before message delivery
The local queue's `queue()` enqueue path ignored the `delaySeconds`
option entirely — every message was delivered immediately, regardless
of the requested delay. VQS-side queues (used by world-vercel and
world-postgres) honor delaySeconds at the broker, so this brings
world-local in line with production semantics.
The runtime needs this to land before the wait-as-continuation
unification in the next commit: that change starts queueing wait
timers as fresh delayed continuations instead of returning
`{ timeoutSeconds }`. Without delaySeconds support, those wait
continuations would fire instantly in dev and trigger spurious
replays.
Sleep happens outside the queue's worker semaphore so a delayed
message doesn't tie up a worker slot during its delay window — other
immediate messages are free to dispatch in parallel.
New tests in queue.test.ts cover:
- delaySeconds > 0 → setTimeout called with the right ms value
- delaySeconds === 0 → no setTimeout (immediate dispatch)
- delaySeconds omitted → no setTimeout (immediate dispatch)
* [core] V2: unify wait+step queue dispatch in suspension processing
Replace the asymmetric "steps go to the queue, waits become a
{ timeoutSeconds } return value" pattern with a single Promise.all
batch that queues every pending operation we are not running inline.
Before this change, suspension processing had three branches that
all needed to keep the wait/step asymmetry consistent:
- pendingSteps.length === 0 returned { timeoutSeconds }
- inlineStep + waits eagerly queued a delayed self-message AND set
inlineStep to undefined (Option A) AND returned { timeoutSeconds }
- inlineStep retry path returned { timeoutSeconds } if there were waits
After this change, every suspension goes through one path:
for non-inline pendingSteps: queue stepId message
if timeoutSeconds defined: queue delayed continuation
await Promise.all(dispatches)
if !inlineStep: return
await executeStep(inlineStep)
Behaviorally, this restores inline step execution even when the
suspension also has a wait (Option A's carve-out is no longer
necessary): the wait timer fires in a separate function invocation
on the queue, in parallel with the inline step. If the sleep wins
the race, that parallel invocation observes wait_completed via the
"complete elapsed waits" pass and finishes the run; if the step
wins, the wait continuation fires later and no-ops on the terminal
run via the existing terminal-event check.
Other cleanups:
- The inline-step retry path no longer needs to forward
suspensionResult.timeoutSeconds — the wait timer was already
enqueued as part of the unified dispatch above.
- A dead post-step `if (timeoutSeconds && pendingSteps.length === 1)`
block (just a comment, no body) is removed; the loop's
"complete elapsed waits" pass handles the same case correctly.
- Step queueing now uses a shared `traceCarrier` rather than
re-serializing per step.
Retry/throttle and hook-conflict paths still return { timeoutSeconds }
since their semantics are "redeliver THIS message after a delay"
rather than "schedule a fresh wait timer." Those can be unified in
a follow-up.
Test plan:
- New e2e tests `sleepWinsRaceWorkflow` and `stepWinsRaceWorkflow`
pass against the `nextjs-turbopack` workbench.
- Event log inspection confirms wait_completed fires at t≈1s (after
wait_created at t≈0s) for the sleep-wins case, and that the inline
step runs only once (no duplicate step_started events that the
earlier eager-queue approach produced in dev).
- All 842 @workflow/core unit tests pass.
- All 346 @workflow/world-local unit tests pass (with the
delaySeconds support added in the previous commit).
Requires the world-local delaySeconds fix in the prior commit;
without it, wait continuations would fire instantly in dev and the
parallel replay would re-enter handleSuspension before the wait
elapsed (recoverable via existing redelivery, but inefficient).
* [docs] V2 unified suspension dispatch + changeset
Update the "Mixed Suspensions" section in eager-processing.mdx to
describe the unified parallel-dispatch model:
- All non-inline pendingSteps are queued with stepId
- The wait timer (if any) is queued as a delayed continuation
- All dispatched in one Promise.all batch
- One owned step is then inline-executed (if any)
The doc previously described Option A (the carve-out where waits
forced all steps to be queued); the unified model removes that
carve-out and explains why the wait continuation works in parallel
with the inline step.
Also notes the dependency on world-local's new delaySeconds support
(landed earlier in the same PR series).
Changeset bumps both @workflow/core and @workflow/world-local since
both packages have user-observable behavior changes.
* [core] Dedupe wait continuations on the wait's correlationId
While a wait is pending, every replay pass over the run re-observes it
(once per step completion in Promise.all([steps..., sleep()]), etc.) and
would enqueue another delayed continuation — each a spurious replay when
the wait elapses, and each a fresh message that resets the
delivery-attempt runaway guard. Key the continuation on the wait's
correlationId so the worlds' idempotency dedupe collapses them.
Near-elapsed waits (<= 2s) are enqueued without the key: a continuation
delivered marginally early (clock skew; the ceil() on the delay can
leave a ~0 margin) re-observes its wait as pending and must be able to
enqueue a fresh short-delay retry. VQS idempotency records persist until
message-retention TTL — reusing the key there would drop the retry and
stall the run permanently.
Also adapts wait-completion-replay tests (from #2038) to the unified
dispatch model: the hook-branch step now executes inline (registered in
the test world, which now returns a step entity from step_started), so
each scenario performs one extra loop-iteration event fetch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [core] Always key wait continuations; bucket the key for near-elapsed waits
CI caught sleepWinsRaceWorkflow failing across the world-postgres lanes:
world-postgres serializes KEY-LESS workflow messages per run
(inflightWorkflowRuns), so a key-less wait continuation parks behind the
flow message that is inline-executing the racing step — wait_completed
lands after step_completed and the race resolves to the step. Keyed
messages take the concurrent dedupe path, so the continuation must
always carry an idempotency key.
The near-elapsed exception (<= 2s) now uses a second-bucketed suffix
instead of omitting the key: an early-delivered continuation re-observes
its wait as pending and re-enqueues with >= 1s delay, which guarantees a
later bucket — a fresh key that dedupe windows cannot drop — while
same-instant duplicates still collapse.
Verified against a local world-postgres setup (express workbench,
Graphile worker): sleepWins/stepWins pass 3/3 with wait_completed at
t+1s; the event log confirms the continuation fires in parallel with
the in-flight inline step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [core] Clamp wait-continuation delays; chain long waits with hop-keyed dedupe
Addresses PR review: the unified dispatch passed delaySeconds to the
queue unclamped while keying the continuation on the bare wait
correlationId. On world-vercel (23h max delay, 24h VQS message
retention) a sleep() longer than the max either failed the dispatch or
was delivered early with its re-enqueue silently dropped by the
still-live idempotency record - stalling the run permanently.
- New runtime/wait-continuation.ts owns delay + idempotency-key
selection: delays clamp to 23h and longer waits chain across hops,
with the hop index suffixed to the key so re-observations within a
hop window dedupe while each hop delivery gets a fresh key. Near-
elapsed threshold and max delay are named constants; full rationale
moved out of the runtime.ts comment block. Unit tests pin the key
selection including chain advancement.
- SuspensionHandlerResult: timeoutSeconds/timeoutWaitCorrelationId
collapsed into waitTimeout?: { seconds, correlationId } so the
pairing can't drift (review nit).
- runtime.test.ts ack-ordering harness adapted to the unified model:
step_created now answers EntityConflictError so the handler observes
the step without owning it and must queue it (the carve-out the tests
relied on - "pending wait disables inline execution" - is exactly
what this branch removes).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [world-local] Abort pending queue sleeps on close()
Addresses PR review: a pending delayed message kept the dev process's
event loop alive for its full delay, and close() only closed the HTTP
agent - a sleep that fired afterwards attempted delivery against the
closed agent and logged a spurious "[local world] Queue operation
failed" error during test/CLI shutdown.
One AbortController owned by the queue now cancels the delaySeconds
sleep, the timeoutSeconds re-delivery sleep, and the retry backoff on
close(); the resulting AbortError is already swallowed by the existing
isAbortError check. close() is idempotent since shutdown paths may
invoke it twice.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [docs] Wait-continuation clamping + hop chaining; changeset
eager-processing.mdx pseudocode now shows the continuation's
idempotency key and clamped delay (PR review nit); the dedupe prose
covers the two key variations (hop suffix for chained long waits,
second bucket for near-elapsed waits). Changeset mentions long-sleep
chaining and world-local's abort-on-close.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
Nathan Colosimo