* [core] V2: pre-schedule the wait timer before inline-executing a step
Fix `Promise.race(step, sleep)` semantics in V2 mixed suspensions
without losing inline step execution.
Inline `await executeStep(...)` blocks the V2 handler for the full
step duration, but `wait_completed` events are only created on the
*next* loop iteration's "complete elapsed waits" pass. So if the
sleep is shorter than the step, replay always picked the step
because the wait_completed event hadn't been written yet —
`sleepWinsRaceWorkflow` returned `'step'` instead of `'sleep'`.
Fix: when a suspension contains both an owned inline step and at
least one pending wait, queue a delayed self-message with
`delaySeconds = suspensionResult.timeoutSeconds` *before* starting
inline execution. The queued continuation fires in a separate
function invocation while the step is still running. That parallel
invocation's "complete elapsed waits" pass writes wait_completed,
replay observes the elapsed wait, and `Promise.race` resolves with
the sleep correctly. The original (still-running) inline invocation
finishes its step, sees `run_completed` on the next loop iteration,
and exits.
This preserves inline-step execution speed for the step-wins case:
the step finishes inline and the workflow returns directly. The
eagerly-queued wait continuation fires after the step has won and
just no-ops on the terminal run.
Test plan:
- New e2e tests `sleepWinsRaceWorkflow` and `stepWinsRaceWorkflow`
exercising `Promise.race` between a step function and `sleep()`,
in both directions.
- Verified locally against `nextjs-turbopack` workbench: both pass.
Event log confirms `wait_completed` is created at t≈1s after
`wait_created` (1s sleep) instead of at t≈11s after the inline
step finishes.
Eager-processing changelog updated with a "Mixed Suspensions"
section describing the pre-scheduled wait approach and its
rationale.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [world-local] Honor delaySeconds before message delivery
The local queue's `queue()` enqueue path ignored the `delaySeconds`
option entirely — every message was delivered immediately, regardless
of the requested delay. VQS-side queues (used by world-vercel and
world-postgres) honor delaySeconds at the broker, so this brings
world-local in line with production semantics.
The runtime needs this to land before the wait-as-continuation
unification in the next commit: that change starts queueing wait
timers as fresh delayed continuations instead of returning
`{ timeoutSeconds }`. Without delaySeconds support, those wait
continuations would fire instantly in dev and trigger spurious
replays.
Sleep happens outside the queue's worker semaphore so a delayed
message doesn't tie up a worker slot during its delay window — other
immediate messages are free to dispatch in parallel.
New tests in queue.test.ts cover:
- delaySeconds > 0 → setTimeout called with the right ms value
- delaySeconds === 0 → no setTimeout (immediate dispatch)
- delaySeconds omitted → no setTimeout (immediate dispatch)
* [core] V2: unify wait+step queue dispatch in suspension processing
Replace the asymmetric "steps go to the queue, waits become a
{ timeoutSeconds } return value" pattern with a single Promise.all
batch that queues every pending operation we are not running inline.
Before this change, suspension processing had three branches that
all needed to keep the wait/step asymmetry consistent:
- pendingSteps.length === 0 returned { timeoutSeconds }
- inlineStep + waits eagerly queued a delayed self-message AND set
inlineStep to undefined (Option A) AND returned { timeoutSeconds }
- inlineStep retry path returned { timeoutSeconds } if there were waits
After this change, every suspension goes through one path:
for non-inline pendingSteps: queue stepId message
if timeoutSeconds defined: queue delayed continuation
await Promise.all(dispatches)
if !inlineStep: return
await executeStep(inlineStep)
Behaviorally, this restores inline step execution even when the
suspension also has a wait (Option A's carve-out is no longer
necessary): the wait timer fires in a separate function invocation
on the queue, in parallel with the inline step. If the sleep wins
the race, that parallel invocation observes wait_completed via the
"complete elapsed waits" pass and finishes the run; if the step
wins, the wait continuation fires later and no-ops on the terminal
run via the existing terminal-event check.
Other cleanups:
- The inline-step retry path no longer needs to forward
suspensionResult.timeoutSeconds — the wait timer was already
enqueued as part of the unified dispatch above.
- A dead post-step `if (timeoutSeconds && pendingSteps.length === 1)`
block (just a comment, no body) is removed; the loop's
"complete elapsed waits" pass handles the same case correctly.
- Step queueing now uses a shared `traceCarrier` rather than
re-serializing per step.
Retry/throttle and hook-conflict paths still return { timeoutSeconds }
since their semantics are "redeliver THIS message after a delay"
rather than "schedule a fresh wait timer." Those can be unified in
a follow-up.
Test plan:
- New e2e tests `sleepWinsRaceWorkflow` and `stepWinsRaceWorkflow`
pass against the `nextjs-turbopack` workbench.
- Event log inspection confirms wait_completed fires at t≈1s (after
wait_created at t≈0s) for the sleep-wins case, and that the inline
step runs only once (no duplicate step_started events that the
earlier eager-queue approach produced in dev).
- All 842 @workflow/core unit tests pass.
- All 346 @workflow/world-local unit tests pass (with the
delaySeconds support added in the previous commit).
Requires the world-local delaySeconds fix in the prior commit;
without it, wait continuations would fire instantly in dev and the
parallel replay would re-enter handleSuspension before the wait
elapsed (recoverable via existing redelivery, but inefficient).
* [docs] V2 unified suspension dispatch + changeset
Update the "Mixed Suspensions" section in eager-processing.mdx to
describe the unified parallel-dispatch model:
- All non-inline pendingSteps are queued with stepId
- The wait timer (if any) is queued as a delayed continuation
- All dispatched in one Promise.all batch
- One owned step is then inline-executed (if any)
The doc previously described Option A (the carve-out where waits
forced all steps to be queued); the unified model removes that
carve-out and explains why the wait continuation works in parallel
with the inline step.
Also notes the dependency on world-local's new delaySeconds support
(landed earlier in the same PR series).
Changeset bumps both @workflow/core and @workflow/world-local since
both packages have user-observable behavior changes.
* [core] Dedupe wait continuations on the wait's correlationId
While a wait is pending, every replay pass over the run re-observes it
(once per step completion in Promise.all([steps..., sleep()]), etc.) and
would enqueue another delayed continuation — each a spurious replay when
the wait elapses, and each a fresh message that resets the
delivery-attempt runaway guard. Key the continuation on the wait's
correlationId so the worlds' idempotency dedupe collapses them.
Near-elapsed waits (<= 2s) are enqueued without the key: a continuation
delivered marginally early (clock skew; the ceil() on the delay can
leave a ~0 margin) re-observes its wait as pending and must be able to
enqueue a fresh short-delay retry. VQS idempotency records persist until
message-retention TTL — reusing the key there would drop the retry and
stall the run permanently.
Also adapts wait-completion-replay tests (from #2038) to the unified
dispatch model: the hook-branch step now executes inline (registered in
the test world, which now returns a step entity from step_started), so
each scenario performs one extra loop-iteration event fetch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [core] Always key wait continuations; bucket the key for near-elapsed waits
CI caught sleepWinsRaceWorkflow failing across the world-postgres lanes:
world-postgres serializes KEY-LESS workflow messages per run
(inflightWorkflowRuns), so a key-less wait continuation parks behind the
flow message that is inline-executing the racing step — wait_completed
lands after step_completed and the race resolves to the step. Keyed
messages take the concurrent dedupe path, so the continuation must
always carry an idempotency key.
The near-elapsed exception (<= 2s) now uses a second-bucketed suffix
instead of omitting the key: an early-delivered continuation re-observes
its wait as pending and re-enqueues with >= 1s delay, which guarantees a
later bucket — a fresh key that dedupe windows cannot drop — while
same-instant duplicates still collapse.
Verified against a local world-postgres setup (express workbench,
Graphile worker): sleepWins/stepWins pass 3/3 with wait_completed at
t+1s; the event log confirms the continuation fires in parallel with
the in-flight inline step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [core] Clamp wait-continuation delays; chain long waits with hop-keyed dedupe
Addresses PR review: the unified dispatch passed delaySeconds to the
queue unclamped while keying the continuation on the bare wait
correlationId. On world-vercel (23h max delay, 24h VQS message
retention) a sleep() longer than the max either failed the dispatch or
was delivered early with its re-enqueue silently dropped by the
still-live idempotency record - stalling the run permanently.
- New runtime/wait-continuation.ts owns delay + idempotency-key
selection: delays clamp to 23h and longer waits chain across hops,
with the hop index suffixed to the key so re-observations within a
hop window dedupe while each hop delivery gets a fresh key. Near-
elapsed threshold and max delay are named constants; full rationale
moved out of the runtime.ts comment block. Unit tests pin the key
selection including chain advancement.
- SuspensionHandlerResult: timeoutSeconds/timeoutWaitCorrelationId
collapsed into waitTimeout?: { seconds, correlationId } so the
pairing can't drift (review nit).
- runtime.test.ts ack-ordering harness adapted to the unified model:
step_created now answers EntityConflictError so the handler observes
the step without owning it and must queue it (the carve-out the tests
relied on - "pending wait disables inline execution" - is exactly
what this branch removes).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [world-local] Abort pending queue sleeps on close()
Addresses PR review: a pending delayed message kept the dev process's
event loop alive for its full delay, and close() only closed the HTTP
agent - a sleep that fired afterwards attempted delivery against the
closed agent and logged a spurious "[local world] Queue operation
failed" error during test/CLI shutdown.
One AbortController owned by the queue now cancels the delaySeconds
sleep, the timeoutSeconds re-delivery sleep, and the retry backoff on
close(); the resulting AbortError is already swallowed by the existing
isAbortError check. close() is idempotent since shutdown paths may
invoke it twice.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [docs] Wait-continuation clamping + hop chaining; changeset
eager-processing.mdx pseudocode now shows the continuation's
idempotency key and clamped delay (PR review nit); the dedupe prose
covers the two key variations (hop suffix for chained long waits,
second bucket for near-elapsed waits). Changeset mentions long-sleep
chaining and world-local's abort-on-close.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
* fix(world-local): return HTTP 200 instead of 503 for queue timeout re-enqueue signals
The local queue used HTTP 503 (Service Unavailable) as an internal signal
for re-enqueueing after a timeout delay. This is misleading since 503 is
an error status, causing confusion when inspecting logs or network traffic.
Changed to return 200 with timeoutSeconds in the body, and updated the
consumer to detect the re-enqueue signal from the response body instead
of the status code.
* fix(world-local): handle timeoutSeconds: 0 and add queue re-enqueue tests
Address review feedback: timeoutSeconds === 0 should trigger an immediate
re-enqueue without delay, not be treated as a normal success. Also adds
test coverage for the handler response codes and re-enqueue behavior.
Nathan Colosimo