mirror of https://github.com/vercel/workflow.git synced 2026-09-14 19:59:43 +08:00

Files

T

History

Karthik Kalyan fc81f4502f perf(core): immediate leading-edge dispatch for idle streams (flush window default 0) (#3088 )

* perf(core): immediate leading-edge dispatch for idle streams (flush window default 0)

Production producer-rate data (24h of client flush spans): most agents
average 1.03-1.21 chunks per flush with 87-98% single-chunk flushes and
>70% of chunks arriving more than 10ms after the previous request had
already settled — a fixed 10ms leading window batches almost nothing
for them while adding ~20% to isolated-chunk publish latency (~50ms
median RTT). The one bursty producer (avg ~4-8 chunks/flush) gets its
batching from in-flight accumulation, which does not depend on the
window at all.

The leading chunk of an idle sink now dispatches immediately by
default (window 0): first chunk goes out at once, chunks arriving
during its request coalesce into the next group, and each settle
dispatches the accumulated group immediately — path-independent
batching with no fixed tax on slow producers. A positive
WORKFLOW_STREAM_FLUSH_INTERVAL_MS (or world.streamFlushIntervalMs,
applying from the second group) opts into a windowed leading edge for
slow-but-steady producers that prefer larger groups over first-chunk
latency. Early-ack, the durability drain barrier, wire caps, and
backpressure bounds are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Update packages/world/src/interfaces.ts

Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Signed-off-by: Karthik Kalyan <105607645+karthikscale3@users.noreply.github.com>

* review: env var overrides world streamFlushIntervalMs; world option governs the leading edge too

WORKFLOW_STREAM_FLUSH_INTERVAL_MS, when set, now takes precedence over
world.streamFlushIntervalMs; otherwise the world option applies from the
very first chunk (no more second-group lazy quirk). Deciding waits for
the world when needed, which adds no latency: sendGroup awaits the same
promise before any request can leave.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Signed-off-by: Karthik Kalyan <105607645+karthikscale3@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>

2026-07-24 12:29:28 -07:00