Enrich the existing v4 event-write `http POST` **CLIENT** span. No new
span is created.
The modified span is emitted by HTTP event writes through `fetchV4` /
`instrumentedFetch` and WebSocket event writes through the existing
synthetic parity span in `postEventFrameOverWs` / `withHttpClientSpan`.
## Added attributes
- `workflow.client.version` on every v4 event write, formatted as
`@workflow/world-vercel/<version>`
- `step.stso_ms` when the event input carries step-to-step overhead
telemetry
- `step.latency_optimizations` when the event input carries the active
runtime optimization list, such as `lazyStepStart`, `turbo`, and
`optimisticStart`
The span already inherits the client application OTEL `service.name` and
carries the event type, transport, URL, and status. The added fields
allow sampled STSO analysis by client service, Workflow SDK version, and
runtime mode on one span.
## Validation
- Dependency-aware `@workflow/world-vercel` build
- `ws-transport-spans.test.ts`: 13 tests passed
- `@workflow/world-vercel` typecheck
- Biome and `git diff --check`
---------
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
* [core] Give events-consumer deferred-check tests a stall-proof budget
The 'duplicate event classes' tests reach their outcome through the
deferred check's multi-stage timer chain (promise queue -> setTimeout(0)
-> idle poll -> delay timer), which a loaded CI runner with coarse
timers can starve for whole seconds. The afterDeferredCheck poll capped
that at 2s and two follow-up assertions used vi.waitFor's 1s default,
inside the 5s default test timeout - regularly starved through on
Windows runners ('does not track hook deliveries' and 'leaves a
duplicate run_cancelled' flaked ~1.5x/day over the last 10 days,
failing with strandedEvent/parkedSummary still undefined).
The polls return as soon as their assertions hold, so raising the poll
timeout to 15s and the suite budget to 30s costs healthy runs nothing
while bounding only genuinely stalled runners.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
* [core] Match the TTL-expiration abort e2e test budget to its siblings
The other distributedAbortController tests run with a 60s budget; the
TTL-expiration one got 30s. Its 3s TTL is trivial, but on a fresh prod
deployment cold starts plus queue backlog routinely push run start +
first stream delivery past 30s - it timed out on two apps
simultaneously in a single Tests run over the last week.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
* Add changeset
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
* [ci] Retry e2e tests once in CI, keeping retried tests visible
Over the last 10 days ~93 Tests runs were manually re-run until green,
some taking 6 attempts: the e2e suites drive real deployments, and a
single test losing a timing race fails a whole 20+ minute matrix job.
A CI-only vitest retry (retry: 1) absorbs those single-test races.
beforeEach/afterEach hooks run per attempt, so suites with file-restore
hooks (dev.test.ts) retry cleanly.
Retries must not hide real races, so a retried-then-passed test stays
visible everywhere a failure would have been: the github-reporter emits
::warning annotations and an e2e-flaky-*.json sidecar, every e2e job
uploads it, and aggregate-e2e-results.js renders a 'Flaky E2E Tests
(passed on retry)' section in both the per-job step summary and the PR
comment, with per-app occurrence counts.
Harnesses whose failures are themselves the signal pin retry: 0:
event-log-race-repro (a pass runs the full configured budget) and
benchmarks (a regression should not be papered over by a luckier
second sample). Local runs keep retry at 0 so races reproduce while
debugging.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
* Add changeset
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
* feat(world-vercel): synthesize per-event client spans on the WS transport
PR #3084 added the opt-in `WORKFLOW_EVENTS_TRANSPORT=ws` path and listed
"no client-side span on the WS path" as a known limitation. Because event
writes become multiplexed frames on one long-lived socket rather than
individual `fetch` calls, the per-event `http POST` CLIENT span that the
HTTP transport produced simply disappeared — traces went from one span
per event to nothing between the invocation and the server.
Restore it by synthesizing a request-shaped span around each frame, and
give the upgrade its own span:
- Extract `withHttpClientSpan` / `recordClientSpanStatus` from
`instrumentedFetch` in `http-core.ts` so the synthetic span is emitted
by the same envelope as the real one and cannot drift from it.
`InstrumentedFetchOptions` now extends `HttpClientSpanOptions`.
- `postEventFrameOverWs` opens `http POST` with `url.full` pointing at the
v4 REST endpoint the frame is forwarded into, so per-event traces and
latency dashboards keep working across the flag. Extract `eventsV4Url`
so that URL cannot drift from the one the HTTP path actually requests.
- Tag both transports with `workflow.events.transport` (`http` | `ws`) and
`workflow.event.type`; the WS path additionally sets
`network.protocol.name=websocket`, `workflow.events.ws.url` (the real
wire destination) and `workflow.events.ws.req_id` (join key to the
server's log line for the frame), so the span is never mistaken for a
real HTTP request.
- Add a `workflow.events.ws.connect` span around the upgrade — the one
genuinely-HTTP request here, previously the invisible half of every WS
write's latency — carrying `workflow.events.ws.reconnect_attempt`. This
also puts `resolveUpgradeHeaders`' trace-context injection inside a
client span, as AGENTS.md requires.
- Fix `parseServer` to treat `wss:` as TLS (port 443, not 80).
Out of scope, deliberately: per-frame `traceparent` (needs a frame-meta
field plus a server change) and Vercel's outgoing-requests view (that
instruments global `fetch`, so a frame structurally cannot appear there).
Covered by `ws-transport-spans.test.ts`, which drives the real selection +
transport + adapter stack over a fake socket and asserts span shape,
failure reporting, retry behaviour and HTTP/WS parity.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* chore: trim WS spans changeset to the user-facing summary
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* fix(world-vercel): only tag event-write spans with transport
Signed-off-by: Shalabh Chaturvedi <shalabh.chaturvedi@vercel.com>
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
* fix(world-vercel): format WS transport span regression test
Signed-off-by: Shalabh Chaturvedi <shalabh.chaturvedi@vercel.com>
Co-Authored-By: Shalabh Chaturvedi <shalabh.chaturvedi@vercel.com>
---------
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
Report end-to-end TTR for a hook resumption — entry into the public resume
API through to the first line of the next durable step — on that step's
`step.execute` span, decomposed into non-overlapping phases that sum
exactly to the total:
workflow.resume.total_ms
workflow.resume.phase.{producer_prep,queue_delivery,resume_setup,
replay,step_dispatch,step_claim,step_prepare}_ms
dimensioned by trigger, dispatch strategy, setup source, and whether the
step ran inline or was dispatched to another invocation.
T0 is stamped by whichever public entry point the caller used, so
`resumeWebhook` — which does its own by-token lookup and key resolution
before reaching the shared implementation — measures the same window as
`resumeHook` rather than a systematically shorter one. T7 is taken inside
`contextStorage.run`, immediately before `stepFn.apply()`, so the
`step_prepare` phase covers the step-context setup it is defined to cover.
`resumeHook()` puts the producer boundaries on an optional
`hookResumeTiming` field on the queue message (both dispatch paths); the
consuming invocation adds its own and hands them to the execution that
will actually ATTEMPT the next durable step. That decision is made against
the dispatch loop's own classification, so an owned-recovery step keeps
the measurement here instead of it riding off on a queued sibling, and a
step converted into a delayed backstop wake — which this delivery does not
attempt — never takes it. Within an inline batch the tracking is shared
and a one-shot latch picks the single step that reaches user code, so the
sample survives the batch's first step losing its create-claim. A
deployment-affinity re-route forwards the timing untouched, keeping the
wasted hop inside `queue_delivery`.
The field is optional in every direction (new producer/old consumer, new
consumer/old message, no workflow-server change) and parses with
`.catch(undefined)` so a malformed value can never fail a delivery. A
sample is emitted only when every required boundary is present, finite,
and monotonic — a skewed or incomplete set is dropped rather than
reported as a negative phase.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
## Summary & Motivation
Adds `events.getMany` to the analytics API, letting dashboard consumers
enrich canonical event pages with ClickHouse provenance in one bounded,
deduplicated batch request instead of N point lookups. Coordinated with
a workflow-server batch endpoint.
## Test Plan
Added unit tests for the client and schema; also ran Biome checks, `git
diff --check`, and world-vercel typecheck.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
## Summary
- reject compact sparse arrays above the supported logical length at the
main devalue hydration boundary
- delegate accepted sparse-array construction to devalue
- cover both current binary payloads and legacy flattened payloads
## Why
Compact sparse-array encodings can represent a logical length that is
disproportionate to the stored payload. Applying one codec-level bound
keeps hydration predictable before downstream consumers process the
decoded value.
## Impact
Compact sparse arrays with logical lengths above 100,000 now fail
hydration with a `RangeError`. Other payloads are unchanged.
## Verification
- `pnpm --filter @workflow/core test` — 2,023 passed, 3 expected
failures
- `pnpm --filter @workflow/core typecheck`
- `pnpm --filter @workflow/core build`
- focused serialization suite — 154 passed
- direct root-argument, bound-step, and aggregate-error payload checks
- reuse, quality, and efficiency review
## Summary & Motivation
Adds `StreamExpiredError` to `@workflow/errors`, carrying the run, stream, and server-reported expiry timestamp from workflow-server's 410 `stream-expired` envelope. The reconnect loop rethrows it instead of retrying, since retention expiry is terminal and a retry budget would only convert it into a generic exhaustion error.
## Test Plan
Unit tests added for the 410 decoding path and the reconnect rethrow; typechecks pass across the touched packages.
* feat(world,world-vercel,core): resilient step dispatch (parallel step_created + queue publish)
Newly created steps are handed to the queue in parallel with their
step_created event write, with the serialized input carried on the
message (stepInput) so the queue consumer can idempotently re-ensure
the event when the direct write failed transiently — mirroring
resilient start (runInput) and resilient hook resume (hookInput).
- @workflow/world: stepInput on WorkflowInvokePayload,
CreateEventParams.viaStepDispatch, WorldCapabilities.resilientStepDispatch
- core (node:vm): suspension handler publishes eligible steps alongside
their create; the dispatch pass skips them (queuedStepCorrelationIds)
- core (quickjs): dispatchPendingOps does the same for overflow steps;
the ineligible fallback is now published in parallel too (removes the
serial per-step enqueue loop)
- consumer: on a redelivery, a stepInput-carrying message re-ensures
step_created (marked viaStepDispatch) before executing
- under an enforced precondition guard the parallel path requires
backend cooperation (capabilities.resilientStepDispatch, declared by
world-vercel): a 412-rejected step's in-flight dispatch is revoked
server-side and its re-ensure refused
- step dispatch/retry idempotency keys are step-identity-scoped
(cid + hashed step name) so a revoked message for a reassigned
correlation id cannot absorb the corrected schedule's dispatch
- kill switch: WORKFLOW_RESILIENT_STEP_DISPATCH=0
* Validate stepInput.input as Uint8Array at the schema boundary
Review feedback: producers only attach stepInput when the dehydrated
input is binary and the queue transport preserves bytes (CBOR), so a
non-binary value means the payload was mangled in transit. Enforcing
Uint8Array in StepDispatchInputSchema fails the message parse instead
of silently writing non-binary data into a step_created, and types the
consumer's re-ensure so the unchecked 'as SerializedData' cast goes
away.
* Keep sequential dispatch under an enforced precondition guard (drop the resilientStepDispatch capability lift)
Review feedback (two P1s): backend-side revocation bookkeeping cannot
carry the guard's correctness property across the queue side-channel —
- nothing orders a slow guarded create's eventual 412 (the moment the
backend learns the dispatch is poisoned and records the revocation
marker) before the consumer's redelivery re-ensure, so attempt > 1
is a probabilistic mitigation, not a happens-before; and
- a best-effort marker that fails open (Redis loss) cannot back a
capability the SDK treats as a correctness attestation.
Only sequencing the publish after the create gives the message a
happens-after edge over the create's guard verdict, so the guard gate
is now unconditional: worlds that enforce the precondition guard keep
the sequential create-then-publish dispatch. The parallel resilient
path remains for unguarded writes (the quickjs engine everywhere, and
worlds without the guard). Removes WorldCapabilities.resilientStepDispatch
and world-vercel's declaration; the viaStepDispatch flag is kept and
re-documented as advisory (server-side defense-in-depth only).
This also dissolves the reviewed dedupe hazard on the step-identity-
scoped dispatch keys: with no 410-ack path in any real SDK flow, a
message for a never-created step keeps redelivering until an entity
exists, execution always hydrates input from the committed entity
(never the message), and a name-mismatched stale start is skipped by
the server's stepName fence.
* Correct the MAX_RESILIENT_STEP_INPUT_BYTES rationale: VQS has no hard message-size cap
256 KB is the queue's inline-vs-S3 threshold, not a rejection limit
(payloads above it spill to S3-backed storage transparently). The
128 KiB bound is a cost/latency choice — keep step messages on the
inline path rather than paying an S3 double-hop for bytes that already
live in the event log.
* Recover a missing step in-band when a stepInput-carrying delivery beats its create
Durabench parallel sweeps (guard-off, node engine) caught ~4-8% of
fan-out runs stalling one branch for ~306s on the resilient dispatch
path. Root cause: the consumer's step_created re-ensure was gated on
metadata.attempt > 1, but world-vercel's failure-retry path re-enqueues
a FRESH message whose attempt resets to 1 — so when a delivery beat the
producer's parallel step_created write, every fast retry hit the same
'step not found' rejection with attempt 1, and the step only recovered
when the ORIGINAL message's ~300s visibility-timeout redelivery finally
arrived with attempt 2.
The recovery is now in-band and attempt-independent: when a
stepInput-carrying execution rejects with the step-missing signature
(WorkflowWorldError, 404 or the local worlds' message shape), the
consumer materializes the step_created from the message payload and
retries the execution once within the same delivery. The eager
attempt>1 ensure is kept as a round-trip saver on genuine redeliveries.
Sweep effect expected: the 305-306s TTLS outliers disappear while the
resilient path keeps its p50 win (1054ms vs 1425ms at 64 branches).
* Add world-sim: a deterministic simulation World for concurrency scenarios
`@workflow/world-sim` is an in-process World implementation whose point is
that nothing in it races. Every world call is a stoppable point with
`before`/`positioned`/`after` phases, every call is attributed to a writer
(orchestrator, a named step body, or an out-of-band external client), and
the clock is virtual — a thirty-day sleep costs no wall time. A scenario
script says "stop this writer here, do that, let it go", so an interleaving
that a real deployment leaves to chance becomes something you can name.
The model follows workflow-server where it matters:
- Event ids are minted at the handler boundary, not at storage append.
DynamoDB does not generate ids, so the server does, in the request
handler — and that id is the log's sort key. This is what makes "earlier
log position, later commit" expressible, and it is the hazard the fence
scenarios are about.
- The out-of-band write marker is keyed on the event's own ULID time and is
forward-only, over `hook_received`, `step_completed` and `step_failed`.
- Both halves of the staleness fence are modelled behind flags: the
watermark (`preconditionGuard`) that clients send today, and the count
(`countGuard`) that they do not. The count is synthesized on the caller's
behalf and keyed by run rather than by writer, since an orchestrator and
its inline step bodies are one process sharing one loaded log.
`workbench/sim-world` is the scenario book — 39 of them, each a workflow
plus a script. Several are pinned corruptions rather than passing
assertions: they record what the runtime does today, so that a change in
behaviour shows up as a diff. The doc-29/30/31 trio is the argument for the
count guard, one flag apart: (B, A, C) corrupts under the watermark alone,
is fenced once the count is on, and (B, C, A) corrupts with both on. That
last one suspends mid-run on purpose. The fence is a conditional append
evaluated inside the storage write, so there is no checked-but-uncommitted
moment to slip past; the window nothing can close is the quiescent gap
between deliveries, where the run makes no writes and so meets no checks.
Both packages are private and unpublished, so no changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* Add the new sim-world workspaces to the lockfile
Kept separate from the source commit because it is not a clean diff. The
two new importers are the part that belongs to this branch; the rest is
re-resolution churn from running `pnpm install` at a later date than
whoever last touched the file — `latest` specifiers like docs' `radix-ui`
move on their own. Drop or regenerate this commit if the churn is
unwelcome; the source commit stands on its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* Make a consistency violation fail the scenario that tripped it
`expect.violations` let a scenario declare the corruption it reproduced and
pass for reproducing it. That is a green suite describing a broken system,
and it goes red on the day someone *fixes* the bug — backwards, and the
opposite of what a test is for.
The field is gone. A scenario now states the outcome the run should have
reached, which for a corruption means the branch its own durable log
implies, and stays red until the runtime gets there. Any violation fails the
scenario. The six reproductions read as ordinary failures now:
expected output "afterSlow:doc-26", got "afterFast:doc-26"
Six scenarios are therefore red, and `run.ts` exits non-zero. The count is
the signal: seven is a regression, five means something was fixed and a
scenario is ready to retire. Five of the six have known fixes — four predate
the count guard, and doc-29 goes green the moment a client sends
`stateEventCount`. doc-31 has none, which is the point of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* Document the world-sim implementation in DESIGN.md
Folds the working notes into a checked-in design doc: module map, the
runtime model the simulator has to match, the interception model and its
three call phases, the determinism machinery, the store's guards and
fault injection, the writer vocabulary, the termination budgets, both
consistency checkers, and current test status.
Links it from both READMEs, and updates the workbench's doc-31 note now
that the append-tail fence it needs exists as a proposal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* Say which fix closes each red scenario, and which are demonstrated
The fix column read "predates the count guard" on three rows, which scans as
"the count guard is involved" when it meant the opposite. It was also wrong on
one: `two racing STEPS, no hook anywhere` cannot be fixed by anything predating
the count guard, since the row below it is that same fault with
`preconditionGuard` on and failing identically.
The column now names the specific change, and a second column separates a fix
that is argued for from one that is shown — a passing scenario that is the red
one with the fix armed, same tempo, one flag apart. Two of the six have that;
three name a fix with no paired scenario yet; doc-31 has none.
Also states the thing the column could imply but does not mean: `countGuard`
requires `stateEventCount`, which no client sends, so three of the five
identified fixes are dark in production.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* Give scenarios stable ids, refer to events by log position, add colour
Four changes to make a rendered scenario something you can cite and read.
Scenario ids. Every scenario gets a hyphenated `id` next to its prose name —
`in-flight-after-decision`, `stale-read-step-count-fork`. The name is a
sentence and will be reworded; the id is what a commit message, a bug report
or `pnpm sim <id>` refers to. The runner matches the id first and falls back
to the name, so existing invocations keep working, and the table of the six
red scenarios in DESIGN.md now cites ids.
One reference scheme. Events were referred to three ways at once: the trace
printed raw correlation ULIDs, violation messages carried `evnt_<ulid>` event
ids, and the docs spoke of "position 7". Now there is only log position. `#12`
is the twelfth event in the durable log sorted the way `events.list` sorts it,
`@7` is the resource created at position 7, and ids inside violation messages
are rewritten on the way out.
That numbering is deliberately in *log* order while the trace prints in
*commit* order, which makes the subject of the red scenarios visible on the
page: a run whose log disagrees with the order its writers committed in shows
positions counting backwards.
Colour, by event family, and only when the destination is a terminal. Off
under `NO_COLOR` or `--no-color`, forced on with `--color`. With colour off
the output is the same plain ASCII, so it stays usable as a golden file.
The workflows under test are now one file. Nothing imported across them and
scenarios read them alongside the tempo that steers them, so five modules were
a two-file hop for no benefit.
Unit tests 61/61. Scenario book unchanged at 33 passed, 6 failed.
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* Add an append-only log world, and split the book into one file per scenario
Three things the book could not do before.
`appendOnlyLog` gives an event its log position at *commit* instead of when
its handler minted one. That is the single change that makes a stale read
impossible — the log can be behind, never wrong — so playing the same 39
scenarios with and without it is how you tell which of the six reds the change
would actually close. All six: 33 pass / 6 violations mint-ordered, 39 pass /
0 violations append-only. It bundles two effects that have to move together:
an overtaken write re-mints at the tail, and a withheld read returns a prefix
rather than a hole, which is why `onStaleRead` now carries `{eventId, hidden,
truncated}` and the trace distinguishes a lagging read from a stale one.
`preconditionGuard` on `RunScenarioOptions` forces the fence off across the
book, asking whether anything relies on it. Violations 6 -> 8 mint-ordered, so
it is load-bearing there; 0 -> 0 append-only, so it is dead weight once
positions are assigned at commit. Both flags are tri-state: `undefined` leaves
it to the spec, which is not the same as `false`.
The scenario book was a 1020-line file; it is now 39 files and an index that
only decides reading order. Same 39 ids. This is the whole answer to "how do I
add a scenario" — copy the file next door — and it is why the README can be
short. Three `in-flight-*` scenarios are reworded so that no expectation is
restated per world: a scenario is one sequence of movements, the only thing a
world changes is what a read returns, and what catches the fault in both is
the invariant that a run's log must replay back into that run.
Also: `loadFlowHandler` moves out of `build.ts` into `load.ts`, so playing
scenarios no longer drags SWC and esbuild into the module graph; the CLI gains
`--report-only`, `--summary-file` and `--detail-file` for CI, with the default
still exiting non-zero; and the workbench's `test` script points at
`--report-only` so a recursive `pnpm -r test` does not go red for the six.
Docs are split by task: the workbench README is how to add a scenario, and the
package README plus DESIGN.md are how to change the simulator.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* Publish the scenario book from CI, without blocking a merge
Plays the book on every pull request, once per log world, and posts both
summaries as one sticky comment.
It never gates. Six scenarios are red on purpose — each is a reproduction of a
corruption the runtime can still produce, stating the outcome its own durable
log implies and staying red until the runtime gets there — so a lane that
failed on them would be red on every PR and read as broken rather than as
informative. What it publishes is the pair of counts, and the thing to look at
is whether they still say 33/6 and 39/0. A seventh red is a regression; five
means something got fixed and a scenario is ready to retire. The append-only
column is the measurement the pair exists for: it says which of the six would
close if positions were assigned at commit.
Non-blocking at the job level rather than only on the two sim steps, so that a
failed install or a PR comment the token cannot write does not turn this into
a red X either. The steps themselves keep their real exit codes, so the run
still says which world was clean.
`--title` is new on the CLI: two summaries land in one comment, and two
headings reading "world-sim scenario book" would leave the chips line as the
only way to tell them apart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* Make the Sim World comment four lines until you open it
Two collapsed folds, one per world, each with its count and a green or orange
dot on the visible line; the table is behind them. Nothing else above the fold.
The failures list is gone. Six scenarios are red on purpose, so a comment that
led with them led with the part that was not news, and grew a wall of text on
exactly the PRs that changed nothing. The count is the signal — and anyone who
wants the names can open the table, which has always had them.
`renderMarkdownSummary` now renders no heading of its own, since it is built
to be stacked under one; the workflow supplies the heading and the one-line
description. `mdCell`, `clip` and `dedupe` existed only for the failures table
and go with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* docs(sim-world): drop the six-red list, add an API reference
The "six red scenarios" section was a second copy of something `pnpm sim`
already prints exactly, and the copy that goes stale. What replaces it says how
to read a red — an open bug stating the outcome its own durable log implies,
green when fixed rather than when seen again — and points at DESIGN.md for the
part that is not re-derivable from a run.
The reference has three tables. Writers: what a concurrent thread of execution
is here, and the four ids one can have. Movements: the instruction that moves a
writer to a place and holds it, described against the world boundary, the
position in the event log, and the commit to storage — three moments, which is
why there are three stops rather than one. Withholdings: the two ways to change
what a reader sees without holding anybody.
Terminology unified on hold/held (the word `Held` and the trace already use)
and on "assigned a position in the event log" / "committed to storage", which
also fixes the `runToEventProduced` docstring: the event has crossed the world
boundary there, so "submitted" was the misleading half.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* docs(sim-world): move the API reference into the package, uniform vocabulary
The reference documents `@workflow/world-sim`, so it belongs beside the API and
not in the workbench that happens to be its first caller. It replaces the old
`## Writers` section, which said most of the same things in a different order
and a different vocabulary.
Three nouns, one meaning each. A *writer* is a thread of execution — the table
lists all three kinds with the handle that names each and the events each one
writes. An *advance* moves one writer to a named place and holds it; the three
`runTo*` stops are the three moments a write has (crossed the world boundary,
assigned a position in the event log, committed to storage), which is why there
are three rather than one, and the table says which way a concurrent commit
sorts at each. A *withholding* hides something from readers without holding
anybody, which is what `withholdNextEvent` and `beginHookDelivery` have in
common and why the latter is not an advance.
"Movements" is gone in favour of "advances", which the code and the older prose
already used. "Held" replaces "stopped"/"paused" throughout, matching `Held` and
`isHeld()`.
The workbench README loses the duplicate tables and keeps the four rules that
bite on a first scenario, so it is a guide to adding one and links out for the
rest.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* docs(world-sim): rename "Two logs" to "World behaviors", one-line motivations
The section described two logs, but there are four world behaviors a scenario
can pick between: the two log orderings and the two guards. Renamed and
regrouped so each one is described by what it does rather than by which of the
current reds it closes.
Measured counts and "production" / "what happens today" framing are out of this
README throughout — they belong to a run of the book, and a doc that carries
them is stale the first time a scenario changes colour. The workbench README
and the CI lane still print them, where they read as measurements.
Every section's motivation is one line; the file-top motivation is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* docs(world-sim): Usage shows both halves — the workflow and the script
The example was a script with no workflow beside it, which hid the fact that
you write both. Replaced with the smallest pair that has something to control:
two steps in flight at once, and a script that decides which of them reaches
the log first.
The scenario in it was run before it was written down — `prepare` held before
it takes a position, `finalize` committed into the earlier slot, #6 then #7 —
so the positions the prose cites are the ones the trace prints.
Link to the workbench moved to the end of the section, where it reads as
"where to go next" rather than as an aside mid-example.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* docs(world-sim): say what "arm" meant instead of using the word
The READMEs told a reader to arm a wait without ever saying that calling an
advance and awaiting it are two separate things. That is the whole mechanism —
`runTo` registers its watch synchronously when called, and the returned promise
only reports arrival — so it is stated plainly once in Advances and the jargon
is dropped everywhere it stood in for the explanation.
"Armed" survives only where it means a guard is switched on, which is a
different word doing a different job.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* docs(world-sim): two symmetric watches in the Usage example
The example started one watch and awaited the other inline, which is exactly
the shape a reader cannot generalise from — it looks like the two calls do
different things. Both are now started, then both awaited, so the sentence
above the block and the code below it say the same thing.
Re-run before committing: same log, finalize at #6 and prepare at #7.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* docs(world-sim): neutral names in the Usage example
`prepare` / `finalize` / `held` / `committed` made a reader decode four
domain-ish names to see a mechanism that has nothing to do with any of them.
stepA and stepB, watchA and watchB: the only thing left to notice is that the
two watches stop at different points, which is the whole lesson.
This detaches the example from the workbench's `parallelStepsWorkflow`, so it
was verified against a throwaway copy of the workflow rather than assumed —
same shape, stepB at #6 and stepA at #7, replay ok.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* refactor(world-sim): one event fold, two call phases, a smaller entry
Three reductions to the same end — less surface to keep consistent — with
the scenario book as the check that none of them changed behaviour.
**One fold.** `create` validated *and* applied an event in a 17-case switch;
`foldSeededEvent` applied it again in 15 cases without validating. Two copies
of the event -> entity state machine that only a replay compares, and any
divergence between them makes a replay disagree with the run it is checking
for a reason that is not the runtime's fault. Both paths now end in one total
`applyEvent`: the write path validates first and calls it, `seedFromLog`
calls it with no validation at all, because those events were accepted once
already and re-litigating them would reject legitimate history. store.ts
loses 175 lines. Characterization tests for the seeded path went in first and
were confirmed to bite before the refactor started.
**Two phases.** `CallPhase` had a `positioned` phase between `before` and
`after`, and `runToPositionMinted` / `runToCall` to park on it. Nothing in the
book used any of the three: the mint/commit gap they existed for is reached
through `beginHookDelivery`, which owns both halves explicitly and does not
block the writer that made the write. Removing the phase also removes an
`await` from the interception path, so this was measured rather than reasoned
about — all four book runs are unchanged.
**A smaller entry.** `index.ts` re-exported the construction kit —
`createSimWorld`, `createSimStore`, `driveQueue`, `verifyReplay`,
`checkInvariants`, the clock — which nothing outside the package imports and
which made every one of their signatures a compatibility promise. The entry
is now the scenario surface; extenders import from the module. `InFlightWrite`
joins it, having been missing though `beginHookDelivery` returns it.
Unchanged, and the point of saying so: 33/6/6 mint-ordered, 39/0/0
append-only, 8 violations with `--no-fence`, 0 with both. 70 unit tests green
(up from 67), tsc and biome clean in both workspaces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* fix(world-sim): follow the new event-result contract
Rebase fixups for four core replay commits, no behaviour change of our own.
`EventResult` is now a union — a populated page (`events` + `cursor` +
`hasMore`) or all three absent — so the three fields cannot be assembled from
three independent variables the type has no way to see agree. They travel as
one `deltaPage` object, spread as a whole or not at all. The spread has to be
conditional rather than optional: `...maybe` widens each field to `T |
undefined`, which is neither arm.
`processExitTriggersQueueRedelivery` is gone from the `World` interface —
#3385 stopped exiting on an exhausted replay budget, so there is nothing left
to tell it not to.
The book is unmoved across the rebase, which is the thing worth reporting:
33/6/6 mint-ordered, 39/0/0 append-only, 8 violations with `--no-fence`, 0
with both — and the same six ids red, not a swap that nets to the same count.
70 unit tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* feat(world-sim): a scenario for the unclaimed-payload delivery race
Reproduces vercel/workflow#3406 as a pair of scenarios. A hook payload
nobody reads registers an unarmed delivery barrier, and a step result is
allowed to skip it — otherwise the workflow stalls until the barrier
registry idles. The skip is transitive, so the step also skips an armed
`wait_completed` merely parked behind that payload. Both branches then
draw their next `step_created` id in the order the log does not record.
The replay invariant cannot see this: live and replay run the same code,
make the same mistake, and agree. What is checkable is the log
disagreeing with itself — `wait_completed` is committed first, yet the
step branch draws the earlier id. Verified red on the current runtime
and green with #3406's diff applied.
`claimed-payload-under-fork` is the control: same steps, same tempo,
same log order through the step result, one branch awaiting the hook so
the payload lands claimed. It passes in both cases.
Getting there needed one new primitive. The delivery loop is serial, so
a held inline step body stops the loop inside its own delivery and no
timer can fire — every interleaving where a `wait_completed` lands while
a step result is outstanding was unreachable. `sim.deliverQueued(select?)`
takes a message out of the pending set and delivers it from the script,
concurrently with the hold; `takeById` removes it first, so the loop can
never pick up the same message.
The book is now 41 scenarios: 34/7 with 6 violations mint-ordered,
40/1 with 0 violations append-only. The six violation reds and the
--no-fence 6 -> 8 / 0 -> 0 measurements are unchanged. The new red is
the first that stays red in both worlds, because no log position is
wrong in it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* fix(world-sim): address review on the simulation world
Fifteen findings from review, and one measurement that moved because of
them.
Correctness:
- tempo: `abort()` stored the raw `reject` rather than the wrapper that
also disposes the watch, so a point reached after teardown blocked a
world call on a promise nobody could release. The deadline is one-shot
and already spent by then, so that is a hang. Route abort through the
disposing path and make the action a no-op once aborted.
- world: `externalDepth` was a plain counter, so a step body writing
concurrently with a scripted `deliverHook` was attributed to the
script — invisible to any armed `runTo`, and `ext` in the trace.
Measured: one misattribution book-wide. Scope it with
`AsyncLocalStorage` instead.
- writers: `runToEventCommitted` matched a *rejected* create, so in
fence scenarios a script could wake believing a write was durable when
it had 412'd. Record `failed` on the observed point and require
`failed: false`.
- clock/ids: reject fractional `advanceBy`, and floor in `ulid()` — a
fractional millisecond silently minted `undefinedundefined…` ids that
failed much later at `z.ulid()`.
- streams: literal NUL bytes as key separators, `limit: 0` paging
forever, and a garbled cursor slicing the array away as `NaN`.
- scenario: register the handler under the exported
`WORKFLOW_QUEUE_PREFIX`.
The count guard, which is the substantive one. Since #3145 `@workflow/core`
sends `stateEventCount` on every replay-context create and the server's
count guard defaults on, so the sim's "no client sends the count" claim
was stale in six places. `countGuard` now follows the fence, the sim
prefers the runtime's own count over its reconstruction, and the two
scenarios whose subject is isolating the watermark half say
`countGuard: false` explicitly. The scoreboard does not move under the
production-shaped default, which is itself the answer to the review's
question.
`log.monotonic-order` could never fire: its only caller fed it the sorted
array. It now takes commit order, supplied only by a world that promises
the two agree — under a mint-ordered log an out-of-order commit is the
premise the scenario injected, not a defect.
Docs: DESIGN §5 gains the two ways the sim's guards are stronger than
production's (FIFO-vs-mint-order pruning, and a fence that is exact
in-process where production's is region-local and fails open); §9's
scoreboard is regenerated and its "no fix armed anywhere real" claim
corrected; §10 gains the parallel hook-resume path, which no sim delivery
takes. `step-vs-step-fork` and its twin now say who the withheld reader
is in production.
`unclaimed-payload-under-fork` is green after the rebase — #3406 fixed
the delivery-barrier ordering — so the book is back to six reds and
stays there. Counts refreshed everywhere they appear: 35/6/6
mint-ordered, 41/0/0 append-only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
* fix(world-sim): make the README's code samples type-check
`packages/docs-typecheck` globs `packages/*/README.md`, so the new README
went into the Docs Code Samples job and five of its `ts` blocks failed.
Four were genuine excerpts with an unbound `sim` or `spec`; one was a match-
object *sketch* containing a `…`, so not TypeScript at all — that one loses
its `ts` tag, joining the eleven other untagged blocks in the file.
The rest name what they use. That needed two `paths` entries in the checker:
`@workflow/world-sim` was unmapped, and an unresolved import is deliberately
tolerated there (`isExpectedMissingModule`), so every identifier in those
samples was `any` — they would have gone green while checking nothing.
Mapped, they are checked against the real declarations: seeding `id: 12345`,
`readyAtMsTYPO`, `dirsTYPO` and `runToEventCommittedTYPO` is caught against
`ScenarioSpec`, `PendingMessageView`, `SimBuildOptions` and `Writer`.
The lead teaser keeps its four unadorned lines and takes a `@skip-typecheck`
marker instead; the same calls appear in full, checked form further down.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>
---------
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
* test: add failing test for Date subclassing in workflow VM
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: ar_tama <arata.makoto@gmail.com>
* fix(core): preserve `new.target` in the deterministic `Date` override so `Date` subclasses work in workflow functions
The VM's `Date` override was a plain function, so `class X extends Date`
lost the subclass identity: `super()` returned a fresh plain `Date` that
became `this`, dropping the subclass's methods and fields. This silently
broke `Date` subclasses like `TZDate` from `@date-fns/tz`.
Using `class Date extends Date_` keeps `new.target` intact, and `extends`
already wires up the prototype chain and statics, so the manual
`prototype` assignment and `Object.setPrototypeOf` fix-ups are no longer
needed. Determinism is unchanged: zero-arg construction still returns the
fixed timestamp and `Date.now()` is still overridden.
Fixes#3371
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: ar_tama <arata.makoto@gmail.com>
* test: add failing test for calling `Date()` without `new`
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: ar_tama <arata.makoto@gmail.com>
* fix(core): keep `Date()` callable without `new`
Use a plain function that branches on `new.target` and constructs via
`Reflect.construct(Date_, args, new.target)` instead of a class: subclassing
still works (`new.target` is forwarded), and calling `Date()` without `new`
now matches the spec — arguments are ignored and the (fixed) time string is
returned, where the previous override returned a `Date` object.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: ar_tama <arata.makoto@gmail.com>
* chore: update changeset to match the final `Reflect.construct` implementation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: ar_tama <arata.makoto@gmail.com>
---------
Signed-off-by: ar_tama <arata.makoto@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
github-actions[bot]