vercel__workflow/workbench/sim-world

sim-world workbench

Worked examples for @workflow/world-sim: workflows written to make ordering visible, and a book of scenarios that pin down exactly when external input arrives.

pnpm sim                             # play every scenario, print every event stream
pnpm sim hook                        # only scenarios whose id or name contains "hook"
pnpm sim in-flight-after-decision    # one scenario, by id

Exits non-zero if any scenario misses an expectation or trips a consistency check, so it doubles as the package's integration test.

This README is about adding a scenario. The API a script is written in — writers, advances, withholdings — is the API reference; how the simulator works and how to change it is the rest of packages/world-sim/README.md, and the internals are DESIGN.md.

Adding a scenario

One scenario, one file in scenarios/, named after its id. Copy the file next door and change what differs — that is the whole workflow, and the book is split this way so that it is.

// scenarios/hook-at-step-started.ts
import type { ScenarioSpec } from '@workflow/world-sim';

export const scenario: ScenarioSpec = {
  id: 'hook-at-step-started',
  name: 'hook arrives inside the step_started commit',
  description: 'The hook payload is written after step_started is durable …',
  workflow: 'approvalWorkflow',
  input: ['doc-1'],
  script: async (sim) => {
    const wf = sim.writer.orchestrator();
    await wf.runToEventCommitted('step_started', 'reserveInventory');
    await sim.deliverHook('approval:doc-1', { approved: true, reviewer: 'ada' });
    await wf.release();
  },
  expect: {
    status: 'completed',
    output: { status: 'settled:reserved:doc-1', reviewer: 'ada' },
  },
};

Then import it in scenarios/index.ts and place it in the scenarios array. Order is the only thing that file decides: simplest first, and each pair of near-identical scenarios adjacent, so a reader meets a distinction right after the thing it is a distinction from. Put yours next to the one it is a variation of.

The id is stable and hyphenated; it is what a commit message or a bug report cites and what the command-line filter matches first. The name beside it is prose and free to be reworded.

The workflow named by workflow must be exported from workflows/index.ts — all of them live in that one file because a scenario is read together with the branch it steers. Prefer reusing one; a new workflow is only worth it when the shape you need to steer does not exist yet.

The shape of a script

Every script is the same three steps: hold a writer at a named point, act while it is held, let it go.

const wf = sim.writer.orchestrator();
await wf.runToEventCommitted('step_started', 'reserveInventory');
await sim.deliverHook('approval:doc-1', { approved: true });
await wf.release();

Because the writer is held inside the world call, everything the script does in between lands in the log before that writer is resumed. That is the entire point of the writer API: the interleaving is stated, not raced for.

Every advance and everything a script can do while one is held is in the API reference. Four things from it come up on the first scenario you write:

• Name the right writer. step_started, wait_created, hook_created and the run's own decisions belong to sim.writer.orchestrator(). A step's step_completed / step_failed belongs to that step body — sim.writer.step('reserveInventory'), or sim.writer.anyStep() for whichever gets there first. Naming the wrong one is a wait that times out, so the failure is loud, but knowing the rule saves the trip.
• Pick the right advance. runToEventCommitted is what most scenarios want. Reach for runToEventProduced when the point is that a write committed during the hold sorts ahead of the held event, and for sim.beginHookDelivery when it has to sort behind one.
• Calling an advance starts watching; awaiting it waits for the hold. To hold two writers at once, call both, then await both.
• runTo is level-triggered. Asking for a point that has already gone by is an error, not a wait that never ends.
• Start B's watch before releasing A. A released writer can reach the next point within the same turn, and a watch started afterwards has missed it.

And one thing the advances cannot do at all: a held writer stops the scheduler, so virtual time stops with it and no timer can fire while anything is held. If the interleaving you need is a timer firing while a step result is outstanding, no arrangement of holds will reach it. sim.deliverQueued is the way out — it delivers a queued message from inside the script, concurrently with the hold. See the API reference for the shape, and unclaimed-payload-under-fork.ts for it in use.

What to assert, and what not to

Two different instruments, for two different things:

• sim.check asserts a sentence about the middle of the run — "the live pass decided the fork without the hook". It is the only way to pin down a fact that exists at one instant and is gone by the end.
• expect asserts the run's outcome: status, and output when the output is the point.

The runner checks the invariant that a run's log must replay back into that run, so expectations should describe durable outcomes rather than incidental intermediate ordering.

There is deliberately no way to expect a violation. A scenario states the outcome the run should have reached and stays red until the runtime gets there.

Per-scenario world flags

preconditionGuard and countGuard on the spec control the guards for that scenario. The command-line fence flags below override them for a whole run.

Flags

flag	effect
--verbose	include queue deliveries in the trace
--color / --no-color	force colour on through a pipe / off. Default: on for a terminal, off otherwise, so pnpm sim > out.txt is already diffable
--fence / --no-fence	force the optimistic-concurrency fence on or off for every scenario
--report-only	print every failure, exit 0 anyway
--summary-file <path>	one collapsed <details> — the count on the visible line, the table behind it — for a PR comment or $GITHUB_STEP_SUMMARY
--detail-file <path>	the full trace, colour forced off, as a CI artifact
--title <text>	heading for the summary file

--no-fence turns the fence off everywhere, asking whether anything relies on it. It is a diagnostic — read the violation count, not the pass count, because a scenario whose whole point is that the guard fired asserts exactly that and fails by design when you disarm it (in-flight-before-decision-counted is the one that does this today).

In CI

.github/workflows/world-sim.yml plays the book on every pull request and posts its summary as one sticky comment:

## Sim World

Simulated world deterministic testing for races. [Traces](…)

▸ 🟢 world-sim scenario book — 0 fail of 41 total

It never blocks a merge: the simulation is an informational measurement, so the published count is the thing to look at rather than the check mark.

That is also why pnpm test in this package is --report-only while pnpm sim stays strict — a recursive pnpm -r test should not go red for the known reds, but someone running the book deliberately wants the exit code.

Reading the output

Events in the printed stream are referred to by log position — #12 is the twelfth event in the durable log, @7 the resource created at position 7 — and the trace prints in commit order. See packages/world-sim/README.md.

What the scenarios show

The first three run the same workflow with the same input and differ only in when the approval hook is delivered — inside the step_started commit, inside the step_completed commit, or inside the hook_created commit. Same result, three different event logs. Diff them against each other; that difference is what a real deployment leaves to chance.

Two of them ("writers: …") make the underlying claim explicit: the two step bodies of a single delivery are separately steerable writers to one log, and holding one does not freeze the other.

The rest cover the properties that make scenarios usable as tests: a hook racing a deadline (both branches, on demand), a thirty-day sleep that costs microseconds, a step that retries twice, cancellation landing mid-step, and a hook that never arrives — which is reported as a stall naming the undelivered token rather than hanging the run.

Red scenarios

Some scenarios fail, on purpose and by construction, and pnpm sim exits non-zero because of them. They are reproductions of corruptions the runtime can still produce: each states the outcome the run should have reached — the branch its own durable log implies — and fails until the runtime gets there. The failure line names both sides, e.g. expected "afterSlow:doc-26", got "afterFast:doc-26".

So a red is an open bug, not a recorded observation, and it goes green when the bug is fixed rather than when the bug is seen once more. Which means the count is the thing to watch, in either direction: one more is a regression, one fewer means a scenario is ready to retire.

Run the book to see the current set — this file deliberately does not keep a list, because a list here is a second copy of something the book already says exactly, and it is the copy that goes stale. The analysis that is not re-derivable from a run — which guard closes which shape, which of those guards is armed in production and which is dark — is in DESIGN.md.

Requirements

run.ts and the scenario book are TypeScript executed directly by Node's type stripping, which needs Node >= 22.18 (the version pinned in .node-version). Every workflow under test is compiled by the normal SDK build pipeline, exactly as a deployment would compile it.