Files
vercel__workflow/docs/content/worlds/v5/building-a-world.mdx

273 lines
12 KiB
Plaintext

---
title: Building a World
description: Implement the World interface to run workflows on any custom infrastructure.
type: guide
summary: Build a custom World adapter to run workflows on your own infrastructure.
prerequisites:
- /docs/deploying
- /docs/foundations/workflows-and-steps
related:
- /worlds/local
- /worlds/postgres
- /worlds/vercel
---
A **World** is the abstraction that allows workflows to run on any infrastructure. It handles workflow storage, step execution queuing, and data streaming. This guide explains the World interface and how to implement your own.
<Callout>
Before building a custom World, check the [Worlds Ecosystem](/worlds) page — there may already be a community implementation for your infrastructure.
</Callout>
<Callout type="info">
**Reference Implementation:** The [Postgres World source code](https://github.com/vercel/workflow/tree/main/packages/world-postgres) is a production-ready example of how to implement the World interface with a database backend and graphile-worker for queuing.
</Callout>
## What is a World?
A World connects workflows to the infrastructure that powers them. The World interface abstracts three core responsibilities:
1. **Storage** — Persisting workflow runs, steps, hooks, and the event log
2. **Queue** — Enqueuing and processing workflow and step invocations
3. **Streamer** — Managing real-time data streams between workflows and clients
{/* @skip-typecheck - interface definition, not runnable code */}
```typescript
interface WorldCapabilities {
hookRetention?: {
active: boolean;
};
}
interface World extends Storage, Queue, Streamer {
capabilities?: WorldCapabilities;
start?(): Promise<void>;
close?(): Promise<void>;
getEncryptionKeyForRun?(run: WorkflowRun): Promise<Uint8Array | undefined>;
getEncryptionKeyForRun?(runId: string, context?: Record<string, unknown>): Promise<Uint8Array | undefined>;
}
```
The optional `capabilities` object advertises additional behavior. Set `hookRetention.active` to `true` only when the World implements Hook token retention. The optional `start()` method initializes background tasks (for example, queue polling). The optional `close()` method releases resources like connection pools and listeners. The optional `getEncryptionKeyForRun()` method returns the AES-256 key used to encrypt data for a run; if it is not implemented, encryption is disabled.
## The Event Log Model
Workflow storage is built on an **append-only event log**. All state changes happen through events — you never modify runs, steps, or hooks directly. Instead, you create events that update the materialized state.
Events fall into three categories: run lifecycle events, step lifecycle events, and hook lifecycle events. See the [Event Sourcing](/docs/how-it-works/event-sourcing) documentation for a complete list of event types and their semantics.
## Storage Interface
The Storage interface provides read access to materialized entities and write access through events:
{/* @skip-typecheck - interface definition, not runnable code */}
```typescript
interface Storage {
runs: {
get(id: string, params?: GetWorkflowRunParams): Promise<WorkflowRun>;
list(params?: ListWorkflowRunsParams): Promise<PaginatedResponse<WorkflowRun>>;
};
steps: {
get(runId: string | undefined, stepId: string, params?: GetStepParams): Promise<Step>;
list(params: ListWorkflowRunStepsParams): Promise<PaginatedResponse<Step>>;
};
events: {
// Create a new workflow run (runId may be client-provided or null for server generation)
create(runId: string | null, data: RunCreatedEventRequest, params?: CreateEventParams): Promise<EventResult>;
// Create an event for an existing run
create(runId: string, data: CreateEventRequest, params?: CreateEventParams): Promise<EventResult>;
list(params: ListEventsParams): Promise<PaginatedResponse<Event>>;
listByCorrelationId(params: ListEventsByCorrelationIdParams): Promise<PaginatedResponse<Event>>;
};
hooks: {
get(hookId: string, params?: GetHookParams): Promise<Hook>;
getByToken(token: string, params?: GetHookParams): Promise<Hook>;
list(params: ListHooksParams): Promise<PaginatedResponse<Hook>>;
};
}
```
### Key Implementation Details
**Event Creation:** When `events.create()` is called, your implementation must:
1. Persist the event to the event log
2. Atomically update the affected entity (run, step, or hook)
3. Return both the created event and the updated entity
**Run Creation:** For `run_created` events, the `runId` parameter may be a client-provided string or `null`. When `null`, your World generates and returns a new `runId`.
**Hook Tokens:** Hook tokens must be unique. If a `hook_created` event conflicts with an active Hook or a token still reserved after its run ended, return a `hook_conflict` event and include the owner's run ID as `eventData.conflictingRunId`.
Keep the owning Run available for at least as long as its token remains unavailable, because conflicts return that Run.
**Automatic Hook Cleanup:** When a run ends, remove its live Hooks. Make each token available unless its `tokenRetentionUntil` is still in the future. A `hook_disposed` event always makes the token available immediately.
### Optional: The Event Creation Precondition Guard
A replay writes events derived from the event log it loaded, so a write made from an event log that no longer matches the store can commit events no correct replay would produce. To let a World fence those writes, `events.create()` params may carry a description of the snapshot the caller replayed from:
- `stateUpdatedAt` — the timestamp encoded in the latest loaded event's ID.
- `stateEventCount` — how many events the caller loaded. Only sent together with `stateUpdatedAt`; ignore a count that arrives without one.
- `stateCursor` — the caller's event-log cursor. Advisory, and only meaningful on the reject path (see below).
Enforcing the guard means rejecting the creation with a `PreconditionFailedError` when either holds:
1. An event was recorded after `stateUpdatedAt` that the caller could not have loaded.
2. More events were recorded at or before `stateUpdatedAt` than `stateEventCount` — the caller's snapshot is missing one.
Two rules make this safe:
- **Compare at or below `stateUpdatedAt`, not strictly below.** Events routinely share a millisecond with the caller's latest event, and a strict comparison misses exactly those.
- **Every uncertainty must allow the write.** If your record of a run's events is incomplete, expired, or cannot answer the question, accept the creation. A rejection must always mean a real discrepancy, because the runtime responds to one by discarding a replay.
A rejection may optionally carry the events the caller was missing, as `{ events, cursor }` on the error's `details`. Only include them when you can prove the set is complete — that those events fully account for the discrepancy and are not truncated — and that every one of them belongs to the run being written. The runtime merges them straight into the replay's event log, so anything else there is worse than no delta at all. Otherwise omit them, and the runtime performs a full reload instead.
A World that enforces the guard should declare `capabilities.preconditionGuard`, which the runtime also reads to keep event-log delta optimizations enabled. A World that ignores these params must not declare it.
## Queue Interface
The Queue interface handles asynchronous workflow execution, including queued step work:
{/* @skip-typecheck - interface definition, not runnable code */}
```typescript
interface Queue {
getDeploymentId(): Promise<string>;
queue(
queueName: ValidQueueName,
message: QueuePayload,
opts?: QueueOptions
): Promise<{ messageId: MessageId | null }>;
createQueueHandler(
queueNamePrefix: QueuePrefix,
handler: (message: unknown, meta: { attempt: number; queueName: ValidQueueName; messageId: MessageId }) => Promise<void | { timeoutSeconds: number }>
): (req: Request) => Promise<Response>;
}
```
### Queue Names
Queue names follow a specific pattern:
- `__wkf_workflow_<name>` — For workflow orchestration and queued step invocations
### Message Payloads
Workflow orchestration and step execution use the same payload. `stepId` and `stepName` identify a queued step invocation:
**Workflow Invocations:**
{/* @skip-typecheck - interface definition, not runnable code */}
```typescript
interface WorkflowInvokePayload {
runId: string;
stepId?: string;
stepName?: string;
traceCarrier?: Record<string, string>; // OpenTelemetry context
requestedAt?: Date;
}
```
The SDK also sends an internal `HealthCheckPayload` through the same workflow queue.
### Implementation Considerations
- Messages must be delivered at-least-once
- Support configurable retry policies
- Track attempt counts for observability
- Implement idempotency using the `idempotencyKey` option when provided
## Streamer Interface
The Streamer interface enables real-time data streaming:
{/* @skip-typecheck - interface definition, not runnable code */}
```typescript
interface Streamer {
streamFlushIntervalMs?: number;
streams: {
write(
runId: string,
name: string,
chunk: string | Uint8Array
): Promise<void>;
writeMulti?(
runId: string,
name: string,
chunks: (string | Uint8Array)[]
): Promise<void>;
close(runId: string, name: string): Promise<void>;
get(
runId: string,
name: string,
startIndex?: number
): Promise<ReadableStream<Uint8Array>>;
list(runId: string): Promise<string[]>;
/** Paginated snapshot of stream chunks. */
getChunks(
runId: string,
name: string,
options?: { limit?: number; cursor?: string }
): Promise<{
data: { index: number; data: Uint8Array }[];
cursor: string | null;
hasMore: boolean;
done: boolean;
}>;
/** Lightweight metadata: tail index and completion flag. */
getInfo(
runId: string,
name: string
): Promise<{ tailIndex: number; done: boolean }>;
};
}
```
Streams are identified by a combination of `runId` and `name`. Each workflow run can have multiple named streams.
`writeMulti()` is an optional optimization for batching multiple writes.
`getChunks` returns a paginated snapshot of currently available chunks (unlike `get` which returns a live `ReadableStream` that waits for new chunks). `getInfo` returns the tail index (last chunk index, 0-based, or `-1` when empty) and whether the stream is complete — useful for resolving negative `startIndex` values into absolute positions.
## Reference Implementations
Study these implementations for guidance:
- **[Local World](https://github.com/vercel/workflow/tree/main/packages/world-local)** — Filesystem-based, great for understanding the basics
- **[Postgres World](https://github.com/vercel/workflow/tree/main/packages/world-postgres)** — Database-backed with graphile-worker for queuing
## Testing Your World
Workflow SDK includes an E2E test suite that validates World implementations. Once your World is published to npm:
1. Add your world to [`worlds-manifest.json`](https://github.com/vercel/workflow/blob/main/worlds-manifest.json)
2. Open a PR to the Workflow repository
3. CI will automatically run the E2E test suite against your implementation
Your world will then appear on the [Worlds Ecosystem](/worlds) page with its compatibility status and performance benchmarks.
## Publishing Your World
1. **Package your World** — Export a default World instance from your package
2. **Publish to npm** — Publish your package to npm
3. **Add to the manifest** — Submit a PR adding your world to [`worlds-manifest.json`](https://github.com/vercel/workflow/blob/main/worlds-manifest.json)
4. **Document configuration** — Clearly document any required environment variables
```json
// worlds-manifest.json entry
{
"package": "your-world-package",
"repository": "https://github.com/you/your-world",
"docs": "https://github.com/you/your-world#readme"
}
```