mirror of
https://github.com/vercel/workflow.git
synced 2026-09-14 19:59:43 +08:00
8a872529fe
* docs: make /worlds the canonical home for World docs The world pages (Local/Postgres/Vercel) and Building a World were duplicated inside the v4 and v5 docs trees while /worlds/[id] rendered the v4 copy — hiding v5-only content like multi-region and leaving two diverging sources of truth. - Move world docs to an unversioned docs/content/worlds/ collection (based on the v5 copies, with inline 4.x callouts for factory naming and 5.x-only env vars), rendered at /worlds/* - Add /worlds/building-a-world; flatten the docs Deploying section to a single intro page and drop its Rocket icon - Point every link, frontmatter ref, and worlds-manifest docs field at /worlds/*; add redirects for the removed v5 and building-a-world URLs - Keep world docs on agent-facing surfaces: search, llms.txt, sitemap.md/.xml, and .md exports now serve the worlds collection - Extend the docs link linter to validate worlds pages (with heading anchors) and their outgoing links Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Pranay Prakash <pranay.gp@gmail.com> * docs: version the world docs like the docs trees (v4/v5 switcher) Instead of a single unversioned copy, world docs now follow the same versioning strategy as the docs pages: content/worlds/v4 is served at /worlds/* (current) and content/worlds/v5 at /v5/worlds/*, restoring the original per-version content. Each world detail page (and Building a World) renders the docs version switcher — the worlds listing page has no natural home for it, so it lives on the world pages themselves. - Render-time href rewriting on v5 pages now covers /worlds/... links (shared rewriteHrefForVersion helper, also used by the v5 docs and cookbook routes), and the markdown-export rewrite does the same - v5 world pages are noindexed with a canonical to /worlds/<id>; community worlds stay unversioned (/v5/worlds/<id> redirects) - /v5/docs/deploying/world/* redirects now land on /v5/worlds/*; /v5/worlds and /v5/worlds/compare redirect to the unversioned pages - Link linter models the versioned worlds URL spaces (v5 pages resolve /worlds hrefs against the v5 collection); sitemap.md and the .md export routes cover /v5/worlds/* Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Pranay Prakash <pranay.gp@gmail.com> * docs: fix v4 multi-region anchor and tighten version-prefix matching Address PR review: - The v4 Deploying page linked /worlds/vercel#multi-region, but the Multi-region section only exists on the v5 world page; use the explicit cross-version /v5/worlds/vercel#multi-region link (this was the Docs Links CI failure) - rewriteHrefForVersion now uses the boundary-checked hasPathPrefix (shared leaf module lib/geistdocs/path-prefix.ts, also used by source.ts) instead of bare startsWith - buildVersionUrl's shared-route fast path is segment-based rather than substring includes() Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Pranay Prakash <pranay.gp@gmail.com> --------- Signed-off-by: Pranay Prakash <pranay.gp@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
252 lines
9.4 KiB
Plaintext
252 lines
9.4 KiB
Plaintext
---
|
|
title: Building a World
|
|
description: Implement the World interface to run workflows on any custom infrastructure.
|
|
type: guide
|
|
summary: Build a custom World adapter to run workflows on your own infrastructure.
|
|
prerequisites:
|
|
- /docs/deploying
|
|
- /docs/foundations/workflows-and-steps
|
|
related:
|
|
- /worlds/local
|
|
- /worlds/postgres
|
|
- /worlds/vercel
|
|
---
|
|
|
|
A **World** is the abstraction that allows workflows to run on any infrastructure. It handles workflow storage, step execution queuing, and data streaming. This guide explains the World interface and how to implement your own.
|
|
|
|
<Callout>
|
|
Before building a custom World, check the [Worlds Ecosystem](/worlds) page — there may already be a community implementation for your infrastructure.
|
|
</Callout>
|
|
|
|
<Callout type="info">
|
|
**Reference Implementation:** The [Postgres World source code](https://github.com/vercel/workflow/tree/main/packages/world-postgres) is a production-ready example of how to implement the World interface with a database backend and graphile-worker for queuing.
|
|
</Callout>
|
|
|
|
## What is a World?
|
|
|
|
A World connects workflows to the infrastructure that powers them. The World interface abstracts three core responsibilities:
|
|
|
|
1. **Storage** — Persisting workflow runs, steps, hooks, and the event log
|
|
2. **Queue** — Enqueuing and processing workflow and step invocations
|
|
3. **Streamer** — Managing real-time data streams between workflows and clients
|
|
|
|
{/* @skip-typecheck - interface definition, not runnable code */}
|
|
```typescript
|
|
interface World extends Storage, Queue, Streamer {
|
|
start?(): Promise<void>;
|
|
close?(): Promise<void>;
|
|
getEncryptionKeyForRun?(run: WorkflowRun): Promise<Uint8Array | undefined>;
|
|
getEncryptionKeyForRun?(runId: string, context?: Record<string, unknown>): Promise<Uint8Array | undefined>;
|
|
}
|
|
```
|
|
|
|
The optional `start()` method initializes background tasks (for example, queue polling). The optional `close()` method releases resources like connection pools and listeners. The optional `getEncryptionKeyForRun()` method returns the AES-256 key used to encrypt data for a run; if it is not implemented, encryption is disabled.
|
|
|
|
## The Event Log Model
|
|
|
|
Workflow storage is built on an **append-only event log**. All state changes happen through events — you never modify runs, steps, or hooks directly. Instead, you create events that update the materialized state.
|
|
|
|
Events fall into three categories: run lifecycle events, step lifecycle events, and hook lifecycle events. See the [Event Sourcing](/docs/how-it-works/event-sourcing) documentation for a complete list of event types and their semantics.
|
|
|
|
## Storage Interface
|
|
|
|
The Storage interface provides read access to materialized entities and write access through events:
|
|
|
|
{/* @skip-typecheck - interface definition, not runnable code */}
|
|
```typescript
|
|
interface Storage {
|
|
runs: {
|
|
get(id: string, params?: GetWorkflowRunParams): Promise<WorkflowRun>;
|
|
list(params?: ListWorkflowRunsParams): Promise<PaginatedResponse<WorkflowRun>>;
|
|
};
|
|
|
|
steps: {
|
|
get(runId: string | undefined, stepId: string, params?: GetStepParams): Promise<Step>;
|
|
list(params: ListWorkflowRunStepsParams): Promise<PaginatedResponse<Step>>;
|
|
};
|
|
|
|
events: {
|
|
// Create a new workflow run (runId may be client-provided or null for server generation)
|
|
create(runId: string | null, data: RunCreatedEventRequest, params?: CreateEventParams): Promise<EventResult>;
|
|
|
|
// Create an event for an existing run
|
|
create(runId: string, data: CreateEventRequest, params?: CreateEventParams): Promise<EventResult>;
|
|
|
|
list(params: ListEventsParams): Promise<PaginatedResponse<Event>>;
|
|
listByCorrelationId(params: ListEventsByCorrelationIdParams): Promise<PaginatedResponse<Event>>;
|
|
};
|
|
|
|
hooks: {
|
|
get(hookId: string, params?: GetHookParams): Promise<Hook>;
|
|
getByToken(token: string, params?: GetHookParams): Promise<Hook>;
|
|
list(params: ListHooksParams): Promise<PaginatedResponse<Hook>>;
|
|
};
|
|
}
|
|
```
|
|
|
|
### Key Implementation Details
|
|
|
|
**Event Creation:** When `events.create()` is called, your implementation must:
|
|
1. Persist the event to the event log
|
|
2. Atomically update the affected entity (run, step, or hook)
|
|
3. Return both the created event and the updated entity
|
|
|
|
**Run Creation:** For `run_created` events, the `runId` parameter may be a client-provided string or `null`. When `null`, your World generates and returns a new `runId`.
|
|
|
|
**Hook Tokens:** Hook tokens must be unique. If a `hook_created` event conflicts with an existing token, return a `hook_conflict` event instead and include the active hook owner's run ID as `eventData.conflictingRunId`.
|
|
|
|
**Automatic Hook Disposal:** When a workflow reaches a terminal state (`completed`, `failed`, or `cancelled`), automatically dispose of all associated hooks to release tokens for reuse.
|
|
|
|
## Queue Interface
|
|
|
|
The Queue interface handles asynchronous execution of workflows and steps:
|
|
|
|
{/* @skip-typecheck - interface definition, not runnable code */}
|
|
```typescript
|
|
interface Queue {
|
|
getDeploymentId(): Promise<string>;
|
|
|
|
queue(
|
|
queueName: ValidQueueName,
|
|
message: QueuePayload,
|
|
opts?: QueueOptions
|
|
): Promise<{ messageId: MessageId }>;
|
|
|
|
createQueueHandler(
|
|
queueNamePrefix: QueuePrefix,
|
|
handler: (message: unknown, meta: { attempt: number; queueName: ValidQueueName; messageId: MessageId }) => Promise<void | { timeoutSeconds: number }>
|
|
): (req: Request) => Promise<Response>;
|
|
}
|
|
```
|
|
|
|
### Queue Names
|
|
|
|
Queue names follow a specific pattern:
|
|
- `__wkf_workflow_<name>` — For workflow invocations
|
|
- `__wkf_step_<name>` — For step invocations
|
|
|
|
### Message Payloads
|
|
|
|
Two types of messages flow through queues:
|
|
|
|
**Workflow Invocations:**
|
|
{/* @skip-typecheck - interface definition, not runnable code */}
|
|
```typescript
|
|
interface WorkflowInvokePayload {
|
|
runId: string;
|
|
traceCarrier?: Record<string, string>; // OpenTelemetry context
|
|
requestedAt?: Date;
|
|
}
|
|
```
|
|
|
|
**Step Invocations:**
|
|
{/* @skip-typecheck - interface definition, not runnable code */}
|
|
```typescript
|
|
interface StepInvokePayload {
|
|
workflowName: string;
|
|
workflowRunId: string;
|
|
workflowStartedAt: number;
|
|
stepId: string;
|
|
traceCarrier?: Record<string, string>;
|
|
requestedAt?: Date;
|
|
}
|
|
```
|
|
|
|
### Implementation Considerations
|
|
|
|
- Messages must be delivered at-least-once
|
|
- Support configurable retry policies
|
|
- Track attempt counts for observability
|
|
- Implement idempotency using the `idempotencyKey` option when provided
|
|
|
|
## Streamer Interface
|
|
|
|
The Streamer interface enables real-time data streaming:
|
|
|
|
{/* @skip-typecheck - interface definition, not runnable code */}
|
|
```typescript
|
|
interface Streamer {
|
|
streamFlushIntervalMs?: number;
|
|
|
|
streams: {
|
|
write(
|
|
runId: string,
|
|
name: string,
|
|
chunk: string | Uint8Array
|
|
): Promise<void>;
|
|
|
|
writeMulti?(
|
|
runId: string,
|
|
name: string,
|
|
chunks: (string | Uint8Array)[]
|
|
): Promise<void>;
|
|
|
|
close(runId: string, name: string): Promise<void>;
|
|
|
|
get(
|
|
runId: string,
|
|
name: string,
|
|
startIndex?: number
|
|
): Promise<ReadableStream<Uint8Array>>;
|
|
|
|
list(runId: string): Promise<string[]>;
|
|
|
|
/** Paginated snapshot of stream chunks. */
|
|
getChunks(
|
|
runId: string,
|
|
name: string,
|
|
options?: { limit?: number; cursor?: string }
|
|
): Promise<{
|
|
data: { index: number; data: Uint8Array }[];
|
|
cursor: string | null;
|
|
hasMore: boolean;
|
|
done: boolean;
|
|
}>;
|
|
|
|
/** Lightweight metadata: tail index and completion flag. */
|
|
getInfo(
|
|
runId: string,
|
|
name: string
|
|
): Promise<{ tailIndex: number; done: boolean }>;
|
|
};
|
|
}
|
|
```
|
|
|
|
Streams are identified by a combination of `runId` and `name`. Each workflow run can have multiple named streams.
|
|
`writeMulti()` is an optional optimization for batching multiple writes.
|
|
|
|
`getChunks` returns a paginated snapshot of currently available chunks (unlike `get` which returns a live `ReadableStream` that waits for new chunks). `getInfo` returns the tail index (last chunk index, 0-based, or `-1` when empty) and whether the stream is complete — useful for resolving negative `startIndex` values into absolute positions.
|
|
|
|
## Reference Implementations
|
|
|
|
Study these implementations for guidance:
|
|
|
|
- **[Local World](https://github.com/vercel/workflow/tree/main/packages/world-local)** — Filesystem-based, great for understanding the basics
|
|
- **[Postgres World](https://github.com/vercel/workflow/tree/main/packages/world-postgres)** — Database-backed with graphile-worker for queuing
|
|
|
|
## Testing Your World
|
|
|
|
Workflow SDK includes an E2E test suite that validates World implementations. Once your World is published to npm:
|
|
|
|
1. Add your world to [`worlds-manifest.json`](https://github.com/vercel/workflow/blob/main/worlds-manifest.json)
|
|
2. Open a PR to the Workflow repository
|
|
3. CI will automatically run the E2E test suite against your implementation
|
|
|
|
Your world will then appear on the [Worlds Ecosystem](/worlds) page with its compatibility status and performance benchmarks.
|
|
|
|
## Publishing Your World
|
|
|
|
1. **Package your World** — Export a default World instance from your package
|
|
2. **Publish to npm** — Publish your package to npm
|
|
3. **Add to the manifest** — Submit a PR adding your world to [`worlds-manifest.json`](https://github.com/vercel/workflow/blob/main/worlds-manifest.json)
|
|
4. **Document configuration** — Clearly document any required environment variables
|
|
|
|
```json
|
|
// worlds-manifest.json entry
|
|
{
|
|
"package": "your-world-package",
|
|
"repository": "https://github.com/you/your-world",
|
|
"docs": "https://github.com/you/your-world#readme"
|
|
}
|
|
```
|