mirror of
https://github.com/vercel/workflow.git
synced 2026-09-14 19:59:43 +08:00
99f4aeb03d
* feat(world-postgres): retain hook tokens after runs end
* refactor(world-postgres): reuse terminal run statuses
* docs: note Postgres Hook retention support
* fix(world-postgres): expose hook retention deadline
* Fix: Exhaustive `Record<AttributeKey, ...>` in `attribute-panel.tsx` is missing the `tokenRetentionUntil` key that was added to `HookSchema`, causing TS2741 and breaking every Vercel build.
This commit fixes the issue reported at packages/web-shared/src/components/sidebar/attribute-panel.tsx:426
## Bug
Commit `ad58321` added `tokenRetentionUntil: z.coerce.date().optional()` to `HookSchema` in `packages/world/src/hooks.ts:106`. This adds `tokenRetentionUntil` to the inferred `Hook` type.
In `packages/web-shared/src/components/sidebar/attribute-panel.tsx`, `AttributeKey` is a union that includes `keyof Hook`, so `tokenRetentionUntil` becomes a required member of the **exhaustive** `Record<AttributeKey, (value: unknown, context?: DisplayContext) => ...>` object literal `attributeToDisplayFn` (starting at line ~426).
Because the literal had no `tokenRetentionUntil` entry, `tsc` fails:
```
src/components/sidebar/attribute-panel.tsx(426,7): error TS2741:
Property 'tokenRetentionUntil' is missing in type '{ ... }' but required in type
'Record<AttributeKey, (value: unknown, context?: DisplayContext | undefined) => ReactNode>'.
```
This breaks `@workflow/web-shared#build` and therefore every Vercel deployment (17 failing deployments observed, all with this identical error).
## Fix
Added a `tokenRetentionUntil` entry to `attributeToDisplayFn`, placed alongside the other Hook date fields (`lastReceivedAt`, `disposedAt`):
```ts
tokenRetentionUntil: timestampWithTooltipOrNull,
```
`tokenRetentionUntil` is a `Date` field, and `timestampWithTooltipOrNull` (defined at line 402) is the display helper used by all the other surfaced date fields (`createdAt`, `startedAt`, `completedAt`, `retryAfter`, `resumeAt`, `occurredAt`). Given the intent of `ad58321` was to expose the hook retention deadline, surfacing it as a tooltip-annotated timestamp is the consistent choice.
Only `attributeToDisplayFn` is a fully exhaustive `Record<AttributeKey, ...>`; the other maps are `Partial<...>` / `Set`, so no other edits are required.
## Verification
`node_modules` are not installed in this sandbox, so `tsc` could not be executed directly. Verified structurally instead: the newly added `tokenRetentionUntil` entry (line 449) references `timestampWithTooltipOrNull`, which is defined in-file at line 402 and already used by the sibling date entries, so the fix satisfies the missing-key requirement without introducing new type errors.
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: VaguelySerious <mittgfu@gmail.com>
* docs(world-postgres): clarify expired hook rows
* feat(world-postgres): enforce Hook retention limit
* fix(world): remove duplicate Hook retention field
* fix(web-shared): remove duplicate retention renderer
* test(world): remove redundant retention coercion case
---------
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: VaguelySerious <mittgfu@gmail.com>
307 lines
11 KiB
Plaintext
307 lines
11 KiB
Plaintext
---
|
|
title: Postgres World
|
|
description: Production-ready, self-hosted world using PostgreSQL for storage and graphile-worker for job processing.
|
|
type: integration
|
|
summary: Deploy workflows to your own infrastructure using PostgreSQL and graphile-worker.
|
|
prerequisites:
|
|
- /docs/deploying
|
|
related:
|
|
- /worlds/local
|
|
- /worlds/vercel
|
|
---
|
|
|
|
The Postgres World is a production-ready backend for self-hosted deployments. It uses PostgreSQL for durable storage and [graphile-worker](https://github.com/graphile/worker) for reliable job processing.
|
|
|
|
Use the Postgres World when you need to deploy workflows on your own infrastructure outside of Vercel - such as a Docker container, Kubernetes cluster, or any cloud that supports long-running servers.
|
|
|
|
## Installation
|
|
|
|
Install the Postgres World package in your workflow project:
|
|
|
|
```package-install
|
|
@workflow/world-postgres
|
|
```
|
|
|
|
<Callout type="info">
|
|
Use the same release channel for `workflow` and `@workflow/world-postgres`. If
|
|
your app uses a beta or other prerelease Workflow version, install the matching
|
|
prerelease Postgres World package, such as
|
|
`npm install @workflow/world-postgres@beta`. Mismatched versions fail before
|
|
starting a run with an error that says the runtime requires a World with a
|
|
matching spec version.
|
|
</Callout>
|
|
|
|
Configure the required environment variables to use the world and point it to your PostgreSQL database:
|
|
|
|
```bash title=".env"
|
|
WORKFLOW_TARGET_WORLD="@workflow/world-postgres"
|
|
WORKFLOW_POSTGRES_URL="postgres://user:password@host:5432/database"
|
|
```
|
|
|
|
Run the migration script to create the necessary tables in your database. Ensure `WORKFLOW_POSTGRES_URL` or `DATABASE_URL` is set when running this command:
|
|
|
|
<Tabs items={["npm", "pnpm", "Yarn", "Bun"]}>
|
|
|
|
<Tab value="npm">
|
|
|
|
```bash
|
|
npx --package=@workflow/world-postgres bootstrap
|
|
```
|
|
|
|
</Tab>
|
|
|
|
<Tab value="pnpm">
|
|
|
|
```bash
|
|
pnpm dlx --package @workflow/world-postgres bootstrap
|
|
```
|
|
|
|
</Tab>
|
|
|
|
<Tab value="Yarn">
|
|
|
|
```bash
|
|
yarn dlx --package @workflow/world-postgres bootstrap
|
|
```
|
|
|
|
</Tab>
|
|
|
|
<Tab value="Bun">
|
|
|
|
```bash
|
|
bunx --package @workflow/world-postgres bootstrap
|
|
```
|
|
|
|
</Tab>
|
|
|
|
</Tabs>
|
|
|
|
<Callout type="info">
|
|
The migration is idempotent and can safely be run as a post-deployment lifecycle script.
|
|
</Callout>
|
|
|
|
## Starting the World
|
|
|
|
To subscribe to the graphile-worker queue, your workflow app needs to start the world on server start. Here are examples for a few frameworks:
|
|
|
|
<Tabs items={["Next.js", "SvelteKit", "Nitro"]}>
|
|
|
|
<Tab value="Next.js">
|
|
|
|
Create an `instrumentation.ts` file in your project root:
|
|
|
|
```ts title="instrumentation.ts" lineNumbers
|
|
export async function register() {
|
|
if (process.env.NEXT_RUNTIME !== "edge") {
|
|
const { getWorld } = await import("workflow/runtime");
|
|
const world = await getWorld();
|
|
await world.start?.();
|
|
}
|
|
}
|
|
```
|
|
|
|
<Callout type="info">
|
|
Learn more about [Next.js Instrumentation](https://nextjs.org/docs/app/guides/instrumentation).
|
|
</Callout>
|
|
|
|
</Tab>
|
|
|
|
<Tab value="SvelteKit">
|
|
|
|
Create a `src/hooks.server.ts` file:
|
|
|
|
```ts title="src/hooks.server.ts" lineNumbers
|
|
import type { ServerInit } from "@sveltejs/kit";
|
|
|
|
export const init: ServerInit = async () => {
|
|
const { getWorld } = await import("workflow/runtime");
|
|
const world = await getWorld();
|
|
await world.start?.();
|
|
};
|
|
```
|
|
|
|
<Callout type="info">
|
|
Learn more about [SvelteKit Hooks](https://svelte.dev/docs/kit/hooks).
|
|
</Callout>
|
|
|
|
</Tab>
|
|
|
|
<Tab value="Nitro">
|
|
|
|
Create a plugin to start the world on server initialization:
|
|
|
|
```ts title="plugins/start-pg-world.ts" lineNumbers
|
|
import { defineNitroPlugin } from "nitro/~internal/runtime/plugin";
|
|
|
|
export default defineNitroPlugin(async () => {
|
|
const { getWorld } = await import("workflow/runtime");
|
|
const world = await getWorld();
|
|
await world.start?.();
|
|
});
|
|
```
|
|
|
|
Register the plugin in your config:
|
|
|
|
```ts title="nitro.config.ts"
|
|
import { defineNitroConfig } from "nitropack";
|
|
|
|
export default defineNitroConfig({
|
|
modules: ["workflow/nitro"],
|
|
plugins: ["plugins/start-pg-world.ts"],
|
|
});
|
|
```
|
|
|
|
<Callout type="info">
|
|
Learn more about [Nitro Plugins](https://v3.nitro.build/docs/plugins).
|
|
</Callout>
|
|
|
|
</Tab>
|
|
|
|
</Tabs>
|
|
|
|
<Callout type="info">
|
|
The Postgres World requires a long-lived worker process that polls the database for jobs. This does not work on serverless environments. For Vercel deployments, use the [Vercel World](/worlds/vercel) instead.
|
|
</Callout>
|
|
|
|
## Observability
|
|
|
|
Use the `workflow` CLI to inspect workflows stored in PostgreSQL:
|
|
|
|
```bash
|
|
# Set your database URL
|
|
export WORKFLOW_POSTGRES_URL="postgres://user:password@host:5432/database"
|
|
|
|
# List workflow runs
|
|
npx workflow inspect runs --backend @workflow/world-postgres
|
|
|
|
# Launch the web UI
|
|
npx workflow web --backend @workflow/world-postgres
|
|
```
|
|
|
|
If `WORKFLOW_POSTGRES_URL` is not set, the CLI defaults to `postgres://world:world@localhost:5432/world`.
|
|
|
|
Learn more in the [Observability](/docs/observability) documentation.
|
|
|
|
## Testing & Compatibility
|
|
|
|
<WorldTestingPerformance worldId="postgres" />
|
|
|
|
## Configuration
|
|
|
|
All configuration options can be set via environment variables or programmatically via `createWorld()`.
|
|
|
|
### `WORKFLOW_POSTGRES_URL`
|
|
|
|
PostgreSQL connection string used by the runtime World.
|
|
|
|
Precedence: `WORKFLOW_POSTGRES_URL` > `DATABASE_URL` > `postgres://world:world@localhost:5432/world`
|
|
|
|
The `bootstrap` migration command uses the same precedence.
|
|
|
|
### `WORKFLOW_POSTGRES_JOB_PREFIX`
|
|
|
|
Prefix for graphile-worker queue job names. Useful when sharing a database between multiple applications. Default: `workflow_`
|
|
|
|
### `WORKFLOW_POSTGRES_WORKER_CONCURRENCY`
|
|
|
|
Number of concurrent workers polling for jobs. Default: `50`.
|
|
|
|
This value also bounds how many parent→child workflow polls can be in flight simultaneously. Every `await childRun.returnValue` inside a workflow holds a worker slot until the child run terminates — if you expect recursive or highly-fanned-out parent/child workflows, raise this ceiling above the peak number of concurrent polls. With the default of 50, the included `fibonacciWorkflow` e2e test (fib(6), ~24 concurrent polls at peak) passes; deeper recursion or larger fanouts need a correspondingly larger setting.
|
|
|
|
### `WORKFLOW_POSTGRES_MAX_POOL_SIZE`
|
|
|
|
Maximum size of the internal `pg.Pool` used when `createWorld()` constructs the pool. Default: the `pg` default (`10`).
|
|
|
|
For higher worker concurrency, Graphile Worker recommends setting `maxPoolSize` to `10` or `queueConcurrency + 2`, whichever is larger.
|
|
|
|
### `WORKFLOW_POSTGRES_APPLICATION_MANAGED_SHUTDOWN`
|
|
|
|
Set to `1` when the application or framework coordinates shutdown and awaits `world.close()` before closing its workflow HTTP server and any caller-owned pool. Default: unset (`false`).
|
|
|
|
### `WORKFLOW_POSTGRES_HOOK_RETENTION_LIMIT_DAYS`
|
|
|
|
Maximum [`experimental_minRetention`](/docs/api-reference/workflow/create-hook#keep-a-token-unavailable-after-the-run-ends) accepted by the Postgres World, in days. Default: `30`. Set this to the same limit as your production World so oversized values fail during development.
|
|
|
|
### `WORKFLOW_QUEUE_NAMESPACE`
|
|
|
|
Queue topic namespace shared by build output and the Postgres World. Default: unset.
|
|
|
|
For example, `custom` changes the queue topic prefix from `__wkf_workflow_` to `__custom_wkf_workflow_`. The value must be lowercase alphanumeric and start with a letter.
|
|
|
|
### `WORKFLOW_STREAM_FLUSH_INTERVAL_MS`
|
|
|
|
Group-commit window, in milliseconds, for the leading chunk of an idle stream. Default: `0` (dispatch immediately). A positive value holds the first chunk up to that long to collect a group — trading first-chunk latency for fewer requests. Chunks arriving while a request is in flight always coalesce into the next group regardless.
|
|
|
|
### Programmatic configuration
|
|
|
|
{/*@skip-typecheck: incomplete code sample*/}
|
|
|
|
```typescript title="my-world.ts" lineNumbers
|
|
import { createWorld } from "@workflow/world-postgres";
|
|
|
|
export default createWorld({
|
|
connectionString:
|
|
process.env.WORKFLOW_POSTGRES_URL ?? process.env.DATABASE_URL!,
|
|
jobPrefix: "myapp_",
|
|
namespace: "myapp",
|
|
queueConcurrency: 50,
|
|
maxPoolSize: 52, // overrides WORKFLOW_POSTGRES_MAX_POOL_SIZE
|
|
streamFlushIntervalMs: 10,
|
|
});
|
|
```
|
|
|
|
Options passed to `createWorld()` take precedence over the environment variables above. Export the World from a module and point `WORKFLOW_TARGET_WORLD` at that module:
|
|
|
|
You can also pass an existing `pg.Pool` as `pool` instead of a connection string.
|
|
|
|
```bash title=".env"
|
|
WORKFLOW_TARGET_WORLD="./my-world.ts"
|
|
```
|
|
|
|
### Application-managed shutdown
|
|
|
|
Graphile Worker responds automatically when the application is asked to shut down. If your application or framework already coordinates a broader shutdown sequence, set `WORKFLOW_POSTGRES_APPLICATION_MANAGED_SHUTDOWN=1` when selecting the package directly with `WORKFLOW_TARGET_WORLD`, or set `applicationManagedShutdown: true` in a programmatic World. Use the application's normal shutdown hook. This prevents Graphile Worker's default handler from terminating the process as soon as its queue stops. The hook must handle cleanup errors and await resources in this order:
|
|
|
|
1. `world.close()`
|
|
2. The workflow HTTP server
|
|
3. Any caller-owned `pg.Pool`
|
|
|
|
Closing the world stops new queue claims and waits for active jobs. After Graphile Worker's grace period, a pending workflow HTTP request is aborted. Graphile Worker unlocks that same row through its normal failure handling. The already-claimed delivery consumes a Graphile attempt and is retried only if its attempt budget remains; a one-attempt or final-attempt job is not retried. The shutdown handler does not insert a successor row. Because a client abort does not prove the server handler stopped, workflow and step handlers must tolerate at-least-once execution.
|
|
|
|
## How It Works
|
|
|
|
The Postgres World uses PostgreSQL as a durable backend:
|
|
|
|
- **Storage** - Workflow runs, events, steps, and hooks are stored in PostgreSQL tables
|
|
- **Job Queue** - [graphile-worker](https://github.com/graphile/worker) handles reliable job processing with retries
|
|
- **Streaming** - PostgreSQL NOTIFY/LISTEN enables real-time event distribution
|
|
|
|
This architecture ensures workflows survive application restarts with all state reliably persisted. For implementation details, see the [source code](https://github.com/vercel/workflow/tree/main/packages/world-postgres).
|
|
|
|
## Deployment
|
|
|
|
Deploy your application to any cloud that supports long-running servers:
|
|
|
|
- Docker containers
|
|
- Kubernetes clusters
|
|
- Virtual machines
|
|
- Platform-as-a-Service providers (Railway, Render, Fly.io, etc.)
|
|
|
|
Ensure your deployment has:
|
|
|
|
1. Network access to your PostgreSQL database
|
|
2. Environment variables configured correctly
|
|
3. The `start()` function called on server initialization
|
|
|
|
<Callout type="info">
|
|
The Postgres World is not compatible with Vercel deployments. On Vercel, workflows automatically use the [Vercel World](/worlds/vercel) with zero configuration.
|
|
</Callout>
|
|
|
|
## Limitations
|
|
|
|
- **Requires long-running process** - Must call `start()` on server initialization; not compatible with serverless platforms
|
|
- **PostgreSQL infrastructure** - Requires a PostgreSQL database (self-hosted or managed)
|
|
- **Not compatible with Vercel** - Use the [Vercel World](/worlds/vercel) for Vercel deployments
|
|
|
|
For local development, use the [Local World](/worlds/local) which requires no external services.
|