- @workflow/world: snapshots interface is now OPTIONAL on Storage (a
World that can't store blobs simply omits it; consumers feature-detect
and fall back to full replay) — keeps this a true minor for community
worlds. New encodeSnapshotEnvelope/decodeSnapshotEnvelope helpers pack
metadata + bytes into ONE self-describing blob; decode validates via
the schema (passthrough for forward compat) and returns null for
anything torn/corrupt/foreign — never fabricated metadata.
- world-local: single envelope file per run replaces the .bin/.json pair
— the torn-save window (crash or concurrent load between two renames
pairing bytes with another suspension's cursor) is structurally gone.
- world-postgres: data column stores the envelope; full metadata
round-trips losslessly (new fields need no migration). Columns remain
as denormalized observability copies.
- world-vercel: envelope is the PUT/GET body, so the full metadata
round-trips without any workflow-server change; loads never invent
metadata from headers/wall time (undecodable body = clean miss).
DELETE treats 404 as success (idempotent like local/postgres).
WORLD_SNAPSHOT_DIAG logs gated behind the package's HTTP debug flag.
* feat(world-postgres): retain hook tokens after runs end
* refactor(world-postgres): reuse terminal run statuses
* docs: note Postgres Hook retention support
* fix(world-postgres): expose hook retention deadline
* Fix: Exhaustive `Record<AttributeKey, ...>` in `attribute-panel.tsx` is missing the `tokenRetentionUntil` key that was added to `HookSchema`, causing TS2741 and breaking every Vercel build.
This commit fixes the issue reported at packages/web-shared/src/components/sidebar/attribute-panel.tsx:426
## Bug
Commit `ad58321` added `tokenRetentionUntil: z.coerce.date().optional()` to `HookSchema` in `packages/world/src/hooks.ts:106`. This adds `tokenRetentionUntil` to the inferred `Hook` type.
In `packages/web-shared/src/components/sidebar/attribute-panel.tsx`, `AttributeKey` is a union that includes `keyof Hook`, so `tokenRetentionUntil` becomes a required member of the **exhaustive** `Record<AttributeKey, (value: unknown, context?: DisplayContext) => ...>` object literal `attributeToDisplayFn` (starting at line ~426).
Because the literal had no `tokenRetentionUntil` entry, `tsc` fails:
```
src/components/sidebar/attribute-panel.tsx(426,7): error TS2741:
Property 'tokenRetentionUntil' is missing in type '{ ... }' but required in type
'Record<AttributeKey, (value: unknown, context?: DisplayContext | undefined) => ReactNode>'.
```
This breaks `@workflow/web-shared#build` and therefore every Vercel deployment (17 failing deployments observed, all with this identical error).
## Fix
Added a `tokenRetentionUntil` entry to `attributeToDisplayFn`, placed alongside the other Hook date fields (`lastReceivedAt`, `disposedAt`):
```ts
tokenRetentionUntil: timestampWithTooltipOrNull,
```
`tokenRetentionUntil` is a `Date` field, and `timestampWithTooltipOrNull` (defined at line 402) is the display helper used by all the other surfaced date fields (`createdAt`, `startedAt`, `completedAt`, `retryAfter`, `resumeAt`, `occurredAt`). Given the intent of `ad58321` was to expose the hook retention deadline, surfacing it as a tooltip-annotated timestamp is the consistent choice.
Only `attributeToDisplayFn` is a fully exhaustive `Record<AttributeKey, ...>`; the other maps are `Partial<...>` / `Set`, so no other edits are required.
## Verification
`node_modules` are not installed in this sandbox, so `tsc` could not be executed directly. Verified structurally instead: the newly added `tokenRetentionUntil` entry (line 449) references `timestampWithTooltipOrNull`, which is defined in-file at line 402 and already used by the sibling date entries, so the fix satisfies the missing-key requirement without introducing new type errors.
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: VaguelySerious <mittgfu@gmail.com>
* docs(world-postgres): clarify expired hook rows
* feat(world-postgres): enforce Hook retention limit
* fix(world): remove duplicate Hook retention field
* fix(web-shared): remove duplicate retention renderer
* test(world): remove redundant retention coercion case
---------
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: VaguelySerious <mittgfu@gmail.com>
* feat(core): lazy hook resumption via parallel event write + queue publish (rebased onto #1834 + #3145)
Rebase of #3230 onto current main (267765375 + #1834 resilient resumeHook
+ #3145 event-count-gated replay restart). Reconstructed as a single commit
since `git rebase -i` is unavailable in this environment.
Reconciliation vs the pre-rebase branch:
- Replaces #1834's version-prediction (`supportsQueueHookInput`,
`QUEUE_HOOK_INPUT_MIN_VERSION`) with #3230's capability protocol
(persisted `hookResumeInputVersion` + static `hookResumeDedup`).
- One idempotency protocol: a single `resumeId` + SHA-256 payload digest
per resume, sent to both the direct event write and the queue `hookInput`.
- Two execution tiers: backend+consumer attest dedup -> parallel
`Promise.allSettled(event write, queue publish)`; otherwise plain
sequential (no hookInput/resumeId, event-write errors propagate).
- Consumer re-ensures the `hook_received` event (keyed by resumeId/digest)
after event loading, before replay; skips when already preloaded.
- Preserves #3145: event-count guard, `preconditionReinvocations`,
in-process replay restart, `insertEventByEventId`.
- Removes #1834's resumeId-only test (never released); adds parallel +
consumer-preload + world-local dedup/producer-consumer suites.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(core): read top-level event.resumeId in replay dedup; reconcile unreleased #1834 docs/changeset
- hook.ts: dedup hook_received replay on top-level event.resumeId (the
backend now hoists it to a first-class column), with the legacy nested
eventData.resumeId retained as a deprecated parse-only fallback.
- workflow.test.ts: cover dedup across both top-level and legacy nested forms.
- resume-hook.ts: emit producer recovery telemetry when a transient
event-write failure is swallowed on the parallel path.
- resume-hook.consumer-preload.test.ts: add terminal-run (consume) and
transient-conflict (rethrow/redeliver) re-ensure cases.
- Consolidate the two overlapping changesets into resilient-resume-hook.md
and delete the redundant lazy-hook-resumption.md.
- Docs: return type back to Promise<Hook> (resume-hook.mdx), rewrite the
resilience changelog to the final parallel/deduplicated design, and correct
the WORKFLOW_DISABLE_LAZY_HOOK_RESUME resilience wording.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs,core: rename "Resilient hook resume" → "Lazy hook resume" for consistency
- changelog/index.mdx: update the changelog entry title.
- hook.ts: update the dedup comment label to "Lazy-resume dedup".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: give #3230 its own changeset instead of repurposing #1834's
The lazy-hook-resume work had been folded into #1834's pre-existing
`resilient-resume-hook.md` changeset. Give this PR its own changeset and
delete the superseded #1834 one, whose `resilientResume: true` flag promise
no longer holds (resumeHook() returns plain Promise<Hook>).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: add #3230's own lazy-hook-resumption changeset
Follow-up to 63d877178, which deleted #1834's superseded changeset but did
not stage the replacement. Adds this PR's own changeset.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: tighten lazy-hook-resumption changeset
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: leave #1834's resilient-resume changeset/changelog/docs untouched
Restore #1834's own artifacts that #3230 had rewritten:
- .changeset/resilient-resume-hook.md (restored verbatim)
- docs/.../changelog/resilient-resume.mdx (restored verbatim)
- docs/.../changelog/index.mdx (restored verbatim)
#3230 keeps only its own changeset plus the two docs its code/config genuinely
require: the resumeHook() Promise<Hook> return type (ResumedHook is removed
from the code) and the new WORKFLOW_DISABLE_LAZY_HOOK_RESUME env var.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Reconcile #1834 ResumedHook contract with #3230 parallel resume
Preserve the resilient-resume contract from #1834 on the parallelized
resumeHook() fast path instead of dropping it:
- Restore the `ResumedHook` type (Hook + optional `resilientResume`) and its
exports (`@workflow/core/runtime`, `workflow/api`); resumeHook/resumeHookImpl
return `Promise<ResumedHook>`.
- Set `resilientResume: true` on the swallow-recover branch (transient direct
write failure + successful queue dispatch), absent on the happy/sequential
paths.
- Restore the producer OTEL convention `workflow.hook.resilient_resume` and the
consumer `workflow.hook.resilient_resume_materialized`, wired where the
consumer re-ensures the event.
- Restore the consumer `occurredAt` derivation from the resume ULID so the
materialized hook_received is dated to resume time, not queue-round-trip time.
- Fix the #3230 changeset's contradictory "Still returns Promise<Hook>" line and
update the resilient-resume changelog + resume-hook API reference to the
shipped parallel/dedup behavior.
- Port the #1834 failure-path coverage into resume-hook.parallel.test.ts
(non-retryable event-write rethrow, both-fail prioritizes the queue error,
resilientResume flag + payload delivery on the recovered path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address review: drop dead nested resumeId fallback, remove server PR link
- Drop the legacy nested `eventData.resumeId` fallback in the hook consumer.
The nested form was only ever written by unreleased preview builds and is
stripped by `EventSchema` parsing (the `hook_received` eventData schema does
not declare it), so the fallback was dead code. Dedup now keys solely off the
top-level `event.resumeId` column. Repoint the replay dedup test to the
surviving top-level path (it previously exercised the nested form only by
building unparsed Event objects in memory).
- Remove the internal workflow-server PR reference from world-vercel's
capability note (the link 404s outside the org); the note keeps the same
information without the dead link.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Fix Biome lint violations and add Biome CI check
Biome was not configured to respect .gitignore, so ~92% of the 13,355
reported diagnostics came from gitignored build artifacts. Enable VCS
integration (useIgnoreFile), apply safe auto-fixes across the repo, fix
the remaining mechanical errors by hand, downgrade judgment-call a11y /
dangerouslySetInnerHTML rules to warnings, and add a 'biome ci' job to
the Lint workflow so violations block PRs going forward.
* Use an empty changeset (no behavior change, no release needed)
Hooks can carry an optional `resumeContext` mirrored from the run at
creation time. When present, `resumeHook`/`resumeWebhook` resume directly
from it instead of fetching the full run, saving a round trip per resume.
When the context also carries the run's `encryptionPublicKey`, the resume
seals its payload (`encp`) directly to that key. Combined with the sealed
envelope work (#3093-#3096), a default webhook resume then needs neither a
run read nor a cross-deployment run-key lookup: the key is resolved only
when the hook actually stores metadata that must be hydrated symmetrically.
Everything falls back transparently to the full run fetch and symmetric
key when the context (or the public key within it) is absent, so new
clients interoperate with old servers and vice versa.
- world: optional `encryptionPublicKey` on `HookResumeContext`
- world-postgres: `resume_context` column migration
- core: combined fast-path + seal in resume-hook; fast-path control-flow
suite split from the real-serialization crypto suite
- world-vercel: cover the `getEncryptionKeyForRun(runId, { deploymentId })`
overload the fast path relies on
- web-shared: render `resumeContext` in the attribute panel
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat: publish each run's X25519 public key on the run entity
A cross-run writer needs the recipient run's public key to seal a payload
to it. Derive that key at `start()` and stamp it on the run, so a hook
resumption or a forwarded-stream writer can find it on a run fetch it was
already making instead of spending ~350ms on `run-key`.
The key is derived from the per-run key material `getEncryptionKeyForRun()`
already returns, so nothing about key acquisition changes. It is not
secret: the matching private scalar is never stored anywhere, only
re-derived on demand from the deployment's own env seed. Storing it beside
run metadata therefore does not weaken the run's confidentiality.
**Presence is the writer-side gate for sealed envelopes.** A run only
carries a public key if the runtime that created it could also open one —
which holds by construction, since derivation and `encp` dispatch both
live in `@workflow/core`, so any core that can stamp can also open. Runs
are pinned to their creating deployment, so the capability this attests to
is still accurate at resume time. Writers seal iff the field is set and
otherwise fall back to the symmetric path, which makes version skew
degrade gracefully instead of wedging a run.
The field rides on `run_created`, and is mirrored onto the queued
`runInput` so the resilient-start path (server recreates the run from the
queue message when the `run_created` write failed) doesn't silently
produce a run that can't receive sealed writes.
world-vercel's compile-time wire-contract guard caught the new field
before it could be silently dropped on the v4 path, exactly as designed —
routed into the frame meta block as plaintext metadata.
Also adds browser- and VM-safe base64 helpers to `sealed-box.ts`, since
neither `Buffer` nor `btoa` can be assumed in every context that module
runs in. `base64ToBytes` returns undefined on malformed input rather than
throwing, so a corrupt stored key degrades to "no usable public key" and
falls back to the symmetric path instead of crashing a resumption. Both
are cross-validated against `Buffer` in tests.
* review: fix public-key loss on resilient start and lifecycle updates
Two real bugs found in review, both in the local worlds. Neither surfaces
as an error — a run just silently stops accepting sealed cross-run writes
and falls back to the slow symmetric path forever.
**Resilient start dropped the key.** When a `run_started` arrives for a
run that was never created, world-local and world-postgres rebuild the run
from the queued message. Neither copied `encryptionPublicKey` onto the run
row or the synthetic `run_created` event they write. That is precisely the
scenario this field exists to survive. (The equivalent server-side path was
already handled.)
**world-local also wiped the key on every lifecycle transition.** Its
run_started / run_completed / run_failed / run_cancelled handlers rewrite
the whole run document field-by-field, so any field not explicitly listed
is dropped — meaning the key was lost on the *first* `run_started`, not
just on the resilient path. All four rebuild sites now carry it.
world-postgres is safe here by construction because it issues
column-scoped SQL UPDATEs rather than rewriting the row.
**base64 decoding is now strict.** The decoder accepted shapes that
cannot describe a whole number of bytes (`length % 4 === 1`) and ignored
anything after a mid-string `=`, returning a short array instead of
`undefined`. That is worse than throwing: a corrupt stored key looked
*present*, so callers sealed to garbage rather than taking the symmetric
fallback. Now rejects out-of-alphabet characters, bad lengths, misplaced
padding, and non-zero trailing bits — with a round-trip test over every
length 0–48 to make sure the strictness does not overshoot.
* fix: send encryptionPublicKey in the v4 POST frame meta
`splitEventDataForV4` lifted the run's public key into the frame meta and
`events.ts` spread that meta into `CreateEventV4Input`, but
`buildPostFrameMeta` — which copies meta onto the wire field by field — never
forwarded `encryptionPublicKey`, and the field was missing from
`CreateEventV4Input` entirely. Because the meta is applied with a spread,
TypeScript's excess-property check doesn't fire, so the key was computed, put
in the meta, and then silently dropped before the request was sent.
The server therefore never received the key, never stored it on the run
entity, and every cross-run writer fell back to the symmetric envelope. Every
symptom pointed away from the SDK: a deliberately oversized key was accepted
rather than rejected (the field never arrived), the key was absent from the run
row, and `resumeHook()` always chose `encr`.
Add the field to `CreateEventV4Input`, forward it in `buildPostFrameMeta`, and
cover it for both `run_created` and resilient-start `run_started`. Also add a
generic guard asserting that every field the splitter puts in the meta reaches
the wire, so the next omission in this hand-maintained mapping fails a test
instead of silently degrading encryption.
* build: declare typescript (catalog:) in every package that runs tsc
Twenty packages invoke tsc in their build/typecheck scripts without
declaring a typescript dependency, resolving whatever tsc pnpm happens
to leave reachable. That broke locally after the TypeScript 6 upgrade
(#2700): base.json now uses the TS6-only 'types': ['*'] wildcard, and
worktrees carrying pre-upgrade node_modules/.bin/tsc shims (orphaned
typescript@5.9.3 bins that pnpm never refreshes for an undeclared
dependency) fail with TS2688 'Cannot find type definition file for *'.
Declaring 'typescript': 'catalog:' (the convention nest already
follows) makes pnpm own each package's tsc bin, so version upgrades
refresh the shims and this staleness class cannot recur. Packages
without tsc in their scripts are left unchanged.
Full pnpm build: 27/27 tasks green.
* Address review: drop duplicate zod devDep; regenerate lockfile minimally
- packages/world listed zod in both dependencies and devDependencies
(pre-existing on main, surfaced by the devDependencies sort) — keep
the runtime dependency only.
- Regenerate pnpm-lock.yaml from a pristine main baseline with
--lockfile-only (a clean-main run produces zero diff, so main has no
drift). Remaining non-typescript changes are mechanical consequences
of the change itself: typescript is an (optional) peer of several
tooling dependencies, so declaring it in 20 importers creates new
peer-resolution snapshot variants and prunes the now-orphaned old
ones; plus one radix-ui 1.6.1->1.6.2 refresh in docs caused by its
floating 'latest' specifier.
- Validated: pnpm install --frozen-lockfile succeeds; full build 27/27.
* perf(core): lazy inline step start to save a world round-trip per step
The owned-inline runtime path used to write step_created (suspension
handler) and then step_started (executeStep) as two separate world
round-trips for a step it already owns and is about to run inline. This
defers the step_created write: executeStep sends a single step_started
carrying the step input, and the world creates the step on the fly
(materializing the step entity plus a synthetic step_created event so
replay still observes it). Mirrors the existing resilient run_started ->
run_created pattern.
Exactly-one ownership is preserved by the world's atomic create-claim:
the loser of a concurrent lazy step_started gets EntityConflictError,
which executeStep maps to `skipped`, so it never runs the body. A lazy
step_started is only ever sent for a brand-new step (the suspension
handler defers only steps with no prior step_created), so crash recovery
still re-runs a `running` step via the normal non-lazy step_started.
Worlds updated: world-local, world-postgres (implicit create + synthetic
step_created event), world-vercel (routes the input as the v4 frame
payload and threads the server's stepCreated flag). @workflow/world adds
optional `input` to step_started and a `stepCreated` EventResult signal.
Rollout: server-first. The matching workflow-server change must deploy
before this ships; the Vercel world targets a single Vercel-operated
backend (server always >= SDK). For local/postgres the world ships in the
same package as the runtime, so there is no version skew.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(core): materialize deferred step before failing unregistered step on lazy inline path
The lazy inline step-start optimization defers a step's step_created write,
expecting executeStep to materialize the step via a lazy step_started carrying
its input. For an UNREGISTERED step, executeStep bails out before sending that
step_started and writes step_failed directly — but the step entity was never
created, so the world's "step must exist" ordering guard rejects the
step_failed and the run wedges (times out).
This regressed the StepNotRegisteredError e2e tests uniformly across every
framework/world (the ghost step never reached `failed`). Fix: on the lazy path,
send the lazy step_started first to materialize the step (entity + synthetic
step_created, keeping replay correct), then write step_failed. The lazy
step_started's atomic create-claim preserves exactly-one-owner: a concurrent
winner makes ours reject with EntityConflictError → skipped, so the failure is
never written twice.
Adds world-level regression tests (world-local, world-postgres) asserting a
lazy step_started followed by step_failed marks the step failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Nathan Rajlich