* feat: export semantic error types and add API reference documentation
Add missing error exports (HookNotFoundError, EntityConflictError,
RunExpiredError, TooEarlyError, ThrottleError, RunNotSupportedError,
WorkflowWorldError) to workflow/internal/errors. Create new error
classes for world-level semantics. Tighten TSDoc comments on all
error classes. Add API reference docs for all error types.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use @setup declarations, workflow/errors import, and errors/ doc section
- Replace @skip-typecheck with proper `declare` + `// @setup` lines
so code samples are typechecked but setup lines hidden from readers
- Add `workflow/errors` export to package.json (public API, replaces
`workflow/internal/errors` in docs)
- Add `workflow/errors` path mapping in docs-typecheck type-checker
- Add HookConflictError to re-export list
- Move all error docs under api-reference/workflow/errors/ subdirectory
- Update all internal cross-references and links
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: move error docs to top-level workflow-errors section
- Move semantic error docs to api-reference/workflow-errors/ (matching
the workflow/errors import path, like workflow-api for workflow/api)
- Keep FatalError and RetryableError in api-reference/workflow/ since
they're imported from workflow, not workflow/errors
- Fix all cross-reference links
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update HTTP debug logger JSDoc to clarify scope
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make TooEarlyError.retryAfter a number (seconds) matching WorkflowWorldError
TooEarlyError.retryAfter is now seconds (number) instead of a Date,
consistent with ThrottleError and WorkflowWorldError. The conversion
from seconds to Date is done at the consumer site (step-handler) rather
than at construction time.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address review feedback on docs accuracy
- WorkflowWorldError docs: add status, code, url, retryAfter properties
to TSDoc; clarify that .is() only matches direct instances (not
subclasses); use instanceof in catch-all example
- TooEarlyError/ThrottleError docs: mark retryAfter as optional (?)
to match actual type definitions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: enforce max queue deliveries in handlers with graceful failure
Replace VQS maxDeliveries cap with handler-level enforcement. Handlers
now gracefully fail runs/steps after excessive queue redeliveries,
preventing "phantom stuck" runs.
- Add MAX_QUEUE_DELIVERIES constant (64) and enforce in both workflow
and step handlers with run_failed/step_failed events
- Remove maxDeliveries from VQS trigger configs (builders + sveltekit)
- Improve world-local queue: safety limit loop, structured logging
with runId/stepId, backoff delay on failures
- Add MAX_DELIVERIES_EXCEEDED error code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* address PR review comments
- Expand max-delivery comments explaining minimal-work approach
- Make workflow handler error message verbose (matching step handler)
- Fix comment: "consume the message silently"
- Reduce local queue safety limit from 1000 to 256
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* change MAX_QUEUE_DELIVERIES to 48 and use 5s linear backoff locally
VQS uses linear 5s backoff for attempts 1-32, then exponential capped
at 2h. At 48 attempts total elapsed time is ~20h, safely under the
24h message visibility limit. Local world now uses 5s linear backoff
to approximate VQS timing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: classify run failure error codes and improve error logging
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add semantic error types to replace HTTP status code checks in runtime
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: classify run failure error codes and improve error logging
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: classify run failure error codes and improve error logging
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* address PR review comments
- Remove dead `meta` option from TooEarlyError constructor (TooTallNate)
- Extract `throwWithTrace` helper to deduplicate span recording in
world-vercel makeRequest (TooTallNate)
- Restore `maxAttempts` const for stable retry count logging (TooTallNate)
- Fix behavioral regression: add WorkflowAPIError 404 fallback in
suspension-handler hook disposal to handle world-vercel path where
makeRequest doesn't map 404 to HookNotFoundError (TooTallNate)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: translate 404 to HookNotFoundError at the world-vercel boundary
Move the 404 → HookNotFoundError translation into world-vercel's
createWorkflowRunEvent, where we know the event type context. For
hook-related events (hook_created, hook_disposed, hook_received,
hook_conflict), a 404 from the server means the hook was not found.
This removes the WorkflowAPIError 404 fallback from the runtime's
suspension-handler, keeping the runtime fully decoupled from HTTP
status codes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: parse Retry-After for 425 responses and narrow hook event set
- Parse Retry-After header unconditionally so TooEarlyError gets
the server-provided delay instead of always falling back to ~1s
- Narrow hookEventsRequiringExistence to only hook_disposed and
hook_received (matching world-local's set), since hook_created
and hook_conflict don't imply the hook must already exist
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* rename WorkflowAPIError to WorkflowWorldError
Breaking change: rename WorkflowAPIError → WorkflowWorldError to
better reflect that this error represents world (storage backend)
failures, not HTTP API errors specifically. Updated across all
packages: errors, core, world-local, world-vercel, world-postgres,
workflow, and web.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* add stepName with events
* add changeset
* add workflowname to run created
* add postgres migration
* update world-local
* update world-local
* preserve the fields in the original shape
* fix tests
* strip only ref/payload fields
* stub the helper into world
* add test coverage
* fix web package
* fix web package to not pass withData: true
* Skip OIDC call in fetchRunKey when token already available
getVercelOidcToken() was called unconditionally before checking
options.token or VERCEL_TOKEN. In the CLI/e2e context where a token
is always provided, this is a wasted call that can hang in CI.
Reorder the fallback chain: options.token -> VERCEL_TOKEN -> OIDC.
* Add changeset for OIDC skip fix
* Update changeset and comments to document token precedence order
* feat: support `deploymentId: 'latest'` in `start()` to resolve most recent deployment
Add resolveLatestDeploymentId() to the World interface and implement it
in world-vercel, which calls the Vercel API to resolve the latest
deployment for the current environment. When start() receives
deploymentId: 'latest', it invokes this method and replaces the sentinel
with the resolved deployment ID before passing it to event creation,
queue dispatch, and encryption key derivation.
* fix: address PR review feedback
- Use encodeURIComponent() for deployment ID in API URL path
- Avoid mutating caller-provided opts; pass resolved deploymentId via
spread into getEncryptionKeyForRun context instead
- Add test asserting getEncryptionKeyForRun receives resolved ID
- Add test for OIDC token success path in resolve-latest-deployment
Multiple layered issues caused encryption key 429 errors to produce
confusing, unrelated-looking test failures:
- awaitCommand() in e2e utils now rejects on non-zero exit codes
instead of silently resolving (the most critical fix)
- cliInspectJson/cliHealthJson throw on empty stdout instead of
falling back to '{}'
- maybeDecryptFields re-throws HTTP errors instead of silently
falling back to encrypted placeholders
- fetchRunKey includes response body and status text in errors
- Added 429 to CLI status text map
* test: enable encryption on nextjs-turbopack workbench
Set VERCEL_DEPLOYMENT_KEY to enable e2e encryption for the
nextjs-turbopack workbench project. This allows the e2e test
suite to exercise encrypted workflow runs alongside existing
unencrypted runs for backwards compatibility testing.
* fix: fetch encryption key from API in external contexts (e2e, CLI)
Use VERCEL=1 env var to distinguish serverless runtime (can use local
HKDF derivation) from external contexts like e2e test runners, CLI,
and web dashboard (must fetch derived key from Vercel API).
Previously, when VERCEL_DEPLOYMENT_ID matched but VERCEL_DEPLOYMENT_KEY
was absent (e.g., e2e runner has the deployment ID from CI but not the
key), getEncryptionKeyForRun returned undefined — silently skipping
decryption. Now external contexts always use the API fetch path.
* fix: add 429 retry with exponential backoff to fetchRunKey
The e2e runner calls fetchRunKey for every run, which can exceed the
API rate limit (30/min per team) when many tests run in parallel.
Add retry logic with exponential backoff + jitter (up to 5 retries)
and respect the Retry-After header when present.
* Fix: Missing NaN guard on `retry-after` header parsing causes `setTimeout(r, NaN)` to resolve immediately (0ms), triggering rapid-fire retries against a rate-limited API endpoint.
This commit fixes the issue reported at packages/world-vercel/src/encryption.ts:138
**Bug Analysis:**
In `packages/world-vercel/src/encryption.ts`, the `fetchRunKey` function handles HTTP 429 (rate limited) responses by reading the `retry-after` header and computing a delay. The original code was:
```js
const retryAfter = response.headers.get('retry-after');
const delay = retryAfter
? Number.parseInt(retryAfter, 10) * 1000
: BASE_DELAY_MS * 2 ** attempt + Math.random() * 500;
```
Per RFC 9110, the `Retry-After` header can contain either a number of seconds OR an HTTP date string (e.g., `"Wed, 21 Oct 2015 07:28:00 GMT"`). When the header contains a date string or any non-numeric value, `Number.parseInt(retryAfter, 10)` returns `NaN`. Since the header string is truthy (non-empty), the ternary takes the parseInt path rather than the exponential backoff fallback. `NaN * 1000` evaluates to `NaN`, and `setTimeout(r, NaN)` resolves immediately (~0ms, confirmed by testing). This causes rapid-fire retries (up to 5 iterations with zero delay) against a rate-limited API endpoint, wasting resources and potentially worsening the rate-limiting situation.
**Fix:**
Parse the integer first, then check if the result is NaN before using it. If NaN, fall back to exponential backoff with jitter. This matches the established pattern already used in `packages/world-vercel/src/utils.ts` (lines 309-315) where the same header is properly guarded with `!Number.isNaN(parsed)`.
```js
const retryAfterHeader = response.headers.get('retry-after');
const parsedRetryAfter = retryAfterHeader
? Number.parseInt(retryAfterHeader, 10)
: NaN;
const delay = !Number.isNaN(parsedRetryAfter)
? parsedRetryAfter * 1000
: BASE_DELAY_MS * 2 ** attempt + Math.random() * 500;
```
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: TooTallNate <n@n8.io>
* fix: add --decrypt to e2e CLI inspect calls
With encryption enabled on the nextjs-turbopack workbench, CLI inspect
output shows encrypted data as placeholders. The e2e tests check for
actual output values, so they need --decrypt to see the real data.
* fix: remove manual 429 retry, rely on undici RetryAgent; add changeset
The shared RetryAgent from getDispatcher() already handles 429/5xx
retries with exponential backoff and Retry-After header support.
The manual retry loop in fetchRunKey was redundant.
Also add changeset for the VERCEL=1 external context fix.
* refactor: use @ts-expect-error on dispatcher instead of as any cast
---------
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Previously, immediate re-enqueues used `timeoutSeconds: 1` as an
arbitrary minimum delay. This changes the convention so that
`timeoutSeconds: 0` signals an immediate re-enqueue:
- world-local: fix truthiness check so 0 triggers a 503 response
- world-vercel: omit `delaySeconds` from queue options when 0
- core: use `timeoutSeconds: 0` for hook conflict re-enqueues
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add isWebhook flag to prevent hooks from being resumed via public webhook endpoint
Hooks created with createHook() are now non-resumable via the public webhook
endpoint by default (isWebhook=false). Only hooks created with createWebhook()
set isWebhook=true, allowing them to be resumed via the public URL.
Also adds HookNotFoundError thrown by all world backends when a webhook
token doesn't match any hook, and an e2e test for the new behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix world-local: default isWebhook to false and fix test assertions
- Default isWebhook to false at write time in events-storage
- Default isWebhook to false at read time in hooks-storage (for old data)
- Update test assertions to match HookNotFoundError message
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix: default isWebhook to true for backwards compat, add postgres migration
- Revert read-side default to `isWebhook ?? true` in world-local for
backwards compatibility with existing hooks that predate the field
- Add postgres migration 0009 to add `is_webhook` column with default true
- Update drizzle schema to match
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix run_failed event schema validation failure in lazy ref mode
The EventResultWireSchema used the strict WorkflowRunSchema (discriminated
union) which requires error: StructuredErrorSchema for status 'failed'. When
run_failed events use remoteRefBehavior 'lazy', the server returns the error
field as a RemoteRef or undefined, causing Zod validation to fail.
Switch to WorkflowRunWireBaseSchema which accepts error as string, structured
object, or undefined, and apply deserializeError() to normalize the run entity
before returning it to consumers.
Closes#1180
* Use strict WorkflowRunSchema for resolve mode, loose wire schema for lazy mode
Split EventResultWireSchema into two variants based on remoteRefBehavior:
- EventResultResolveWireSchema uses WorkflowRunSchema (discriminated union)
for events where the client reads result.run (run_created, run_started,
step_started), preserving type safety.
- EventResultLazyWireSchema uses WorkflowRunWireBaseSchema for events where
the client discards the response, allowing error to be a string or
undefined in lazy ref mode.
* Update changeset to reflect resolve/lazy schema split
* Simplify changeset description
* [@workflow/world-vercel] Handle null key response from run-key API endpoint
The API endpoint now returns { key: null } instead of a 404 when a
deployment has no encryption key. Update fetchRunKey to accept the
nullable response and return undefined, signaling that encryption
is disabled for that workflow run.
* [@workflow/world-vercel] Add changeset for null run-key handling
* Address review feedback: improve error message and fix test mock cleanup
- Include Zod error details in the validation error message for better debugging
- Move fetch mock restoration to afterEach to prevent mock leakage on test failure
* Add browser-compatible AES-GCM to core and HKDF key derivation to world-vercel
* update changeset
* Move HKDF key derivation server-side: API returns per-run derived key
* Refactor encrypt/decrypt to accept CryptoKey, export importKey for callers to import once per run
* Pass WorkflowRun entity to getEncryptionKeyForRun where available (avoids redundant lookup)
* .
* Refactor handleSuspension to accept WorkflowRun, pass run entity to getEncryptionKeyForRun
* Overload getEncryptionKeyForRun: accept context for start(), fetch WorkflowRun in resume-hook
* Split changeset into per-package descriptions for world, world-vercel, and core
* Remove unnecessary Uint8Array.from() wrapper around Buffer.from()
* Use zod to parse Vercel API response
* Address peter's comment
Peter Wielander