* Fix race condition allowing duplicate hook_disposed events
Concurrent workflow invocations could both post hook_disposed for the
same hook, corrupting the event log with duplicate events. This mirrors
the wait_completed race condition fixed in #1057/#1434.
- world-local: Add writeExclusive lock file for hook_disposed (same
pattern as wait_completed and step terminal states)
- world-postgres: Use DELETE ... RETURNING to atomically detect if
another caller already deleted the hook entity
- suspension-handler: Improve log messages to distinguish hook-already-
disposed (EntityConflictError) from run-already-completed (RunExpiredError)
Fixes#1266
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Accept either EntityConflictError or HookNotFoundError in race test
The concurrent hook_disposed race has two possible orderings: the loser
may hit the lock file (EntityConflictError) or find the hook entity
already deleted by the winner (HookNotFoundError). Both are correct.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Assert only one hook_disposed event in event log after race
Verifies the losing concurrent caller didn't sneak an event in before
the guard threw.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: export semantic error types and add API reference documentation
Add missing error exports (HookNotFoundError, EntityConflictError,
RunExpiredError, TooEarlyError, ThrottleError, RunNotSupportedError,
WorkflowWorldError) to workflow/internal/errors. Create new error
classes for world-level semantics. Tighten TSDoc comments on all
error classes. Add API reference docs for all error types.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use @setup declarations, workflow/errors import, and errors/ doc section
- Replace @skip-typecheck with proper `declare` + `// @setup` lines
so code samples are typechecked but setup lines hidden from readers
- Add `workflow/errors` export to package.json (public API, replaces
`workflow/internal/errors` in docs)
- Add `workflow/errors` path mapping in docs-typecheck type-checker
- Add HookConflictError to re-export list
- Move all error docs under api-reference/workflow/errors/ subdirectory
- Update all internal cross-references and links
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: move error docs to top-level workflow-errors section
- Move semantic error docs to api-reference/workflow-errors/ (matching
the workflow/errors import path, like workflow-api for workflow/api)
- Keep FatalError and RetryableError in api-reference/workflow/ since
they're imported from workflow, not workflow/errors
- Fix all cross-reference links
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update HTTP debug logger JSDoc to clarify scope
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make TooEarlyError.retryAfter a number (seconds) matching WorkflowWorldError
TooEarlyError.retryAfter is now seconds (number) instead of a Date,
consistent with ThrottleError and WorkflowWorldError. The conversion
from seconds to Date is done at the consumer site (step-handler) rather
than at construction time.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address review feedback on docs accuracy
- WorkflowWorldError docs: add status, code, url, retryAfter properties
to TSDoc; clarify that .is() only matches direct instances (not
subclasses); use instanceof in catch-all example
- TooEarlyError/ThrottleError docs: mark retryAfter as optional (?)
to match actual type definitions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: enforce max queue deliveries in handlers with graceful failure
Replace VQS maxDeliveries cap with handler-level enforcement. Handlers
now gracefully fail runs/steps after excessive queue redeliveries,
preventing "phantom stuck" runs.
- Add MAX_QUEUE_DELIVERIES constant (64) and enforce in both workflow
and step handlers with run_failed/step_failed events
- Remove maxDeliveries from VQS trigger configs (builders + sveltekit)
- Improve world-local queue: safety limit loop, structured logging
with runId/stepId, backoff delay on failures
- Add MAX_DELIVERIES_EXCEEDED error code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* address PR review comments
- Expand max-delivery comments explaining minimal-work approach
- Make workflow handler error message verbose (matching step handler)
- Fix comment: "consume the message silently"
- Reduce local queue safety limit from 1000 to 256
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* change MAX_QUEUE_DELIVERIES to 48 and use 5s linear backoff locally
VQS uses linear 5s backoff for attempts 1-32, then exponential capped
at 2h. At 48 attempts total elapsed time is ~20h, safely under the
24h message visibility limit. Local world now uses 5s linear backoff
to approximate VQS timing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: classify run failure error codes and improve error logging
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add semantic error types to replace HTTP status code checks in runtime
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: classify run failure error codes and improve error logging
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: classify run failure error codes and improve error logging
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* address PR review comments
- Remove dead `meta` option from TooEarlyError constructor (TooTallNate)
- Extract `throwWithTrace` helper to deduplicate span recording in
world-vercel makeRequest (TooTallNate)
- Restore `maxAttempts` const for stable retry count logging (TooTallNate)
- Fix behavioral regression: add WorkflowAPIError 404 fallback in
suspension-handler hook disposal to handle world-vercel path where
makeRequest doesn't map 404 to HookNotFoundError (TooTallNate)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: translate 404 to HookNotFoundError at the world-vercel boundary
Move the 404 → HookNotFoundError translation into world-vercel's
createWorkflowRunEvent, where we know the event type context. For
hook-related events (hook_created, hook_disposed, hook_received,
hook_conflict), a 404 from the server means the hook was not found.
This removes the WorkflowAPIError 404 fallback from the runtime's
suspension-handler, keeping the runtime fully decoupled from HTTP
status codes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: parse Retry-After for 425 responses and narrow hook event set
- Parse Retry-After header unconditionally so TooEarlyError gets
the server-provided delay instead of always falling back to ~1s
- Narrow hookEventsRequiringExistence to only hook_disposed and
hook_received (matching world-local's set), since hook_created
and hook_conflict don't imply the hook must already exist
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* rename WorkflowAPIError to WorkflowWorldError
Breaking change: rename WorkflowAPIError → WorkflowWorldError to
better reflect that this error represents world (storage backend)
failures, not HTTP API errors specifically. Updated across all
packages: errors, core, world-local, world-vercel, world-postgres,
workflow, and web.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [world-postgres] Fix TOCTOU race in step_started that corrupts event log
The step_started UPDATE had no conditional guard on step status, allowing
a concurrent execution to revert a completed step back to 'running'. This
caused duplicate step_completed events, triggering CORRUPTED_EVENT_LOG.
Add notInArray guard to match the existing pattern on step_completed and
the DynamoDB conditional expression used in the Vercel world.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [world-postgres] Add atomic terminal-state guards to all entity UPDATEs
Add conditional WHERE clauses to match the Vercel world's DynamoDB
conditional expressions, preventing TOCTOU races where concurrent
requests could bypass pre-validation and write invalid state transitions.
Changes:
- step_started: add NOT IN (completed, failed, cancelled) guard
- step_retrying: add terminal-state guard (was unguarded)
- step_completed/step_failed: add cancelled to guard
- run_completed/run_failed/run_cancelled: add terminal-state guards
- isStepTerminal: include cancelled status
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [world-local] Add atomic terminal-state guards and concurrent race tests
Local world fixes:
- step_completed/step_failed: use writeExclusive lock to prevent
concurrent duplicate terminal transitions
- step_started: check for terminal lock file before allowing start
- wait_completed: use writeExclusive lock (port from PR #1388)
- isStepTerminal: include cancelled status
Tests:
- Concurrent step_completed race (exactly one succeeds, one gets 409)
- Concurrent step_failed race
- step_started rejection after concurrent step_completed
- Concurrent wait_completed race
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Address PR review feedback
- Fix lock file extension: use .lock instead of .json via taggedPath to
avoid polluting entity directories with empty JSON files that cause
SyntaxError during listing/parsing
- Make startedAt update atomic using COALESCE in SQL instead of
deriving isFirstStart from the TOCTOU validation read
- Split changeset into separate entries per package
- Remove unnecessary context from changeset text
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Address review feedback from @TooTallNate
- step_started: replace fs.access() with a fresh re-read of the step
entity — honest about being best-effort rather than claiming atomicity
(local world is dev-only; postgres world has SQL-level atomic guards)
- wait_completed: clean up lock file on 404 to avoid leaked lock files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move lock files to .locks/ subdirectory off basedir
Lock files in entity directories (steps/, waits/) broke tests that
expect only tagged .json files. Move all locks to basedir/.locks/
and clean them up in clear().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pass ISO string to COALESCE instead of Date object
The postgres driver can't serialize a Date object inside a raw sql
template literal. Convert to ISO string for proper parameterization.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: embed tag in lock file names for test isolation
Parallel vitest workers with different tags would collide on untagged
lock files, causing spurious 409s. Include the tag in the lock filename
(e.g. stepId.terminal.vitest-0) matching the pattern from PR #1388.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
* add stepName with events
* add changeset
* add workflowname to run created
* add postgres migration
* update world-local
* update world-local
* preserve the fields in the original shape
* fix tests
* strip only ref/payload fields
* stub the helper into world
* add test coverage
* fix web package
* fix web package to not pass withData: true
* fix(world-local): return HTTP 200 instead of 503 for queue timeout re-enqueue signals
The local queue used HTTP 503 (Service Unavailable) as an internal signal
for re-enqueueing after a timeout delay. This is misleading since 503 is
an error status, causing confusion when inspecting logs or network traffic.
Changed to return 200 with timeoutSeconds in the body, and updated the
consumer to detect the re-enqueue signal from the response body instead
of the status code.
* fix(world-local): handle timeoutSeconds: 0 and add queue re-enqueue tests
Address review feedback: timeoutSeconds === 0 should trigger an immediate
re-enqueue without delay, not be treated as a normal success. Also adds
test coverage for the handler response codes and re-enqueue behavior.
Previously, immediate re-enqueues used `timeoutSeconds: 1` as an
arbitrary minimum delay. This changes the convention so that
`timeoutSeconds: 0` signals an immediate re-enqueue:
- world-local: fix truthiness check so 0 triggers a 503 response
- world-vercel: omit `delaySeconds` from queue options when 0
- core: use `timeoutSeconds: 0` for hook conflict re-enqueues
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
hooks.list() in world-local and world-postgres defaulted to descending
order, while world-vercel defaults to ascending (creation order). This
caused the webhookWorkflow e2e test to deadlock: hooks were returned
newest-first, so the test sent the first webhook request to the manual-
response hook instead of the default-response hook.
- world-local: default sortOrder to 'asc', fix getCreatedAt returning
new Date(0) which broke ascending cursor pagination
- world-postgres: respect the sortOrder param (was hardcoded to desc)
and default to 'asc'
* Add isWebhook flag to prevent hooks from being resumed via public webhook endpoint
Hooks created with createHook() are now non-resumable via the public webhook
endpoint by default (isWebhook=false). Only hooks created with createWebhook()
set isWebhook=true, allowing them to be resumed via the public URL.
Also adds HookNotFoundError thrown by all world backends when a webhook
token doesn't match any hook, and an e2e test for the new behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix world-local: default isWebhook to false and fix test assertions
- Default isWebhook to false at write time in events-storage
- Default isWebhook to false at read time in hooks-storage (for old data)
- Update test assertions to match HookNotFoundError message
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix: default isWebhook to true for backwards compat, add postgres migration
- Revert read-side default to `isWebhook ?? true` in world-local for
backwards compatibility with existing hooks that predate the field
- Add postgres migration 0009 to add `is_webhook` column with default true
- Update drizzle schema to match
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Peter Wielander