mirror of
https://github.com/vercel/workflow.git
synced 2026-09-14 19:59:43 +08:00
Nathan Rajlich
b5c567c91a
* [DEBUG] Trace replay event log and step/hook/sleep assignments
Temporary diagnostic instrumentation for investigating intermittent
CorruptedEventLogError 'step consumer mismatch' failures.
Emits console.log lines tagged 'WF_TRACE' at four points:
- runWorkflow start: dumps the full event array the replay will consume
(eventIds, types, correlationIds, stepNames) plus a sha256 digest
- step/hook/sleep subscribe: per-replay correlationId -> name assignment
- step consumer mismatch: structured record of the failure including the
event index in the SDK's view of the log
- runWorkflow end: completed | failed | suspended
Used to diff successive replays of the same runId and confirm whether
the SDK actually sees the same event array each time.
* [DEBUG] Extend OCC fence to all branch-decision writes
Peter's PR #2113 fences `wait_completed` writes from the elapsed-wait
scan. This commit extends the fence to every other write whose outcome
depends on a branch decision the workflow VM made from its loaded event
log — per the table @VaguelySerious himself laid out in his PR comment:
suspension-handler.ts:
- step_created (the smoking gun on wrun_01KSPS7XEGHF4A6WYF4DB03D40)
- hook_created
- hook_disposed
- wait_created
runtime.ts terminal writes:
- run_completed
- run_failed
`hook_received` is deliberately NOT fenced (Peter's reasoning preserved
verbatim: fencing the user's signal would drop it on contention; stale-
snapshot protection belongs on the writes that consume hooks, not the
ones that deliver them).
The fence value is the load-time tail of the events array passed into
`runWorkflow`. `suspension-handler` receives the fence + cursor from
the runtime and reloads on conflict; the runtime's terminal writes read
the cursor directly.
The new `__fenced-write.ts` helper encapsulates the retry loop so we
don't have to copy/paste Peter's pattern six times. It's named with the
leading-underscore convention to flag it as throwaway diagnostic code,
matching `__debug-replay-trace.ts`.
* [DEBUG] Point at Peter's workflow-server PR 447 preview + map HTTP 412
Two changes both needed for the extended-fence test loop to actually
exercise the OCC code path on the server:
1. Hardcode WORKFLOW_SERVER_URL_OVERRIDE to
https://workflow-server-83nn57dvc.vercel.sh (preview deployment of
workflow-server PR 447, branch alias
workflow-server-git-peter-event-write-cas.vercel.sh). The previous
preview at workflow-server-7pxaxn4d4.vercel.sh was Pranay's monotonic-
append PR 456 \u2014 different fix, doesn't implement the CAS the SDK side
now sends.
2. Map HTTP 412 \u2192 EntityConflictError in the world-vercel error mapper.
workflow-server PR 447 returns 412 with a 'fence conflict' message
for EventLogFenceConflictError; the SDK's existing fence-retry loops
(Peter's wait_completed scan + the new ones in suspension-handler
and runtime terminal writes) match on /fence conflict/i against the
message of an EntityConflictError. Without this mapping the 412 falls
through to WorkflowWorldError and the regex match never fires.
* fix(core): chain fences for replay-created events
* test: clear workflow server debug override
* fix(core): scope step dispatch to owners
* fix(core): recover wait-raced step dispatch
* Remove diagnostic instrumentation, rename fenced-write helper
Strip the WF_TRACE replay tracing that was used to diagnose the
CORRUPTED_EVENT_LOG race \u2014 it's served its purpose now that the fix
is in. Specifically:
- Delete packages/core/src/__debug-replay-trace.ts and its 8 call sites
in workflow.ts, step.ts, workflow/hook.ts, workflow/sleep.ts.
- Drop the matching [DEBUG] inline narrative comments at each call site.
- Rename packages/core/src/runtime/__fenced-write.ts \u2192 fenced-write.ts
(the leading-underscore convention marked it as throwaway diagnostic
code; the helper is intended to stay).
- Trim the file header on fenced-write.ts and the related narrative
comment in suspension-handler.ts to drop the failing-runId / PR-number
references that only made sense in the debug context.
No behavioral change. typecheck clean (0 errors); 1014/1014 unit tests
pass (same as parent commit 77f057a75).
* world-vercel: always preserve fence-conflict marker on HTTP 412
Address Copilot review on PR 2132 (https://github.com/vercel/workflow/pull/2132#discussion_r3319911635).
The fence-retry loop in runtime/fenced-write.ts detects OCC conflicts
via /fence conflict/i.test(err.message). The 412 branch was relying on
the server's JSON body to populate that message via errorData.message,
but parseResponseBody().catch(() => ({})) swallows JSON parse failures
silently — so any non-JSON 412 response (CDN HTML, gateway timeout
page, intermediate proxy error) would surface as
EntityConflictError("<METHOD> /endpoint -> HTTP 412: Precondition
Failed"), the regex would miss it, and the retry loop would
mis-classify the conflict as terminal.
Prefix the message with `fence conflict:` whenever the parsed body
didn't already carry the marker, so the retry detection is robust to
response-body parse failures.
Tests: world-vercel 69/69 pass.
---------
Co-authored-by: Peter Wielander <peter.wielander@vercel.com>