Commit Graph

1496 Commits

Author SHA1 Message Date
Pranay Prakash 599cf72a65 Merge remote-tracking branch 'origin/main' into pgp/returnvalue-stream
* origin/main:
  test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
  docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
  Retry transient connection timeouts (#3013)
  fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
  [ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
  [ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
  docs: fall back to first child page for sidebar folders without an index (#3009)
  [nest] Fix NestJS Vercel build output (#2988)
  Avoid resolving run data for background steps (#2993)
  chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
  fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
  ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
  fix(core): batch stream writes via writeMulti (#2995)
2026-07-21 13:50:50 +07:00
Pranay Prakash 37a71e6a0a refactor(core): make stream returnValue the default with a kill switch
Owner feedback on the opt-in fast path: stream-based `await run.returnValue`
should be on by default and work on every World, since all worlds already
implement streams.

- `WORKFLOW_RETURN_VALUE_STREAM` is now a default-on emergency kill switch
  (`=0`/`false` restores the pure fixed 1s poll), mirroring `WORKFLOW_TURBO`
  and `WORKFLOW_INLINE_OWNERSHIP`.
- Drop the `returnValueSignalStream` World capability gate. Verified the
  marker-write + close / `startIndex: 0` catch-up contract the waiter relies on
  works out of the box on world-local (filesystem) and world-postgres
  (NOTIFY + durable rows); world-vercel relies on workflow-server's durable
  chunk replay to late readers (documented). Nothing needed fixing.
- Keep the 5s fallback poll (`WORKFLOW_RETURN_VALUE_FALLBACK_POLL_MS`) as the
  internal never-hang backstop.
- Tests: kill-switch-off is byte-identical to the legacy poll; per-world
  catch-up tests for local (in core) and postgres (testcontainers); drop the
  missing-capability tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:40:57 +07:00
Pranay Prakash 9a2770ab34 test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
* test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident)

Reproduces the o2flow v5 upgrade failure (workflow@5.0.0-beta.26, fixed by
#2752 in beta.28): a plain API route importing defineHook() from the root
`workflow` entry and calling .resume() failed with Turbopack's
"Cannot find module as expression is too dynamic" stub, because the world
registration was tree-shaken out of the route bundle and getWorldLazy()'s
dynamic-import fallback got stubbed.

The bug only manifests when a route bundle loads in isolation (a Vercel
lambda): local `next dev`/`next start` evaluates next.config.ts, whose
workflow/next import chain registers the world process-wide and masks it —
which is why no existing server-driven suite caught it.

- route-bundle-isolation.test.ts: production Turbopack build of the
  nextjs-turbopack workbench, then loads ONLY the compiled route bundle in a
  bare Node subprocess (cold-lambda simulation) and invokes its POST handler.
  Fails with the exact incident error on regressed code; passes on main.
  Wired into the build-error-messages CI job.
- e2e: plainModuleDoneHook round-trip through a plain API route on the two
  Next workbenches (deployed matrix covers real lambda isolation).
- Workbench fixtures mirroring o2flow: a directive-less defineHook module
  shared by a workflow (create) and a plain route (resume). The webpack
  workbench gets a real route file because `next dev` (webpack) does not
  serve directory-symlinked app routes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

* test: authenticate plain hook resume request

* test: address review — marker-based harness output parsing, changeset summary

- route-bundle-isolation: prefix the harness result line with a unique
  marker and locate it explicitly instead of JSON.parse()ing the last
  stdout line, so stray logging from the route bundle or the world can't
  break parsing; failures now include the full subprocess stdout.
- changeset: add a human-readable summary to the (release-less) changeset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

---------

Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Karthik Kalyanaraman <karthik.kalyanaraman@vercel.com>
Co-authored-by: Karthik Kalyan <105607645+karthikscale3@users.noreply.github.com>
2026-07-21 13:24:17 +07:00
Pranay Prakash 3c3adf3651 feat(core): fast-path await run.returnValue via a run-scoped stream signal
`await run.returnValue` polls the run record on a fixed ~1s interval, adding
up to a second of quantization latency for a run that finishes mid-interval
(production trace 60cc034ba74b04e0fa2f70aa302a027f).

Add an opt-in fast path: when `WORKFLOW_RETURN_VALUE_STREAM` is on and the
World declares the new `returnValueSignalStream` capability, the poll loop
waits on a run-scoped system stream (`strm_…_system_return`) instead of the
fixed sleep. Every terminal transition in the live flow runtime — plus both
cancellation entry points — writes a tiny marker to that stream and closes it,
waking the waiter within a stream round-trip. The stream is a signal only: the
waiter always re-reads the authoritative run via `runs.get`, and a slow
fallback poll backstops any missed signal (crash between the terminal event
write and the stream write, transient stream failures, pre-feature runs).

Flag off, or a World without the capability, is byte-identical to the previous
fixed-poll behavior. world-vercel declares the capability.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:13:40 +07:00
Pranay Prakash 2b63c6ed72 docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-21 12:55:21 +07:00
Nathan Colosimo 9078126c43 Retry transient connection timeouts (#3013)
* fix: retry transient connection timeouts

* test: extend webpack canary HMR timeout

* Update packages/world-vercel/src/http-client.ts

Co-authored-by: Peter Wielander <mittgfu@gmail.com>
Signed-off-by: Nathan Colosimo <110621881+NathanColosimo@users.noreply.github.com>

---------

Signed-off-by: Nathan Colosimo <110621881+NathanColosimo@users.noreply.github.com>
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
2026-07-20 23:29:46 +00:00
Rui 4ecbe7ecf5 fix(world-vercel): append caller User-Agent products instead of discarding them (#2998) 2026-07-20 16:20:21 -07:00
Peter Wielander 96719d8220 [ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011) 2026-07-20 14:21:17 -07:00
Peter Wielander 0bc22c8e9b [ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005) 2026-07-20 13:50:22 -07:00
Peter Wielander aafc3fd7f4 docs: fall back to first child page for sidebar folders without an index (#3009) 2026-07-20 13:44:33 -07:00
Peter Wielander 542138dc0b [nest] Fix NestJS Vercel build output (#2988) 2026-07-20 12:14:09 -07:00
Nathan Colosimo 6d1d7006cf Avoid resolving run data for background steps (#2993)
* perf(core): avoid resolving run data for background steps

* fix(core): restore input for background replay

Signed-off-by: Nathan Colosimo <110621881+NathanColosimo@users.noreply.github.com>

---------

Signed-off-by: Nathan Colosimo <110621881+NathanColosimo@users.noreply.github.com>
2026-07-20 18:51:11 +00:00
Rich Haines 21448a8ab3 chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
Co-authored-by: Peter Wielander <mittgfu@gmail.com>
2026-07-20 11:23:08 -07:00
Rich Haines e892e8b3c5 fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003) 2026-07-20 10:54:51 -07:00
Nathan Rajlich 3e3dd8c587 ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006) 2026-07-20 09:46:20 -07:00
Peter Wielander 6353c8c6cf fix(core): batch stream writes via writeMulti (#2995) 2026-07-20 09:19:03 -07:00
Joey Hotz d8071bb49a perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
Signed-off-by: Joey Hotz <joeyhotz1@gmail.com>
2026-07-18 09:48:45 -07:00
Mitul Shah 621b04ed52 feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985) 2026-07-17 18:54:37 -04:00
Joey Hotz 3ddf42ed5f fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
Signed-off-by: Joey Hotz <joeyhotz1@gmail.com>
2026-07-17 14:52:51 -07:00
Peter Wielander d53b055a2b [ci] Run benchmarks in-deployment to avoid proxy overhead (#2967) 2026-07-17 14:49:42 -07:00
Peter Wielander bb773e9507 Enable additional perf optimizations when correctness guarantees are met (#2970) 2026-07-17 14:02:14 -07:00
Nathan Colosimo 268fede627 perf(core): prepare replay payloads concurrently (#2980)
* perf(core): cache prepared replay payloads

* test: benchmark workflow-server PR 632

* test: remove workflow-server benchmark pin

* refactor(core): simplify replay preparation types

* fix(core): preserve replay prewarm failures

* refactor(core): use modular replay decrypt

* refactor(core): encapsulate replay payload cache

* fix(core): avoid reawaiting cached replay payloads
2026-07-17 10:18:21 -07:00
Nathan Colosimo 927b61ab41 Fix dotted tsconfig alias workflow discovery (#2963)
* Fix dotted alias workflow discovery

* Increase streamer stress test cleanup timeout

* Increase canary HMR rediscovery timeout
2026-07-16 22:14:10 -07:00
Mitul Shah 1973317488 Adjust helper position on trace viewer (#2968)
* cool

* Update split-pane.tsx
2026-07-16 22:59:50 +00:00
Mitul Shah 774e12c283 feat(web-shared): tooltip on event list icons in new trace viewer (#2962)
Wrap each event row's icon with a Tooltip labeled 'Workflow', 'Step',
'Hook', 'Sleep' (or 'Event' fallback). Uses a 500ms delay so tooltips
don't appear instantly on incidental hover.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-07-16 14:02:45 -07:00
Alex Langenfeld 457e671ca9 fix(world-vercel): log queue handler retry errors (#2959) 2026-07-16 12:33:22 -07:00
christopherkindl 5554c253ae chore(docs): update @vercel/geistdocs to 1.11.4 (#2955)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 09:27:40 -07:00
Joey Hotz 7d29babaef feat(world): add optional getMany() for batch run reads (#2915)
Signed-off-by: Joey Hotz <joeyhotz1@gmail.com>
2026-07-16 08:29:56 -07:00
Pranay Prakash 8a872529fe docs: make /worlds the canonical home for World docs (#2934)
* docs: make /worlds the canonical home for World docs

The world pages (Local/Postgres/Vercel) and Building a World were
duplicated inside the v4 and v5 docs trees while /worlds/[id] rendered
the v4 copy — hiding v5-only content like multi-region and leaving two
diverging sources of truth.

- Move world docs to an unversioned docs/content/worlds/ collection
  (based on the v5 copies, with inline 4.x callouts for factory naming
  and 5.x-only env vars), rendered at /worlds/*
- Add /worlds/building-a-world; flatten the docs Deploying section to a
  single intro page and drop its Rocket icon
- Point every link, frontmatter ref, and worlds-manifest docs field at
  /worlds/*; add redirects for the removed v5 and building-a-world URLs
- Keep world docs on agent-facing surfaces: search, llms.txt,
  sitemap.md/.xml, and .md exports now serve the worlds collection
- Extend the docs link linter to validate worlds pages (with heading
  anchors) and their outgoing links

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

* docs: version the world docs like the docs trees (v4/v5 switcher)

Instead of a single unversioned copy, world docs now follow the same
versioning strategy as the docs pages: content/worlds/v4 is served at
/worlds/* (current) and content/worlds/v5 at /v5/worlds/*, restoring the
original per-version content. Each world detail page (and Building a
World) renders the docs version switcher — the worlds listing page has
no natural home for it, so it lives on the world pages themselves.

- Render-time href rewriting on v5 pages now covers /worlds/... links
  (shared rewriteHrefForVersion helper, also used by the v5 docs and
  cookbook routes), and the markdown-export rewrite does the same
- v5 world pages are noindexed with a canonical to /worlds/<id>;
  community worlds stay unversioned (/v5/worlds/<id> redirects)
- /v5/docs/deploying/world/* redirects now land on /v5/worlds/*;
  /v5/worlds and /v5/worlds/compare redirect to the unversioned pages
- Link linter models the versioned worlds URL spaces (v5 pages resolve
  /worlds hrefs against the v5 collection); sitemap.md and the .md
  export routes cover /v5/worlds/*

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

* docs: fix v4 multi-region anchor and tighten version-prefix matching

Address PR review:
- The v4 Deploying page linked /worlds/vercel#multi-region, but the
  Multi-region section only exists on the v5 world page; use the
  explicit cross-version /v5/worlds/vercel#multi-region link (this was
  the Docs Links CI failure)
- rewriteHrefForVersion now uses the boundary-checked hasPathPrefix
  (shared leaf module lib/geistdocs/path-prefix.ts, also used by
  source.ts) instead of bare startsWith
- buildVersionUrl's shared-route fast path is segment-based rather than
  substring includes()

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>

---------

Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:15:07 +07:00
Karthik Kalyan 6f032d73fe fix(world-vercel): decode legacy structured errors (#2951) 2026-07-15 15:17:41 -07:00
github-actions[bot] 784f03231e Version Packages (beta) (#2919) workflow@5.0.0-beta.35 2026-07-15 14:31:53 -07:00
Mitul Shah 0d5305b9df fix(web-shared): use gray for hook bars in the trace viewer (#2950)
Hook timeline bars were amber/yellow via RESOURCE_COLORS; passive
spans should be gray to match the event list icons and minimap.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-07-15 14:00:06 -07:00
Mitul Shah cbfe13bb63 Give Metadata Token and Hook ID copy + truncation support (#2947)
* Give Metadata Token and Hook ID copy + truncation

Add token to the copyable metadata attributes set and constrain
copyable key-value rows so MiddleTruncate can shrink long IDs.

* Remove AttributePanel copy unit tests

The Metadata Token/Hook ID copy change is small enough that the
dedicated panel render tests are unnecessary.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-07-15 16:59:02 -04:00
Peter Wielander 1933e294cf Report RSFS/replay latency telemetry on step terminal events (#2929) 2026-07-15 13:50:41 -07:00
Peter Wielander fd107b9c33 [core] Fix time parsing for region-tagged run IDs (#2943) 2026-07-15 13:05:46 -07:00
Michael J. Sullivan 60c33d4038 Make sure that everyone is still /docs code owners (#2939) 2026-07-15 11:42:20 -07:00
Roey D. Chasman 6b8efd58ce feat: cross-run lineage via reserved run attributes (#2153)
Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
2026-07-15 11:41:31 -07:00
Ricardo Gonzalez d07bbf2ef7 update python docs to no longer mention experimentalServices (#2935) 2026-07-15 12:38:49 +00:00
Michael J. Sullivan 45c29e5e2a Add team-python to CODEOWNERS for docs (#2936)
We need to be able to write docs for the python workflow SDK.
2026-07-15 14:37:10 +02:00
Peter Wielander a00d169470 Add stateUpdatedAt precondition guard to event creation (#2266) 2026-07-15 02:11:14 +00:00
Nathan Colosimo c44b4f8586 fix(nitro): skip generated build artifacts (#2925) 2026-07-14 15:37:54 -07:00
Nathan Colosimo 35899580bd Fix Nitro cleanup for React Router and add setup guides (#2908)
* fix(nitro): support React Router Vite builds

* refactor(nitro): simplify React Router cleanup

* docs(react-router): specify cleanup version

* fix(nitro): close temporary Vite servers
2026-07-14 13:34:43 -07:00
Elliot Dauber 2588e53c02 [docs] fix code highlight in child workflows cookbook (#2922) 2026-07-14 13:32:33 -07:00
Karthik Kalyan f72184dc83 feat(world-local): add WORKFLOW_LOCAL_RECOVER_ACTIVE_RUNS env var (#2914)
The recoverActiveRuns factory option had no environment variable, so
disabling startup re-enqueueing of pending/running runs required a custom
world module via WORKFLOW_TARGET_WORLD. Wire an env fallback
(0/false disables, 1/true enables, explicit factory option wins) and
document it in the worlds configuration reference and local world guide.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 12:45:59 -07:00
Karthik Kalyan cf96800ec6 Update vitest from 4.0.18 to 4.1.10 (#2916) 2026-07-14 11:49:01 -07:00
github-actions[bot] bd5fc50f66 Version Packages (beta) (#2913)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
workflow@5.0.0-beta.34
2026-07-13 20:26:51 -07:00
Karthik Kalyan 9242ddb02c telemetry: move client stream spans from world-vercel to core (#2901)
* fix(deps): dedupe @opentelemetry/api to a single workspace instance

The lockfile resolved both 1.9.0 and 1.9.1, so the copy that registers
the tracer provider (via @vercel/otel in the app) and the copy a package
imports could differ. The API's global-registration version check rejects
a consumer newer than the registered copy and silently hands back a noop
tracer — which is why world-vercel's spans (workflow.stream.write/
chunk_rtt, read.connect, its http spans) never reached Datadog from
deployed apps while core's spans flowed in the same process. Root-caused
via the DEBUG=workflow:* run on #2900: import succeeds, no warn, spans
dropped.

Pin a single version via a workspace override so every bundle shares one
API instance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* debug: one-shot OTEL runtime diagnostic in core + world-vercel; DEBUG on turbopack workbench

The dedupe alone did not restore world-vercel span emission (verified on
this PR's own preview: stream traffic flowed, zero workflow.stream.write
spans). Under DEBUG=workflow:*, both packages now log once how their
module instance of @opentelemetry/api sees the world — global
registration version, provider/delegate/tracer/probe constructor names,
and whether a probe span is recording. Diffing the core line (spans work)
against the world-vercel line (spans dropped) in one deployment's logs
pinpoints the divergence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* debug: log span identity for named world-vercel spans; namespace otel probes per package

Diag round 1 showed world-vercel's tracer records and instrumentedFetch
handles the stream PUTs, yet the named spans are unfindable in the
backend. Round 2: log traceId/spanId/isRecording for every named
instrumentedFetch span under DEBUG so export can be checked for a
specific span id, and split the probe span names (.core /
.world_vercel) so per-package export is attributable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* telemetry: emit stream RPC latencies from core (chunk_rtt, connect_ms, close span)

world-vercel's instrumentedFetch spans never export from deployed apps
(root cause still open — see PR discussion), so the operationally
needed client-side latency signals move one layer up to core, whose
spans are proven to export:

- workflow.stream.write.chunk_rtt on the workflow.stream.flush span:
  the World write RPC duration, network included (same attribute key as
  world-vercel's per-request span so queries are layer-agnostic).
- workflow.stream.read.connect_ms on the workflow.stream.read span:
  the world.streams.get await (read dispatch -> stream handle).
- new workflow.stream.close span: the close RPC round trip.

Bonus: measured at the World interface, these cover world-local and
world-postgres too, not just Vercel deployments.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* telemetry: emit read-completion span (total duration, chunks, bytes)

Completes the read-side picture: workflow.stream.read.complete is
back-dated to the read dispatch so its duration is the total read, with
chunk/byte counts for throughput. Cancelled reads emit nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: drop DEBUG from turbopack workbench; tighten changeset

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* telemetry: cover createReconnectingFramedStream in read telemetry

Ordinary serialized streams read through createReconnectingFramedStream
(which calls world.streams.get directly), so connect_ms / ttfc /
read.complete never fired for that path — only WorkflowServerReadableStream
was instrumented. Wire the same helpers into the framed reader: first-
connect duration, first-frame TTFC, and completion totals — plus
workflow.stream.read.reconnects, which only this path can know.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 19:34:01 -07:00
Karthik Kalyan 200845d885 docs: clarify Vercel-specific analytics notes and align sidebar chevrons (#2909)
- analytics: attribute the serving path to the Vercel observability data
  pipeline instead of naming the ClickHouse store, drop the Vercel-only
  "Plan-bounded lookback" bullet, and mark the pageInfo lookback ceiling
  and upgradeAvailable comments as Vercel-specific.
- sidebar: folder rows without an index page render as <button>s, which
  don't stretch to the row width like the <a> folder links, leaving their
  ms-auto chevron hugging the label. Stretch those triggers to full width
  so every chevron sits at the end of the row.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:51:00 -07:00
Nathan Rajlich 6bbb5e264e docs: fix stale and broken vercel.com links (#2910)
Audited every vercel.com link in docs/content (20 unique URLs,
HTTP-validated including anchor fragments):

- project-configuration#regions (2x): the #regions anchor no longer
  exists on that page — content moved to the vercel-json subpage; now
  links project-configuration/vercel-json#regions
- gateway/api-reference/overview (2x): hard 404; the AI Gateway docs
  restructured — 'get an API key' context now points at
  ai-gateway/authentication
- observability/otel-overview (1x): redirects to
  tracing/instrumentation; link the final URL
- docs/workflow and docs/workflow/python (8x): redirect to the plural
  docs/workflows paths; link the final URLs (also drops a redundant
  ?language=py param that the redirect discards)

All other links (queues, queues/pricing + anchors, plans/hobby,
limits, regions, sandbox, workflows/pricing#storage-retention,
cli/project-linking, audit-log, home, help, blog) verified 200 with
live anchors.
2026-07-13 16:49:10 -07:00
github-actions[bot] 4ecef5303e Version Packages (beta) (#2904)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
workflow@5.0.0-beta.33
2026-07-13 16:01:51 -07:00