Files
Karthik Kalyan 9242ddb02c telemetry: move client stream spans from world-vercel to core (#2901)
* fix(deps): dedupe @opentelemetry/api to a single workspace instance

The lockfile resolved both 1.9.0 and 1.9.1, so the copy that registers
the tracer provider (via @vercel/otel in the app) and the copy a package
imports could differ. The API's global-registration version check rejects
a consumer newer than the registered copy and silently hands back a noop
tracer — which is why world-vercel's spans (workflow.stream.write/
chunk_rtt, read.connect, its http spans) never reached Datadog from
deployed apps while core's spans flowed in the same process. Root-caused
via the DEBUG=workflow:* run on #2900: import succeeds, no warn, spans
dropped.

Pin a single version via a workspace override so every bundle shares one
API instance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* debug: one-shot OTEL runtime diagnostic in core + world-vercel; DEBUG on turbopack workbench

The dedupe alone did not restore world-vercel span emission (verified on
this PR's own preview: stream traffic flowed, zero workflow.stream.write
spans). Under DEBUG=workflow:*, both packages now log once how their
module instance of @opentelemetry/api sees the world — global
registration version, provider/delegate/tracer/probe constructor names,
and whether a probe span is recording. Diffing the core line (spans work)
against the world-vercel line (spans dropped) in one deployment's logs
pinpoints the divergence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* debug: log span identity for named world-vercel spans; namespace otel probes per package

Diag round 1 showed world-vercel's tracer records and instrumentedFetch
handles the stream PUTs, yet the named spans are unfindable in the
backend. Round 2: log traceId/spanId/isRecording for every named
instrumentedFetch span under DEBUG so export can be checked for a
specific span id, and split the probe span names (.core /
.world_vercel) so per-package export is attributable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* telemetry: emit stream RPC latencies from core (chunk_rtt, connect_ms, close span)

world-vercel's instrumentedFetch spans never export from deployed apps
(root cause still open — see PR discussion), so the operationally
needed client-side latency signals move one layer up to core, whose
spans are proven to export:

- workflow.stream.write.chunk_rtt on the workflow.stream.flush span:
  the World write RPC duration, network included (same attribute key as
  world-vercel's per-request span so queries are layer-agnostic).
- workflow.stream.read.connect_ms on the workflow.stream.read span:
  the world.streams.get await (read dispatch -> stream handle).
- new workflow.stream.close span: the close RPC round trip.

Bonus: measured at the World interface, these cover world-local and
world-postgres too, not just Vercel deployments.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* telemetry: emit read-completion span (total duration, chunks, bytes)

Completes the read-side picture: workflow.stream.read.complete is
back-dated to the read dispatch so its duration is the total read, with
chunk/byte counts for throughput. Cancelled reads emit nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: drop DEBUG from turbopack workbench; tighten changeset

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* telemetry: cover createReconnectingFramedStream in read telemetry

Ordinary serialized streams read through createReconnectingFramedStream
(which calls world.streams.get directly), so connect_ms / ttfc /
read.complete never fired for that path — only WorkflowServerReadableStream
was instrumented. Wire the same helpers into the framed reader: first-
connect duration, first-frame TTFC, and completion totals — plus
workflow.stream.read.reconnects, which only this path can know.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 19:34:01 -07:00
..
…

Workflow SDK Docs

Check out the docs here