mirror of
https://github.com/vercel/workflow.git
synced 2026-09-14 19:59:43 +08:00
Pranay Prakash
5f0b845211
* feat(core,world): gzip-compress serialized payloads behind specVersion 5 Add a composable 'gzip' format prefix layer to the serialization pipeline (compress before encrypt: encr(gzip(devl))), cutting stored payload bytes by ~70-87% on real-world-style workloads. Compression is gated on run specVersion 5 (new SPEC_VERSION_SUPPORTS_COMPRESSION) and on target-deployment capabilities for cross-deployment writes; payloads under 1KB or that don't compress meaningfully are stored unchanged. Reads dispatch on the format prefix so both compressed and uncompressed data are always readable. WORKFLOW_DISABLE_COMPRESSION=1 disables writes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(core): add CPU/perf compression benchmark + shared workloads Split the compression benchmark into reproducible size and CPU scripts sharing deterministic workloads (lib/workloads.mjs). The CPU benchmark measures serialize/deserialize overhead per payload, total CPU across thousands of events, and compares gzip levels/brotli/deflate. Documents how to run the size, CPU, and end-to-end (bench.bench.ts) benchmarks against local and Vercel in scripts/README.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(world-vercel): advertise specVersion 5 to enable compression on Vercel Now that workflow-server declares spec-5 support (vercel/workflow-server#520), bump the Vercel world's advertised specVersion from 4 to 5 so new Vercel runs are stamped spec 5 and become eligible for gzip payload compression. Payloads stay opaque to the server (compression is client-side); spec 5 is a superset of spec 4, so initial run attributes still work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(core): emit OTel span attributes for compression impact Track gzip payload compression on both the serialize (write) and deserialize (read) paths via span attributes: workflow.serialization.{operation,compressed,uncompressed_bytes, stored_bytes,compression_ratio}. Sizes are measured at the compression boundary (pre-encryption), so they reflect compression's effect rather than the at-rest size. The compression codec stays pure — compress/decompress optionally populate a CompressionStats sink, threaded through CodecOptions to the mode serializers and read by the dehydrate/hydrate wrappers, which set attributes on the active span. Telemetry failures are swallowed so they can never break the serialize/deserialize data path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(core,web-shared): prefer zstd compression codec (gzip fallback) Switch the payload compression codec to zstd, which benchmarks 3–7× faster than gzip at an equal-or-better ratio on representative workloads (compression runs at every step boundary, so the write CPU is a per-step tax). zstd uses node:zlib (>= 22.15); gzip via the portable CompressionStream remains the fallback when zstd is unavailable, and WORKFLOW_COMPRESSION_CODEC=gzip forces it. Reads dispatch on the format prefix, so 'zstd' and 'gzip' payloads are both always decodable. zstd is Node-only (Web CompressionStream has no zstd), so the browser o11y read path registers a WASM-backed decoder (@tootallnate/zstd-wasm) via a new registerZstdDecoder hook; node:zlib handles Node-side reads (runtime replay, CLI, server o11y). A new workflow.serialization.codec span attribute reports which codec applied. gzip and zstd read support co-ship, so the existing specVersion-5 capability gate is unchanged. Verified end-to-end: spec-5 runs store zstd-prefixed payloads on disk and replay/complete correctly; the WASM decoder round-trips node:zlib zstd output. Benchmarks updated to compare zstd vs gzip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
674 B
674 B
@workflow/core
| @workflow/core |
|---|
| minor |
Compress serialized payloads (step inputs/outputs, workflow arguments/return values, errors, hook payloads) before storage using composable codec format prefixes. zstd is the preferred codec (markedly faster than gzip at an equal-or-better ratio, via node:zlib); gzip (CompressionStream) is the portable fallback when zstd is unavailable. Reads dispatch on the prefix, so both codecs are always decodable. Compression is applied before encryption, gated on run specVersion 5, and skipped for small or incompressible payloads. WORKFLOW_DISABLE_COMPRESSION=1 disables writes; WORKFLOW_COMPRESSION_CODEC=gzip forces the portable codec.