3 Commits

Author	SHA1	Message	Date
James Anderson	76de660c20	fix(image): emit static imports as managed assets (#2040) ... * fix(image): emit static imports as managed assets * fix(image): handle static asset rebuilds and caching * fix(image): scope managed media hash etags * test(image): escape all URL ampersands	2026-06-15 23:04:40 +01:00
James Anderson	d504f022b0	fix(assets): default assetsDir to _next/static (Next.js parity) (#1411) ... * fix(assets): default assetsDir to _next/static (Next.js parity) Closes #1337. Supersedes #1383. vinext's default `assetsDir` was Vite's historical `assets/`, so URL emission (`/_next/static/...` via `resolveAssetUrlPrefix("")`) and on-disk layout (`dist/client/assets/...`) disagreed in the empty-`assetPrefix` case. The `build.assetsDir` and `experimental.renderBuiltUrl` Vite overrides only applied when `assetPrefix` was configured, leaving the no-prefix branch serving from a different path than the URL contract Next.js's client runtime and test harness assert against. This commit flips the default to Next.js's canonical layout: - `resolveAssetsDir("")` returns `_next/static` (was `assets`) - `build.assetsDir` is now set unconditionally from `resolveAssetsDir` - Vite's default `base + assetsDir` composition produces correct URLs in the no-prefix case; `renderBuiltUrl` stays gated on `assetPrefix` because it's only needed for the configured cases - Drops the legacy `/assets/` branches in prod-server (3 sites) and static-file-cache (hard cutover — see PR description for rationale) - Updates `_headers` generation, precompress default, fonts plugin `DEFAULT_ASSETS_DIR` to match the new default With the layout aligned, invalid `_next/static/*` requests naturally return plain-text `404 + "Not Found"` from the static-file layer instead of falling through to the page renderer (which would produce an HTML 404 with bootstrap scripts and CSS). This replaces #1383's parity short-circuit with the natural code path: - Node prod-server (App + Pages branches): missing asset under `resolveAppRouterAssetPath` returns `text/plain; charset=utf-8` 404 - Cloudflare worker entry: `isNextStaticPath` recognises asset-shape after ASSETS-binding misses; returns `notFoundStaticAssetResponse` Mirrors Next.js: packages/next/src/server/lib/router-server.ts. Adds `tests/invalid-static-asset-404.test.ts` (3 App Router + 3 Pages Router cases — no prefix / basePath / assetPrefix) ported from Next.js e2e suites. Updates 11 source files + 12 test files; 914+ tests pass in the touched suites (asset-prefix, app-router, pages-router, deploy, font-google, static-file-cache, serve-static, precompress, features, routing, isr-cache, build-optimization, app-rsc/ssr, standalone, middleware, shims). Ported from Next.js: - test/e2e/invalid-static-asset-404-app/*.test.ts - test/e2e/invalid-static-asset-404-pages/*.test.ts * fix(e2e): update cloudflare-pages-router hydration spec for _next/static * PR #1411 approved. Clean fix, 2 nits. Co-authored-by: james-elicx <james-elicx@users.noreply.github.com> * refactor(assets): derive isHashed checks from ASSET_PREFIX_URL_DIR Replace hardcoded '_next/static' string literals in runtime code paths with the ASSET_PREFIX_URL_DIR constant so the asset URL convention has a single source of truth. - server/prod-server.ts isHashed: pathname.includes(`/${ASSET_PREFIX_URL_DIR}/`) - server/prod-server.ts public-dir guard: startsWith(`/${ASSET_PREFIX_URL_DIR}/`) - server/static-file-cache.ts isHashed: both startsWith + includes derived - plugins/fonts.ts DEFAULT_ASSETS_DIR = ASSET_PREFIX_URL_DIR Behavioural no-op — the constant is '_next/static' so the resolved strings are identical. Centralises the URL contract so future changes to ASSET_PREFIX_URL_DIR (or a refactor that derives it from build config) flow through to every consumer. Doc-comment references to '_next/static' left in place — they describe the canonical value for human readers and are not code paths. --------- Co-authored-by: ask-bonk[bot] <ask-bonk[bot]@users.noreply.github.com> Co-authored-by: james-elicx <james-elicx@users.noreply.github.com>	2026-05-21 20:36:04 +00:00
Nathan Nguyen	6da45a680a	perf: build-time precompression + startup metadata cache for static serving (#641) ... * perf: add build-time precompression for hashed static assets Generate .br (brotli q11), .gz (gzip l9), and .zst (zstandard l19) files alongside compressible assets in dist/client/assets/ during vinext build. These are served directly by the production server, eliminating per-request compression overhead for immutable build output. Only targets assets/ (hashed, immutable). Public directory files still use on-the-fly compression since they may change between deploys. * perf: add startup metadata cache for zero-syscall static serving StaticFileCache walks dist/client/ once at server boot and caches: - File metadata (path, size, content-type, cache-control, etag) - Pre-computed response headers per variant (original, br, gz, zst) - In-memory buffers for small files (< 64KB) for res.end(buffer) - Precompressed variant paths and sizes Per-request serving is Map.get() + res.end(buffer) with zero filesystem calls, zero object allocation, and zero header construction. Modeled after sirv's production mode but with in-memory buffering for small files which eliminates createReadStream fd overhead. * perf: refactor tryServeStatic to async + cache + precompressed serving Refactor tryServeStatic to use StaticFileCache for the hot path: - Pre-computed response headers (zero object allocation per request) - In-memory buffer serving for small precompressed files - 304 Not Modified via ETag + If-None-Match - HEAD request optimization (headers only, no body) - Zstandard serving (zstd > br > gzip > original fallback chain) - Async filesystem fallback for non-cached files (replaces blocking existsSync + statSync) - Skip decodeURIComponent for clean URLs (no % in path) Wire StaticFileCache.create() into both startAppRouterServer and startPagesRouterServer at startup. Integrate precompressAssets() into the vinext build pipeline. CONTENT_TYPES is now a single source of truth exported from static-file-cache.ts (was duplicated in prod-server.ts). * docs: fix stale comments in precompress (mention .zst) * fix: deduplicate buffer reads for HTML alias entries, fix stale JSDoc * docs: fix stale comment in cli.ts (mention zstd) * fix: address code review findings - Add Vary: Accept-Encoding to original variant when compressed siblings exist - Use pipeline() instead of .pipe() for stream error safety - Remove legacy accessor fields from StaticFileEntry (all new code) - Exclude Content-Type from 304 responses per RFC 9110 - Fix misleading Accept-Encoding comment in prod-server - Batch stat() calls in walkFilesWithStats (chunks of 64) - Bound precompression concurrency to availableParallelism (max 16) - Gracefully skip zstd when zlib.zstdCompress unavailable (Node <22.15) - Add --no-precompress CLI flag to skip build-time compression - Drop brotli to q5 (fast, nearly same ratio as q11 at build time) - Max out zstd at level 22 (build time, no reason to hold back) * fix: address remaining review findings in prod-server - Gate negotiateEncoding zstd on runtime availability (HAS_ZSTD) - Add ZSTD_e_flush for streaming mode (progressive SSR decodability) - Serve HEAD without body on slow path (compressed + uncompressed) - Handle pipeline errors instead of silently swallowing them - Lowercase Accept-Encoding in fast path per RFC 9110 - Fix stale comments: add zstd to module header and variant detection - Add tests for slow path, HEAD, and malformed percent-encoded pathnames * perf: move precompression to Vite plugin with edge auto-detection Move build-time precompression from CLI post-step into a writeBundle plugin hook (vinext:precompress). The plugin already knows the deployment target — auto-skips for Cloudflare Workers and Nitro where the CDN handles compression at the edge. - Add precompress option to VinextOptions: 'auto' | boolean - CLI --no-precompress forwards via VINEXT_NO_PRECOMPRESS env var - Works for any build invocation (CLI, API, CI), not just vinext build * docs: fix inaccurate comments in precompress and static-file-cache - precompress: remove false idempotency claim (overwrites, not skips) - static-file-cache: "frozen objects" → "reusable objects" (no freeze) - static-file-cache: fix unrealistic compression ratio example * fix: address code review findings - Remove dead code: redundant .br/.gz/.zst extension checks already caught by COMPRESSIBLE_EXTENSIONS filter - Fix unreachable pipeline error handlers: res.writeHead() was called before pipeline(), so res.headersSent was always true in the callback. Use res.destroy(err) to cleanly tear down the socket on stream errors. - Read file content lazily inside compression chunks instead of loading all files upfront, keeping peak memory at O(CONCURRENCY * max_file_size) * perf: tune compression levels and restore auto mode - Drop zstd from level 22 to 8 (250x faster build, 7% larger output) - Drop gzip from level 9 to 8 (marginal size difference, faster) - Restore precompress: 'auto' | boolean config with auto as default (auto-enables for Node.js targets, skips for Cloudflare/Nitro) - Add progress bar for precompression step (matches prerender UX) - Clarify compression ratio output as brotli-specific * fix: use filename hash for ETag on hashed assets Hashed assets (assets/app-DqZc3R4n.js) now derive their ETag from the content hash in the filename instead of mtime. More robust across Docker builds, CI artifacts, and cp -r where mtime changes between deploys even when content is identical. - Use weak validator (W/) since the tag is shared across content-encoded variants (.br, .gz, .zst) which are byte-different (RFC 9110 §8.8.1) - Fall back to mtime-based ETag when filename has no recognizable hash (custom assetFileNames, plugin-emitted assets, public/ files) * fix: make precompress opt-in per maintainer feedback Real-world benchmark shows 13.5% build-time regression on fumadocs with auto mode (2.4s → 2.7s). Switch to explicit opt-in via `precompress: true` or `--precompress` CLI flag until the feature stabilises. * fix: address code review findings - ETag regex: use lastIndexOf('-') to extract only the final hash segment, avoiding capture of filename fragments in multi-hyphenated names (e.g. `before-pop-state-D3R3Ien6` now extracts `D3R3Ien6`) - Progress bar: track processed count (including skipped files) so the bar always reaches 100% - Tests: update ETag test for filename-hash behavior, add coverage for mtime fallback on non-hashed and unhashed-in-assets files * chore: use type instead of interface per lint rule * fix final precompression review comments * perf: optimize precompress build path - Overlap precompression with SSR build phase (async writeBundle + closeBundle await) - Lower concurrency cap from 16 to 8 (reduces zlib queue contention) - Fix progress counter race in chunked processing * fix: address final review feedback - Fix progress counter race in precompress (single onProgress call site after readFile, monotonic by construction) - Add console.error before res.destroy in all pipeline error handlers for operational visibility - Chunk buffer reads at 64 concurrent to bound FD usage on large projects - Defer createCompressor until after HEAD check on slow path - Use Math.floor for mtime ETag (matches send/Express convention) - Document Rolldown hash length assumption * fix: address final bonk review nits - Move progress reporting to after chunk completion to avoid non-deterministic ordering within Promise.all - Use Math.trunc instead of Math.floor for mtime-based ETags (semantically correct for truncating to whole seconds) * address bonk review comments: counters post-write, Math.floor, Content-Length comment, warn log level - precompress.ts: move filesCompressed++ and totalOriginalBytes+= to after writes complete so partial failures (e.g. ENOSPC) don't inflate totals - precompress.ts: add comment clarifying that progress tracks all files including skipped ones below MIN_SIZE, unlike filesCompressed - static-file-cache.ts: Math.trunc -> Math.floor for mtime-based ETags (matches send/Express convention; correct for pre-epoch mtimes) - prod-server.ts: add comment explaining Content-Length is intentionally omitted on slow-path compressed responses (chunked transfer encoding) - prod-server.ts: downgrade all three pipeline error logs from console.error to console.warn and add comments explaining the destroy-only pattern * address bonk: slow-path ETag+304 parity, visible precompress error log - prod-server.ts slow path: add mtime-based ETag (W/"size-mtime") and 304 Not Modified support, matching fast-path cache behaviour; also propagate mtimeMs through resolveStaticFile/statIfFile - index.ts: add console.error in precompress .catch() so build errors are visible even if the SSR closeBundle hook never fires * address bonk: slow-path 304 Vary header, HAS_ZSTD comment - prod-server.ts slow path: include Vary: Accept-Encoding in 304 responses when the content type is compressible, matching fast-path notModifiedHeaders behaviour and preventing shared caches from collapsing compressed/uncompressed representations - prod-server.ts fast path: add comment explaining HAS_ZSTD is not checked during precompressed variant selection — we're serving a pre-existing .zst file from disk, not invoking zstdCompress() at runtime; HAS_ZSTD only guards the slow-path on-the-fly compressor * address bonk: clarify Vary/compress=false intent in slow-path 304 comment Expand the 304 comment to explain that Vary: Accept-Encoding is intentionally omitted when compress=false (matching fast-path behaviour where compress=false skips all compressed variants), and note that spreading undefined is a deliberate no-op per ES2018+. * address bonk: readFile-before-stat comment, document VINEXT_PRECOMPRESS env var - precompress.ts: add comment on MIN_SIZE early-return explaining the read-first vs stat-first tradeoff (one wasted read vs extra syscall per tiny file; read-first cheaper for typical hashed asset sets) - index.ts: document VINEXT_PRECOMPRESS=1 env var in VinextOptions.precompress JSDoc alongside the existing --precompress CLI flag mention * address bonk: filename-hash ETag on slow path, entry-level dedup in buffer pass - static-file-cache.ts: export etagFromFilenameHash so prod-server can reuse the same filename-hash extraction logic - prod-server.ts: slow path now uses etagFromFilenameHash for /assets/* files, falling back to mtime-based ETag for non-hashed files; this matches fast-path cache behaviour and makes ETags stable across deploys - static-file-cache.ts buffer pass: deduplicate at the StaticFileEntry level (using a seenEntries Set) rather than at the variant path level; HTML aliases share the same entry by reference so the old approach iterated variants multiple times for sites with many HTML pages * address bonk: fix etagFromFilenameHash arg, explicit re-run safety comment - prod-server.ts: use resolved.path (not pathname) for etagFromFilenameHash so that ext and the hash extraction always come from the same file; pathname and resolved.path diverge after HTML fallback (e.g. a request for /assets/widget-abc123 resolving to widget-abc123.html would have extracted a hash from the wrong name) - precompress.ts: add inline comment on the COMPRESSIBLE_EXTENSIONS filter making explicit that .br/.gz/.zst are intentionally absent, so precompressed variants from a previous run are never re-compressed * address bonk: add slow-path 304 test coverage --------- Co-authored-by: James <james@eli.cx>	2026-04-01 23:33:12 +01:00