## Summary
Builds on #97591. Webpack currently removes most of `.next` on devserver
startup. We can remove more as well.
This keeps the age-based removal introduced in #97591 and expands it to
`traces`, logs, etc. However, for any other directory that is not
`cache` or `lock`, completely remove it on startup. This includes types,
etc.
Apply age-based deletion to Turbopack-owned output that can be restored
from cache or recomputed, including route entries, chunks, assets,
service workers, middleware, and instrumentation. Completely remove
other dev output so logs, traces, generated types, diagnostics, and
future framework-owned output cannot be left as mixed generations.
Use exact-path exclusion sets in `recursiveDeleteSyncWithAsyncRetries`.
The complete pass retains `cache`, `lock`, and Turbopack-owned paths;
the age-based pass then sweeps those Turbopack paths. Both passes finish
before project creation.
## Verification
- `pnpm test-unit test/unit/recursive-delete.test.ts`
- `pnpm --filter=next types`
---------
Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Will Binns-Smith <755844+wbinnssmith@users.noreply.github.com>
## Summary
Turbopack dev never removes the content-hashed chunks it supersedes, so
`.next/dev` grows without bound. Webpack dev avoids this by wiping
`distDir` on every start.
Adds a `maxAgeMs` argument to `recursiveDeleteSyncWithAsyncRetries`:
delete files whose mtime is at least that old, remove directories once
empty. Turbopack dev passes one week.
## Verification
- `pnpm test-unit test/unit/recursive-delete.test.ts`
- ~Not run:~ dev-server behavior against a real `.next/dev`
<!-- NEXT_JS_LLM -->
**Update:** This also fixes a bug pointed out by Graphite where the deletion function could hang forever because the `t` variable used for tracking attempts wasn't properly incremented.
---
https://vercel.slack.com/archives/C04KC8A53T7/p1759360239193629
This is a critical path for `next dev` and `next build` initialization.
On Windows, it's quite common to get an error while trying to recursively delete directories for one of two reasons:
**Temporarily open files:**
1. You open the directory.
2. The user's AV detects the directory as opened and quickly scans the files in that directory.
3. You try to delete a file, but on Windows, files cannot be deleted when open.
**Deletion is not instant/atomic:** It can take some time after deleting files for the directory to become empty.
For this reason, we perform two retries with a 100ms delay after the first attempt and a 200ms delay after the second attempt.
* Node does not use retries by default, but has an option to do so with a `100ms * attemptNumber` (linear backoff) delay, similar to what we do: https://github.com/nodejs/node/blob/b5638e1765d3d1241d07457e767f27e3ee6b9dd2/lib/internal/fs/utils.js#L774-L779
* Node used to use 3 retries by default, but that was changed in v14: https://github.com/nodejs/node/commit/7e85f068a48b284a253abcb5eb3173320869b8fe
* Rust tries to delete 64 times in a loop without sleeping, but with a call to `thread::yield_now`: https://github.com/rust-lang/rust/blob/4da69dfff1929cc79872b3d05ab7112d84753dba/library/std/src/sys/fs/windows/remove_dir_all.rs#L154-L171
In the vein of reducing time spent sleeping in the critical initialization path (@timneutkens), I think we should be more aggressive, I'm thinking using a 10ms delay (arbitrary number), but with an exponential backoff, so that we still retry over the same 300ms period as we currently do? Thoughts?
The difficulty is that this is on Windows, so it's hard for me to get good numbers about what impact this would have (if any).
This replaces the existing recursive directory reading beheviour with a more efficient implementation. Rather than previously doing actual recursion with the function calls to go deeper into each directory, this has been rewritten to instead use a while loop and a stack. This should improve memory usage for projects with very deep directory structures.
The updated design that performs directory scanning on all directories found at each layer, allowing more filesystem calls to be sent to the OS instead of waiting like the previous implementation did.
**Currently the new implementation is about 3x faster.**
Support top level static `robots.txt` and `sitemap.xml` as metadata
route in app directory. When those files are placed in top root
directory
Refactored a bit the page files matching logic, to reuse it between dev
server and build
Closes NEXT-267
* Move unit tests to one folder
* Migrate unit tests to TypeScript
* add test types to lint
* Ensure ts(x) tests are run with util
* Add tsx extension to jest config
* bump
Will Binns-Smith
