Files
Joseph fbe0d56219 docs: rewrite the bots and crawlers section in the caching guide (#98782)
## Summary

The Bots and crawlers section in the caching guide said that all bots
and crawlers skip the static shell because "they need a complete
document". Only HTML-limited bots do, and the reason is metadata: with
Partial Prerendering, `generateMetadata` can resolve dynamically and
stream in after the shell, which puts the tags in the body rather than
the `<head>`. Bots that need them in the `<head>` get a dynamic render
instead.

The second paragraph now states the consequence directly: if the
build-time prerender read something unreachable at request time, the
render throws and those bots get a 500. Which user agents this covers,
and the `htmlLimitedBots` option for changing the list, stay owned by
the Streaming guide, which the section links to.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-09-17 13:17:59 +00:00
..
2026-05-27 18:25:48 +00:00