### What
When PPR is off, app router prefetches will render the component tree up
until it encounters a `loading` segment, at which point it'll just
return some metadata about the segment and won't do any further
rendering. This is an optimization to ensure prefetches are lightweight
and don't potentially invoke expensive dynamic subtrees. However,
there's a bug in this logic that is causing it to bail unexpectedly if a
segment deeper in the tree contained a `loading.js` file.
This would mean the loading state wouldn't be triggered until the second
request for the full RSC data is initiated, resulting in an unintended
delta between when a link is clicked and the loading state is shown.
### Why
The `shouldSkipComponentTree` flag was incorrectly being set to `true`
even if the `loading.js` segment appeared deeper in the tree. Prefetch
requests from the client will always contain `FlightRouterState`, so the
logic to check for loading deeper in the tree will always be missed.
### How
This removes the `flightRouterState` check as it doesn't make sense:
prefetches will currently _always_ include the `flightRouterState`,
causing this to always short-circuit. I believe that this check is
vestigial from when we were generating static `prefetch.rsc` outputs
which is no longer the case.
This mirrors a [similar
check](https://github.com/vercel/next.js/blob/b87d8fc49983a3be568517d7ae14087749bb8ce3/packages/next/src/server/app-render/create-component-tree.tsx#L393)
when determining if parallel route(s) should be rendered.
Reland #54403
Also modifies the implementation of #55950 to not change the prefetch behavior when there is flight router state - we should only check the entire loader tree in the static prefetch case, otherwise we're inadvertently rendering the component tree for prefetches that match the current flight router state segment. ([slack x-ref](https://vercel.slack.com/archives/C03S8ED1DKM/p1695862974745849))
This includes a few other misc fixes for static prefetch generation:
- `next start` was not serving them (which also meant tests weren't catching a few small bugs)
- the router cache key casing can differ between build and runtime, resulting in a bad cache lookup which results suspending indefinitely during navigation
- We cannot generate static prefetches for pages that opt into `searchParams`, as the prefetch response won't have the right cache key in the RSC payload
- Layouts that use headers/cookies shouldn't use a static prefetch because it can result in unexpected behavior (ie, being redirected to a login page, if the prefetch contains redirect logic for unauthed users)
Closes NEXT-1665
Closes NEXT-1643
Sebastian "Sebbie" Silbermann