2 Commits

Author	SHA1	Message	Date
Luke Sandberg	236a76dd0f	[turbopack] Remove turbo_tasks::function from ModuleReference getters (#91229) ... ### What? Refactors the `ModuleReference` trait to make `chunking_type()` and `binding_usage()` methods return direct values instead of `Vc<T>` wrapped values, removing the need for async task functions. Also removes the `get_referenced_asset` task from `EsmAssetReference`, inlining its logic into the callers. ### Why? This change simplifies the API by eliminating unnecessary async overhead for methods that typically return simple, computed values. The previous implementation required `#[turbo_tasks::function]` annotations and `Vc<T>` wrappers even when the methods didn't need to perform async operations or benefit from caching. ### Impact | Metric | Base | Change | Delta | |--------|------|--------|-------| | Hits | 35,678,143 | 35,845,124 | **+166,981** | | Misses | 9,418,378 | 7,910,986 | **-1,507,392** | | Total | 45,096,521 | 43,756,110 | **-1,340,411** | | Task types | 1,306 | 1,277 | **-29** | 29 task types were removed, eliminating **2.6M total task invocations** (1.1M hits + 1.5M misses): - **`chunking_type`** — 21 task types removed across all `ModuleReference` implementors (~952k invocations) - **`binding_usage`** — 6 task types removed (~527k invocations) - **`BindingUsage::all`** — helper task removed (~36k invocations) - **`EsmAssetReference::get_referenced_asset`** — removed and inlined (~1.08M invocations: 628k hits + 451k misses) The removed `get_referenced_asset` hits reappear as +628k hits on `EsmAssetReference::resolve_reference` and `ReferencedAsset::from_resolve_result` (with zero increase in misses), confirming the work is now served from cache through the existing callers. No tasks had increased misses — the removal is clean with no cache invalidation spillover. I also ran some builds to measure latency ``` # This branch $ hyperfine -p 'rm -rf .next' -w 2 -r 10 'pnpm next build --turbopack --experimental-build-mode=compile' Benchmark 1: pnpm next build --turbopack --experimental-build-mode=compile Time (mean ± σ): 52.752 s ± 0.658 s [User: 376.575 s, System: 106.375 s] Range (min … max): 51.913 s … 54.161 s 10 runs # on canary $ hyperfine -p 'rm -rf .next' -w 2 -r 10 'pnpm next build --turbopack --experimental-build-mode=compile' Benchmark 1: pnpm next build --turbopack --experimental-build-mode=compile Time (mean ± σ): 54.675 s ± 1.394 s [User: 389.273 s, System: 114.642 s] Range (min … max): 53.434 s … 58.189 s 10 runs ``` so a solid win of almost 2 seconds MaxRSS also went from 16,474,324,992 bytes to 16,359,309,312 bytes (from one measurement) so a savings of ~100M of max heap size. ### How? - Changed `chunking_type()` method signature from `Vc<ChunkingTypeOption>` to `Option<ChunkingType>` - Changed `binding_usage()` method signature from `Vc<BindingUsage>` to `BindingUsage` - Removed `ChunkingTypeOption` type alias as it's no longer needed - Updated all implementations across the codebase to return direct values instead of wrapped ones - Removed `#[turbo_tasks::function]` annotations from these methods - Updated call sites to use `into_trait_ref().await?` pattern when accessing these methods from `Vc<dyn ModuleReference>` - Removed `EsmAssetReference::get_referenced_asset`, inlining its logic into callers - Added validation for `turbopack-chunking-type` annotation values in import analysis - Fixed cache effectiveness analysis script	2026-03-13 11:54:56 -07:00
Luke Sandberg	7cc4990f49	[turbopack] Track task durations in the task_statistics file (#83522) ... # Track task execution duration in TaskStatistics ## What? This PR adds tracking of task execution duration in Turbopack's task system, enabling better performance analysis and optimization. ## Why? Understanding how long tasks take to execute helps identify optimization opportunities, especially for determining whether caching is beneficial for specific tasks. ## How? - Added `track_task_duration` method to record execution time for tasks - Updated task statistics to track execution count and duration - Added a Python script `analyze_cache_effectiveness.py` to identify tasks that would benefit from removing caching - Updated tests to account for the new statistics fields Sample output from the script ``` Tasks ranked by estimated time savings from removing caching layer Savings Hit Rate Exec Time Operations Task Name --------------------------------------------------- 2.33s 39.5% 765ns 661,176 turbopack-ecmascript@turbopack_ecmascript::references::esm::base::EsmAssetReference::ChunkableModuleReference::chunking_type 2.29s 18.5% 1.6μs 490,488 turbopack-ecmascript@turbopack_ecmascript::references::esm::base::EsmAssetReference::ChunkableModuleReference::export_usage 1.99s 9.0% 9.8μs 430,149 turbopack@turbopack::ModuleAssetContext::AssetContext::resolve_asset 1.17s 51.7% 1.2μs 462,164 turbopack-ecmascript@turbopack_ecmascript::EcmascriptModuleAsset::ResolveOrigin::get_inner_asset 1.10s 54.1% 1.2μs 462,387 turbopack-core@turbopack_core::resolve::ModuleResolveResult::is_unresolvable 916.01ms 0.0% 19.2μs 152,669 turbopack@turbopack::apply_module_type 807.37ms 74.5% 1.1μs 722,106 turbopack-ecmascript@turbopack_ecmascript::references::esm::base::ReferencedAsset::from_resolve_result 782.16ms 69.7% 1.5μs 680,828 turbopack-core@turbopack_core::resolve::ModuleResolveResult::primary_modules 749.54ms 4.0% 80ns 129,625 turbopack-core@turbopack_core::ident::AssetIdent::new_inner 717.59ms 94.5% 5ns 887,040 turbopack-ecmascript@turbopack_ecmascript::EcmascriptModuleAsset::ResolveOrigin::asset_context 522.31ms 30.2% 1.7μs 136,180 turbopack-core@turbopack_core::resolve::ResolveResult::is_unresolvable 452.88ms 0.0% 5.2μs 75,484 next-core@next_core::next_server::resolve::ExternalCjsModulesResolvePlugin::AfterResolvePlugin::after_resolve 415.54ms 45.2% 937ns 134,377 turbopack-core@turbopack_core::resolve::pattern::Pattern::new_internal 388.03ms 0.0% 191.1μs 64,672 turbopack-ecmascript@turbopack_ecmascript::parse::parse ``` The script analyzes task statistics to find tasks where the overhead of caching exceeds the benefit, providing recommendations for optimization based on execution patterns. It leverages data from the overhead.rs benchmark which is also enhanced to provide an estimate on the delta between the measured duration and the actual duration. ## Conclusions? There are a few items of low hanging fruit but the real issue is `trait` items. We need to provide more flexibility to `value_trait` items to make it possible to have non-turbotask items that are `async`	2025-09-18 16:33:29 -07:00