Jun Yamog 1de83f91c3 Fix OOM regression in _apply() for quantized models during inference (#13372) ...

Skip unnecessary clone of inference-mode tensors when already inside
torch.inference_mode(), matching the existing guard in set_attr_param.
The unconditional clone introduced in 20561aa9 caused transient VRAM
doubling during model movement for FP8/quantized models.

2026-04-15 02:10:36 -07:00

52 KiB

Raw Blame History

View Raw