Peak-Memory Schedules You Can Trust: Exact Realization, Certified Optimality, and the Limits of Recomputation for Inference
Real-time inference on small devices needs low activation memory without extra latency, which makes operator order the most valuable memory lever: it is free in time.
ProofPaper ↗
Key points
- We show that the usual way of deploying a better order, handing it back to the compiler and regenerating code, is unreliable.
- We prove that this realizes the modeled peak whenever allocation does not depend on history.
- On the GPU we characterize why the caching allocator breaks the condition.
- Beyond the certified optimum we study selective recomputation with per-operator caps.
Sources (1)
- [1]Peak-Memory Schedules You Can Trust: Exact Realization, Certified Optimality, and the Limits of Recomputation for InferencearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 11:31 AM
Real-time inference on small devices needs low activation memory without extra latency, which makes operator order the most valuable memory lever: it is free in time.
We show that the usual way of deploying a better order, handing it back to the compiler and regenerating code, is unreliable.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 16, 2026One Communicator, Three Vendors: Extending the MoA Predictive Theory to a Real NVIDIA+AMD+CPU MPI Ensemble
- Oct 15, 2026Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint
- Oct 15, 2026REKI: Registering Knowledge in VLAs
- Oct 15, 2026When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel Prefill
- Oct 14, 2026Imperfect Is Perfect: Learned Column Prefilling for Branch-Price-and-Cut
- Oct 14, 2026NVIDIA/TensorRT-LLM v1.4.0rc0