ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

Key points

• We measure up to a 4.1-fold increase in energy for the same four-sample query as GPU load rises.1
• We present ATEST, a resource-aware TTS framework that jointly considers the expected value of additional samples and their current GPU cost.1
• ATEST-Online, its default mode, requires no offline calibration: it incrementally generates samples, estimates the stability of the plurality answer, monitors live GPU conditions, and learns step energy cost online to decide when to stop.1
• We evaluate both modes on four reasoning and question-answering datasets using Phi-3-mini on an NVIDIA A100 under controlled varying load, and test transfer to an NVIDIA L40S and Qwen2.5-3B.1

Sources (2)

• [1]ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 08:23 PM

  Test-time scaling (TTS) improves large language model (LLM) reasoning by sampling multiple answers, but fixed sampling budgets ignore both query difficulty and the changing energy cost of inference under GPU contention.

  We measure up to a 4.1-fold increase in energy for the same four-sample query as GPU load rises.

• [2]ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference
  Hugging Face Daily Papers · Oct 18, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.