Ask Less, Ask Better: Allocating User Attention in Proactive LLM Agents with Interruption Budgets
We propose an evaluation protocol that distinguishes how often agents ask from how well they ask.
ProofPaper ↗
Key points
- LLM agents can ask clarifying questions to complete tasks, but answering each question demands users' time and effort.
- Beyond question counts, this cost captures perceived effort: on conversation pairs with the same number of questions, human annotators agreed with the cost on which conversation required more effort 75% of the time.
- Across eight LLMs and four customer-service domains, limiting agents to one to three questions reduces user-attention cost by 36-73% while keeping 86-94% of the task reward they earn with unlimited questions.
- Models also differ sharply in how well they ask: at the same user-attention cost, the most efficient models gain more than twice as much reward from their questions as the others.
Sources (1)
- [1]Ask Less, Ask Better: Allocating User Attention in Proactive LLM Agents with Interruption BudgetsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 03:23 AM
We propose an evaluation protocol that distinguishes how often agents ask from how well they ask.
LLM agents can ask clarifying questions to complete tasks, but answering each question demands users' time and effort.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026sparqlite: SPARQL 1.1 inside SQLite, written in Rust
- Oct 16, 2026Ori: Drift-Aware Streaming $k$-PCA for Very Low-Rank LLM Training
- Oct 16, 2026OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners
- Oct 15, 2026Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
- Oct 15, 2026System1-Arena: Which Decisions Can Be Left to System 1?