PAIR: Memory-Efficient and Fast Inference with Reading--Reasoning Fusion
For context-based question answering, using one large model for both evidence extraction and answer reasoning requires substantial memory and computation.
ProofPaper ↗
Key points
- Existing approaches trade accuracy for efficiency within a single model only.
- We propose a multi-model approach for efficient inference: our method, PAIR, employs two differently sized models for evidence extraction and answer reasoning, respectively.
- A smaller Reader model processes and retains the full context, while a larger Reasoner model sees only the question and generated sequence without directly attending to the context.
- At 45.7K-token contexts, PAIR achieves up to 3.2$\times$ higher end-to-end throughput than the full-context 32B model on the same GPU.
Sources (1)
- [1]PAIR: Memory-Efficient and Fast Inference with Reading--Reasoning FusionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 12:10 AM
For context-based question answering, using one large model for both evidence extraction and answer reasoning requires substantial memory and computation.
Existing approaches trade accuracy for efficiency within a single model only.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026Lightweight Scheduling Policies for Continuously Batched LLLM Serving Across GPU Architectures
- Oct 11, 2026Every Model That Can Be Run On 10-16GB VRAM Ranked
- Oct 10, 2026SAFE-KV: Causal Suffix Repair for Behavior-Preserving Non-Prefix KV Cache Reuse in RAG
- Oct 10, 2026Engineer / developer observations of Gemma4-31B, Qwen3.8-27B, and 6.1-Sol for software engineering work
- Oct 10, 2026Swarm Telepathy: Prefill-Free KV-Cache Transfer Across Families
- Oct 10, 2026Toward System One Clinical Intelligence: Parameter-Efficient Fine-Tuning and Counterfactual Explainability of Decision Foundation Models
