Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation.
Key points
- Multimodal agents commonly generate freeform reasoning before each action.
- Using pre-specified reasoning traces enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache.
- We evaluated SSR on seven multimodal search benchmarks using 2B and 4B models.
- Across multiple reinforcement learning objectives and supervised fine-tuning, SSR delivers significant efficiency gains without sacrificing task performance.
Sources (1)
- [1]Selection-Based Structured Reasoning: Toward Efficient Multimodal Search AgentsApple Machine Learning Research · Oct 15, 12:00 AM
To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation.
Multimodal agents commonly generate freeform reasoning before each action.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026How to Train Your Little Dragon: Teaching a Small Language Model to Construct and Refine GPU Optimization Scripts
- Oct 14, 2026Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning
- Oct 14, 2026PN-Mix: Pair-Norm Guided Mixed-Precision Key Cache Quantization for Efficient Long-Output LLM Inference
- Oct 14, 2026The Design Space of Tri-Modal Masked Diffusion Models
- Oct 13, 2026PMO-Dock: Benchmarking Docking, Specificity, and Generalization in Molecular Optimization
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B