Multimodal Reranking Is a System One Decision
Multimodal rerankers are usually dedicated models: a vision-language model (VLM) is trained on retrieval data to score the candidates of a query.
ProofPaper ↗
Key points
- We therefore argue that a general VLM can serve as a multimodal reranker through System One decisions, and that decision tuning can improve its reranking.
- On MMEB-V2 image retrieval, this choice scores 5.4 to 7.3 hit@1 above a yes/no question per candidate, and the yes/no question falls below the first stage.
- With the choice, zero-shot Qwen3.8-27B beats the supervised Qwen3-VL-Reranker-2B on all three benchmarks and comes within 1.6 points of Reranker-8B on each.
- JEV-27B-VL, the same backbone tuned on text decisions only, improves page reranking by 0.6 to 1.1 points, with gains on most tasks (sign test, p <= 0.017), and leaves natural images unchanged.
Sources (1)
- [1]Multimodal Reranking Is a System One DecisionHugging Face Daily Papers · Oct 20, 12:00 AM
Multimodal rerankers are usually dedicated models: a vision-language model (VLM) is trained on retrieval data to score the candidates of a query.
We therefore argue that a general VLM can serve as a multimodal reranker through System One decisions, and that decision tuning can improve its reranking.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Not Every Rollout Needs Distillation: Correctness-Filtered On-Policy Self-Distillation
- Oct 19, 2026TorchNeuron: running HF models on AWS Trainium is easier than ever
- Oct 18, 2026Divergence based Ranked Preference Alignment for LLMs
- Oct 18, 2026P(Yes): 15$\times$ Improvements via Test-Time Scaling of Decision Models
- Oct 18, 2026System X: An LLM Decision Model That Replies from Its Own KV Cache
- Oct 17, 202620 Jev Alternatives: The Decision Models Taking On TypeSafe in 2026