RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation
To address this gap, we introduce RoboJudge-Bench, a benchmark for evaluating MLLMs as judges of embodied video generation.
ProofPaper ↗
Key points
- Video generation models are promising foundations for embodied world models, but only if their generated rollouts are physically valid and faithful to task instructions.
- Multimodal large language models (MLLMs) are increasingly used to judge these properties, yet their agreement with human judgments on embodied videos has not been systematically evaluated.
- Motivated by these findings, we construct RoboJudge-Train, comprising 12K videos annotated with 25K human-verified scores and model-assisted rationales for Physical Adherence and Instruction Alignment.
- We further use RoboJudge-9B as a downstream evaluator to benchmark video generation models according to Physical Adherence and Instruction Alignment.
Sources (1)
- [1]RoboJudge: Multimodal Language Models as Judges for Embodied Video GenerationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 07:48 AM
To address this gap, we introduce RoboJudge-Bench, a benchmark for evaluating MLLMs as judges of embodied video generation.
Video generation models are promising foundations for embodied world models, but only if their generated rollouts are physically valid and faithful to task instructions.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026DiffResearch: Accelerating Deep Research with Diffusion Language Models
- Oct 20, 2026AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics
- Oct 19, 2026TEX: Test-Time Scaling Testing Agents via Execution-based Cross-Validation
- Oct 19, 2026MiniMax M3.1 Flash shows China's AI price race has no floor in sight
- Oct 19, 2026Think, Compress, Answer: Reinforcement Learning for Multimodal Grounding via Reasoning Compression
- Oct 19, 2026HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL