MAGMA-Bench: Evaluating Long-Horizon Embodied Agents under Dynamic Tasks and Execution Failures
We introduce MAGMA-Bench, an architecture-agnostic benchmark for persistent task reasoning in dynamic multi-robot environments.
ProofPaper ↗
Key points
- Long-horizon embodied agents must maintain task-relevant information, coordinate heterogeneous robots, and adapt their decisions as execution unfolds.
- Existing benchmarks investigate dynamic task execution and embodied multi-agent coordination, but provide limited evaluation of their intersection.
- MAGMA-Bench evaluates agents across increasing task horizons and controlled online perturbations, including execution failures, task interruptions, and mission updates, with scenarios requiring coordination across robots with heterogeneous capabilities.
- We release the benchmark, 261,646 task-level training examples, and an architecture-agnostic expert-trace generation pipeline.
Sources (1)
- [1]MAGMA-Bench: Evaluating Long-Horizon Embodied Agents under Dynamic Tasks and Execution FailuresarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 12:54 PM
We introduce MAGMA-Bench, an architecture-agnostic benchmark for persistent task reasoning in dynamic multi-robot environments.
Long-horizon embodied agents must maintain task-relevant information, coordinate heterogeneous robots, and adapt their decisions as execution unfolds.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices
- Oct 9, 2026Latent World Action Model for GUI Agents
- Oct 9, 2026Rethinking Relative Policy Differences Transfer of On-Policy Distillation
- Oct 7, 2026reflection/Beam-1.0-501B-A23B
- Oct 7, 2026JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF
- Oct 7, 2026nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4-LiLiCorr