MuSTMM: Learning Sound-Triggered Mobile Manipulation across Multiple Scenes
We introduce Multi-scene Sound-Triggered Mobile Manipulation (MuSTMM), a dataset that combines spatial audio rendering and physical interaction across 31 scenes with diverse architectures and object layouts.
ProofPaper ↗
Key points
- Sound-Triggered Mobile Manipulation (STMM) requires robots to respond to sound events by navigating to the associated objects and performing appropriate physical interactions.
- We further augment the Navigation Policy Model with explicit spatial memory that complements acoustic cues with observed geometry.
- Fusing map features with sound context produces a goal descriptor that provides both target-location and scene-structure priors, guiding navigation to the target region required for subsequent physical interaction.
- MuSTMM and the navigation policy provide a testbed for the new challenge of studying sound-triggered mobile manipulation in unseen scenes.
Sources (1)
- [1]MuSTMM: Learning Sound-Triggered Mobile Manipulation across Multiple ScenesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 07:40 AM
We introduce Multi-scene Sound-Triggered Mobile Manipulation (MuSTMM), a dataset that combines spatial audio rendering and physical interaction across 31 scenes with diverse architectures and object layouts.
Sound-Triggered Mobile Manipulation (STMM) requires robots to respond to sound events by navigating to the associated objects and performing appropriate physical interactions.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026openai/openai-agents-python v0.24.0
- Oct 14, 2026mistralai/Timestral-209M
- Oct 14, 2026Cortex: Consolidating Context into Parametric Memory
- Oct 14, 2026ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models
- Oct 14, 2026From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents
- Oct 13, 2026Gemini 4 Argon for Robotics