Listen Before Act: Audio-Conditioned Robot Manipulation with ListenVLA
To address this gap, we present ListenVLA, an audio-conditioned VLA policy built on GR00T N1.7 for sound-guided manipulation.
ProofPaper ↗
Key points
- Vision-language-action (VLA) models have advanced robotic manipulation by connecting visual perception and language instructions to action generation.
- ListenVLA combines semantic audio embeddings with waveform-derived acoustic cues over a causal audio history and conditions continuous action generation through audio tokens and feature modulation.
- Moreover, we introduce an audio-action binding objective that uses context-matched demonstrations to link auditory differences to demonstrated action differences.
- Furthermore, we develop LIBERO-Audio, an eight-task benchmark covering sound-based target selection, responses to auditory events, and physical interactions that reveal acoustic evidence.
Sources (1)
- [1]Listen Before Act: Audio-Conditioned Robot Manipulation with ListenVLAarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 07:38 AM
To address this gap, we present ListenVLA, an audio-conditioned VLA policy built on GR00T N1.7 for sound-guided manipulation.
Vision-language-action (VLA) models have advanced robotic manipulation by connecting visual perception and language instructions to action generation.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers
- Oct 12, 2026AUGUstus: Autonomous Driving VLA Model with Unified Generation and Understanding
- Oct 11, 2026$Δ$ Flow Matching for Robot Foundation Models
- Oct 11, 2026A Persian Vision-Language-Action Dataset and Benchmark for Ambiguity- and Infeasibility-Aware Instruction-to-Plan Grounding
- Oct 11, 2026AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
- Oct 10, 2026Verify, Act, Distill: Turning Test-Time Compute into Test-Time Adaptation for Vision-Language-Action Models