UniWALM: Unified World Action Language Model for Embodied Robot Control

Key points

• We ask: can jointly generating language enrich this shared representation, improving manipulation and supporting memory-dependent behavior?1
• To keep the action stream reliable under this richer objective, we further introduce Flow Matching with Correction (FMC), a training objective that teaches recovery from perturbed action trajectories at no inference-time cost.1
• UniWALM consistently improves over same-backbone baselines (e.g., FastWAM) on compositional pick-and-place (MolmoSpaces) and across the memory-dependent subset of RMBench, where it reaches 34.5% without any pretrained language or reasoning model and exceeds Mem-0's keyframe-equipped setup (28.5%) using only a single anchor frame.1
• On a real-world Find-the-Item task, generating textual memory lifts success from 11.1% to 61.1%, while state-of-the-art VLAs stay at or below chance (<=33.3%).1

Sources (1)

• [1]UniWALM: Unified World Action Language Model for Embodied Robot Control
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:59 PM

  Motivated by the human ability to integrate experience across language, perception, and action, we introduce UniWALM, a unified world-action-language model that learns video, action, and language in a single shared representation, where each modality is both a conditioning signal and a supervision target.

  We ask: can jointly generating language enrich this shared representation, improving manipulation and supporting memory-dependent behavior?

Extractive summary: sentences quoted from the sources.