HAVANA: A Hierarchical Video-Action Model for Vision-Language Navigation

Key points

• Video-action models (VAMs) offer a promising way to learn predictive representations for navigation.1
• However, existing VAMs jointly predict visual futures and local trajectories in a single process, even though they operate at different paces: future prediction remains stable across multiple updates while local trajectories must react to fresh observations.1
• To bridge this gap, we introduce HAVANA, a dual-system VAM that decouples predictive reasoning from reactive local planning.1
• Experiments on OmniNavBench demonstrate state-of-the-art navigation performance over prior baselines, while real-world deployments further show robust generalization to unseen environments across different robot embodiments.1

Sources (1)

• [1]HAVANA: A Hierarchical Video-Action Model for Vision-Language Navigation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 06:48 PM

  Vision-Language Navigation (VLN) requires long-horizon instruction-consistent planning and short-horizon local planning in current geometry.

  Video-action models (VAMs) offer a promising way to learn predictive representations for navigation.

Extractive summary: sentences quoted from the sources.