Relational Spatial Alignment for Vision-Language-Action Models

Key points

• Spatial understanding is essential for robots to perceive their surrounding environments and execute actions.1
• Existing approaches typically encode 3D awareness into vision-language-action (VLA) models by directly aligning their representations with pre-trained 3D foundation models.1
• However, we observe that such direct geometric feature alignment essentially exploits 3D foundation priors as a form of low-rank regularization in the feature space, resulting in severe rank collapse of the pre-trained VLA representations.1
• Extensive experiments on public benchmarks, our new spatial reasoning benchmark, and real-world manipulation scenarios demonstrate that our approach effectively incorporates 3D spatial representations into VLA reasoning, achieving robust performance and enhanced spatial awareness across diverse manipulation tasks.1

Sources (1)

• [1]Relational Spatial Alignment for Vision-Language-Action Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 02:08 PM

  We propose a representation-alignment method that enables the injection of 3D spatial awareness without compromising the model's original expressivity.

  Spatial understanding is essential for robots to perceive their surrounding environments and execute actions.

Extractive summary: sentences quoted from the sources.