HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control

Key points

• Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet adapting generalist vision-language-action (VLA) policies to real-world humanoid loco-manipulation remains challenging.1
• Built upon a pretrained flow-matching VLA, HAF-VLA inherits the vision-language prefix encoder and action expert weights, and adapts them to humanoid whole-body loco-manipulation with learnable stage embeddings and cross-stage KV-cache conditioning.1
• To preserve structural dependencies across stages, we introduce a cross-stage KV cache conditioning mechanism, enabling subsequent action generation stages to refine and augment their outputs based on the action representations from preceding stages.1
• Experimental results across six real-world tasks demonstrate that our framework significantly enhances the performance of existing models in humanoid loco-manipulation tasks, demonstrating an effective pathway for adapting strong mobile-manipulation VLAs to humanoid whole-body loco-manipulation.1

Sources (1)

• [1]HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 03:33 PM

  We propose HAF-VLA, a hierarchical action-flow framework for humanoid VLA control.

  Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet adapting generalist vision-language-action (VLA) policies to real-world humanoid loco-manipulation remains challenging.

Extractive summary: sentences quoted from the sources.