Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning.
ProofPaper ↗
Key points
- However, if the basket is shifted five centimeters to the left, it drops the soup off the table.
- We developed an interpretability pipeline to investigate what such a policy is computing.
- Injecting task A's activations into the action expert while the policy looks at task B's scene makes pi0.5 execute task A's motor trajectory in 99.6\% of episodes (n{=}1{,}968).
- The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report.
Sources (1)
- [1]Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action ModelsHugging Face Daily Papers · Oct 20, 12:00 AM
A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning.
However, if the basket is shifted five centimeters to the left, it drops the soup off the table.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control
- Oct 19, 2026TorchNeuron: running HF models on AWS Trainium is easier than ever
- Oct 19, 2026ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
- Oct 18, 2026Comfy-Org/ComfyUI v0.42.0
- Oct 18, 2026GPT-2 KV cache from scratch: correctness, latency, and memory measurements [P]
- Oct 18, 2026VLA-Rolling: Fewer Denoising Steps, Better Control via Adaptive Action Reuse