VLA-Retain: Benchmarking Cross-Dimension Effects of Vision Tuning in VLAs

Key points

• Therefore, we introduce VLA-Retain, a benchmark that compares five open VLA models, tuned on Camera-view or Sensor-noise data, with their own base checkpoints on LIBERO-Plus dataset spanning seven perturbation dimensions.1
• First, targeted vision tuning is not local, and its target gain does not predict retention.1
• Fourth, tuning strategies are model-dependent, as DELTA-style feature preservation raises X-VLA's mean off-target gain to $+1.2$ points but lowers OpenVLA-OFT's to $-23.1$.1
• Overall, these findings show that target-domain success alone is insufficient and highlight cross-dimensional retention as a key criterion for targeted VLA adaptation.1

Sources (1)

• [1]VLA-Retain: Benchmarking Cross-Dimension Effects of Vision Tuning in VLAs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:17 AM

  Vision-language-action (VLA) models transfer knowledge from large-scale pretraining into robot control, but their manipulation performance remains sensitive to changes in observation conditions, which recent work addresses by fine-tuning the visual module on target-domain data.

  Therefore, we introduce VLA-Retain, a benchmark that compares five open VLA models, tuned on Camera-view or Sensor-noise data, with their own base checkpoints on LIBERO-Plus dataset spanning seven perturbation dimensions.

Extractive summary: sentences quoted from the sources.