TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control

Key points

• We present TS-WAM, a lightweight Teacher-Student World-Action Model that addresses these challenges through self-supervised temporal learning and explicit future prediction.1
• By coupling vlm-jepa with delta-lwm, TS-WAM learns to predict latent actions from the current observation and instruction and uses them to predict future visual features that guide action generation at inference.1
• A unified end-to-end training architecture jointly optimizes temporal representation learning, visual dynamics, and robot control without a separate latent-action or world-model pretraining stage.1
• Experiments on LIBERO, LIBERO-Plus, and SimplerEnv, together with real-world manipulation experiments on Franka Research 3 and RevoTron, demonstrate that TS-WAM achieves strong manipulation performance, generalization, and robustness.1

Sources (1)

• [1]TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 02:05 PM

  Many vision-language-action policies rely on action-labeled robot demonstrations and lack an explicit mechanism for predicting future visual states.

  We present TS-WAM, a lightweight Teacher-Student World-Action Model that addresses these challenges through self-supervised temporal learning and explicit future prediction.

Extractive summary: sentences quoted from the sources.