ObjDream: Guiding Vision-Language-Action Models with Object-Level Understanding and Future State Prediction

Key points

• To address these limitations, we introduce ObjDream, an object-centric VLA framework centered on object-level understanding and future object state prediction.1
• ObjDream uses a spatio-temporal object encoder to organize visual observations into temporal object slots, where persistent object identity and dynamic object state are represented through separate channels.1
• In addition, an action-conditioned object dynamics module predicts compact future object states relevant to manipulation during training, explicitly supervising object state evolution under robot actions without requiring future prediction at inference.1
• Experiments on LIBERO, LIBERO-Plus, and RoboTwin 2.0 show that ObjDream improves generalization under diverse distribution shifts and scene perturbations.1

Sources (1)

• [1]ObjDream: Guiding Vision-Language-Action Models with Object-Level Understanding and Future State Prediction
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 02:06 PM

  Vision-language-action (VLA) models have emerged as a powerful paradigm for robotic manipulation, enabling robots to generate actions from visual observations and language instructions.

  To address these limitations, we introduce ObjDream, an object-centric VLA framework centered on object-level understanding and future object state prediction.

Extractive summary: sentences quoted from the sources.