LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence

Key points

• World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection.1
• Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions.1
• Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling.1
• Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories.1

Sources (1)

• [1]LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 11:39 AM

  We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress.

  World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection.

Extractive summary: sentences quoted from the sources.