DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous Driving
In this work, we propose DriveWLA, a unified World-Language-Action model based on Mixture-of-Transformers (MoT) that jointly models semantic understanding, future scene dynamics, and trajectory generation.
ProofPaper ↗
Key points
- Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving.
- DriveWLA consists of three experts: a World Expert for future prediction, a Language Expert for semantic understanding, and an Action Expert for trajectory generation.
- We further introduce Cross-Expert Attention, which enables controlled information flow across experts while aligning semantic and dynamic representations.
- To improve trajectory generation beyond imitation learning, we apply Flow-GRPO reinforcement learning post-training to refine the Action Expert.
Sources (1)
- [1]DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous DrivingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 07:56 AM
In this work, we propose DriveWLA, a unified World-Language-Action model based on Mixture-of-Transformers (MoT) that jointly models semantic understanding, future scene dynamics, and trajectory generation.
Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching
- Oct 13, 2026PrT-Flow: Probability-Tractable Flow Matching for Reinforcement Learning
- Oct 13, 2026Fish Audio Drama 3 Technical Report
- Oct 12, 2026World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models
- Oct 12, 2026A Survey of Flow Matching in Reinforcement Learning
- Oct 11, 2026ObsSpec: Accelerating Agentic Reinforcement Learning with Observation Speculation