VITAL: From Visual Motion to Precise Action
We therefore introduce VITAL (Vision-Induced Tokens for Action Learning), an action tokenizer that couples a visually derived motion scaffold with a compact residual encoding the trajectory error left by that scaffold.
ProofPaper ↗
Key points
- World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined independently of visual dynamics.
- This motivates deriving action targets directly from visual motion.
- We first examine a simpler setting in which trajectories are reconstructed from recorded future images.
- Using flow matching, the action expert jointly predicts both components from the current image, ego state, and navigation command, without generating future images at inference.
Sources (2)
- [1]VITAL: From Visual Motion to Precise ActionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 05:59 PM
We therefore introduce VITAL (Vision-Induced Tokens for Action Learning), an action tokenizer that couples a visually derived motion scaffold with a compact residual encoding the trajectory error left by that scaffold.
World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined independently of visual dynamics.
- [2]VITAL: From Visual Motion to Precise ActionHugging Face Daily Papers · Oct 14, 12:00 AM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026What Matters in Quality-Aware VLA Pretraining: A Benchmark and Empirical Study
- Oct 13, 2026SonicBoom: 52 tok/s decode on a laptop RTX 3050 (llama.cpp: 45) with the same quantized kernels, the difference is a graph compiler. No Python.
- Oct 13, 2026LinMorph: Learning to Morph Weights for Efficient Video DiT Linearization
- Oct 13, 2026MissPath-FM: Flow Matching with Structured Priors for Partially Observed Time Series
- Oct 13, 2026Scaling Online Optimal Transport Minibatches for Flow Matching
- Oct 13, 2026DeltaTTG: Training Multi-Turn Agents with a Token-to-Go Progress Meter