ResearchResearch paperRobotics & Embodied AI · Multimodal Models · Computer Vision2 sources · Oct 21, 2026

ALPS: Visual State Encoders for World Models

World models turn visual representations into predictions and actions.

We argue that their visual foundation should combine physical-state information, native language alignment, and pixel reconstruction, so that a single representation can support prediction, instruction, and inspection.
However, existing encoders typically specialize in only one or two of these capabilities; we therefore introduce ALPS, a pretrained visual state encoder that unifies all three in a single set of patch tokens.
Using only still images and text, we pretrain a ViT-B/16 from scratch with language, reconstruction, and spatial correspondence objectives, and then refine it via cross-view self-distillation.
In a unified evaluation across fifteen public encoders, ALPS achieves the lowest aggregate LIBERO state-probe error among ViT-B models (matching the 3.5x larger V-JEPA 2-L) while maintaining strong language alignment and accurate pixel reconstruction.

[1]ALPS: Visual State Encoders for World Models
Hugging Face Daily Papers · Oct 21, 12:00 AM
World models turn visual representations into predictions and actions.
We argue that their visual foundation should combine physical-state information, native language alignment, and pixel reconstruction, so that a single representation can support prediction, instruction, and inspection.
[2]ALPS: Visual State Encoders for World Models
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 05:57 PM

Extractive summary: sentences quoted from the sources.

Before this