GeoMem-VLA: Internalizing 3D Scene Understanding into Temporal Memory for Vision-Language-Action Models
We introduce GeoMem-VLA, a VLA framework that internalizes explicit 3D supervision into dedicated geometric tokens and maintains them as temporal memory for action generation, without directly aligning the VLM's native visual embeddings with the teacher feature space.
ProofPaper ↗
Key points
- Geometric structure is essential for robotic manipulation and understanding the 3D world.
- Most VLA models rely on 2D representations of current observations, while geometry-aware approaches often require additional 3D information.
- During training, a frozen geometry foundation model supervises learnable GeoPlan tokens through a geometric flow-matching objective over the current scene point map and future end-effector tracks.
- These results suggest that internalizing and remembering 3D scene understanding is a promising approach to robot control.
Sources (1)
- [1]GeoMem-VLA: Internalizing 3D Scene Understanding into Temporal Memory for Vision-Language-Action ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 03:23 PM
We introduce GeoMem-VLA, a VLA framework that internalizes explicit 3D supervision into dedicated geometric tokens and maintains them as temporal memory for action generation, without directly aligning the VLM's native visual embeddings with the teacher feature space.
Geometric structure is essential for robotic manipulation and understanding the 3D world.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Perceives but Doesn't Brake: A Perception-Action Binding Gap in Reasoning Driving VLAs
- Oct 20, 2026From Predictive Geometry to Action: Learning Complementary Predictive Representations for Embodied Manipulation
- Oct 20, 2026PACT-VLA: Speculative Action-Chunk Verification with Pre-Commit Observations
- Oct 20, 2026PLaCA: Learning Patch-Level Future Latent Change for Action Generation
- Oct 20, 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
- Oct 20, 2026Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary Rigs