MemAct: Streaming Spatial Understanding with Memory and Active Exploration
We introduce MemAct, a framework for streaming spatial understanding with memory and action.
ProofPaper ↗
Key points
- Spatial understanding is essential for embodied agents navigating and interacting with the physical world.
- Yet streaming understanding must also be closed-loop: task-relevant evidence may require both recalling past observations and actively acquiring new views.
- We develop the MemAct suite, a data generation pipeline and collection across real and simulated environments, with approximately 1M training examples for spatial reasoning, memory writing, retrieval, and exploration.
- Experiments show that MemAct outperforms existing spatial understanding and streaming baselines and approaches frontier multimodal models on MemAct-bench while retaining streaming inference efficiency.
Sources (1)
- [1]MemAct: Streaming Spatial Understanding with Memory and Active ExplorationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 03:08 AM
We introduce MemAct, a framework for streaming spatial understanding with memory and action.
Spatial understanding is essential for embodied agents navigating and interacting with the physical world.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026Releasing the Inner Jev: A Simple Recipe for Open Decision Models
- Oct 12, 2026REEL: Learning to Retrieve, Evaluate, and Evolve Visual Memory for Any Long-Horizon Multi-Shot Video Generation
- Oct 12, 2026PIGEONS: Incremental Long-Video Captioning with Persistent Identity Gallery and Evolving Overarching Narrative Summary
- Oct 12, 2026InfoVis: Unified Slide Image Generation and Editing
- Oct 11, 2026MoMaTune: Model, Match, Then Tune Language Models for Numerical Records
- Oct 11, 2026Data-Centric Recursive Self-Improvement for Navigation via Policy-Synchronized Language-Action Dual Bootstrapping