Reflect After Failure: Brain-Inspired Self-Evolving Agents for Embodied Spatial Intelligence
To address this, we propose HiMeco, a brain-inspired self-evolving agent for embodied spatial intelligence, inspired by metacognitive monitoring and hippocampal memory consolidation.
ProofPaper ↗
Key points
- Vision-Language Models (VLMs) have made substantial progress in spatial reasoning, and self-evolving agents further extend these capabilities to embodied environments by learning from interactions.
- However, current self-evolution remains particularly challenging in embodied spatial tasks, complex and multi-step decisions from partial visual evidence making it difficult to determine why an execution fails.
- Through Hippocampal-Metacognitive Reflection, HiMeco diagnoses failures in perception-exploration-planning (PEP), then compares execution attempts to derive reusable strategies and guidance for avoiding recurring errors.
- These strategies and skills guide subsequent interactions, with memory updated only between rounds and model parameters kept frozen.
Sources (1)
- [1]Reflect After Failure: Brain-Inspired Self-Evolving Agents for Embodied Spatial IntelligencearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 02:59 PM
To address this, we propose HiMeco, a brain-inspired self-evolving agent for embodied spatial intelligence, inspired by metacognitive monitoring and hippocampal memory consolidation.
Vision-Language Models (VLMs) have made substantial progress in spatial reasoning, and self-evolving agents further extend these capabilities to embodied environments by learning from interactions.
Extractive summary: sentences quoted from the sources.