InteractAgent: Agentic Human Motion Interaction with Memory-Augmented Reflection
We present InteractAgent, an agentic multimodal LLM system for long-horizon human-scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor.
ProofPaper ↗
Key points
- Long-horizon human-scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors.
- Text-driven motion models are largely scene-agnostic and lack closed-loop correction, while geometry-conditioned approaches depend on paired scene-motion training or explicit 3D preprocessing that limits generalization to novel layouts and open-vocabulary goals.
- InteractAgent integrates Online Observation for step-wise recovery decisions (continue/redo/replan) and Memory-Augmented Reflection for multi-round plan refinement with adaptive stopping.
- On UniHSI, InteractAgent improves task success and reduces penetration compared with training-free baselines.
Sources (1)
- [1]InteractAgent: Agentic Human Motion Interaction with Memory-Augmented ReflectionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 01:51 AM
We present InteractAgent, an agentic multimodal LLM system for long-horizon human-scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor.
Long-horizon human-scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026HarnessPro: Recursive Harness Self-Improvement with Pareto Optimization for Performance-Cost Trade-offs
- Oct 18, 2026Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark: DeepSeek-V4.1-Flash-1017 EXL3 2.9bpw on 2x NVIDIA DGX Spark (GB10) with TensorFold: 2.1-2.3x the vLLM kit's decode speed.
- Oct 17, 2026Blown away by Minimax H3 sometimes. It nailed this one!!!!
- Oct 17, 2026Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026MS-Eval: Benchmarking Sub-Agent Delegation and Orchestration in MCP-Intensive Working Scenarios