AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation

Key points

• Vision-language-action (VLA) models have shown promise in robotic manipulation, but imitation learning (IL) alone often struggles to master complex long-horizon skills.1
• AtomicRL introduces Semantic-Aware Policy Optimization (SAPO), which uses semantic-aware gating to constrain TD residuals from failed atomic subtasks from propagating across task boundaries, mitigating credit contamination while preserving intra-subtask credit assignment and value bootstrapping.1
• Finally, we develop a Real2Sim2Real pipeline that bridges online RL in simulation with real-world deployment, facilitating practical transfer to physical robots.1
• These results demonstrate that AtomicRL effectively improves the learning efficiency and manipulation performance of VLA policies on long-horizon tasks.1

Sources (1)

• [1]AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 05:45 PM

  To address these challenges, we propose AtomicRL, an RL post-training framework that leverages atomic task structures for reliable credit assignment and adaptive exploration.

  Vision-language-action (VLA) models have shown promise in robotic manipulation, but imitation learning (IL) alone often struggles to master complex long-horizon skills.

Extractive summary: sentences quoted from the sources.