CFG-Policy: Unified Diffusion Policy Learning Framework via Classifier-free Guidance

Key points

• Diffusion policies are increasingly used in robotic control and manipulation, vision-language-action models, and world-model-based agents.1
• Their ability to represent multimodal action distributions offers rich opportunities for exploration in reinforcement learning (RL).1
• For online RL, a frozen exploration branch sustains behavioral diversity while an exploitation branch learns from task rewards; for offline-to-online RL, a pretrained exploitation branch remains frozen while a trainable branch steers the composed behavior.1
• CFG-Policy preserves the underlying RL objectives and improves performance with QSM and QVPO on 5 MuJoCo tasks and with Flow-SDE+PPO on 4 RoboTwin manipulation tasks.1

Sources (1)

• [1]CFG-Policy: Unified Diffusion Policy Learning Framework via Classifier-free Guidance
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 02:09 PM

  We introduce CFG-Policy, a unified, plug-and-play framework that separates exploration and exploitation into two generative policies and combines their score or velocity fields during sampling through classifier-free guidance.

  Diffusion policies are increasingly used in robotic control and manipulation, vision-language-action models, and world-model-based agents.

Extractive summary: sentences quoted from the sources.