Occupancy Modeling via Drifting: One-Step Generation Enables Direct Policy Extraction
The growing interest in behavior foundation models, combined with recent advancements in generative modeling, has led to an appealing recipe for reinforcement learning from unlabeled data: pre-train a generative model to model occupancies, i.e., discounted state visitations, on reward-free trajectories, such that a policy can be easily extracted once a small reward-labeled dataset becomes available.
ProofPaper ↗
Key points
- We decompose this recipe along three axes: (i) the degree to which the multi-modal structure of occupancies must be captured, (ii) the class of generative model, and (iii) the method used for policy extraction.
- Along each axis, we consider existing choices, propose simpler alternatives, and validate each decision experimentally.
- This analysis culminates in a novel, streamlined method that leverages drifting, a modern one-step generative sampler, to model the occupancy.
- We show that this enables policy extraction via direct differentiation through the occupancy model, and bypasses the need for critic distillation.
Sources (1)
- [1]Occupancy Modeling via Drifting: One-Step Generation Enables Direct Policy ExtractionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 09:37 AM
The growing interest in behavior foundation models, combined with recent advancements in generative modeling, has led to an appealing recipe for reinforcement learning from unlabeled data: pre-train a generative model to model occupancies, i.e., discounted state visitations, on reward-free trajectories, such that a policy can be easily extracted once a small reward-labeled dataset becomes availabl
We decompose this recipe along three axes: (i) the degree to which the multi-modal structure of occupancies must be captured, (ii) the class of generative model, and (iii) the method used for policy extraction.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026The Design Space of Tri-Modal Masked Diffusion Models
- Oct 14, 2026RepForcing: Representation-Space Initialization for Autoregressive Video Distillation
- Oct 14, 2026Glance Forcing: Making Bidirectional Diffusion Autoregressive at a Glance
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026Fish Audio Drama 3 Technical Report
- Oct 13, 2026GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning
