MaskSense: Confronting the Visual Exploration Trap in Masked Image Generation
Reinforcement learning (RL) has shown strong potential for aligning generative models with human intent, but adapting RL to masked generative models (MGMs) remains largely underexplored.
ProofPaper ↗
Key points
- Existing approaches that directly adapt Group Relative Policy Optimization (GRPO) to MGMs leave a foundational conflict unaddressed: MGMs' exploitative decoding nature is inherently at odds with the exploratory diversity that GRPO presupposes.
- In this work, we reveal a structural decoding bias termed the visual exploration trap: confidence-based sampling trades the exploration of diverse subject realizations for the greedy resolution of low-uncertainty background regions, causing premature collapse of the generative exploration space and starving GRPO of the rollout diversity it depends on.
- To this end, we propose MaskSense, a novel RL framework for MGMs that confronts the visual exploration trap at both the sampling and optimization stages.
- Specifically, we introduce a semantic anchored routing sampling mechanism that leverages semantic priors to preserve high-entropy subject token exploration while amplifying intra-group reward variance through exploratory-exploitative routing to yield more discriminative advantage estimates.
Sources (1)
- [1]MaskSense: Confronting the Visual Exploration Trap in Masked Image GenerationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 03:52 PM
Reinforcement learning (RL) has shown strong potential for aligning generative models with human intent, but adapting RL to masked generative models (MGMs) remains largely underexplored.
Existing approaches that directly adapt Group Relative Policy Optimization (GRPO) to MGMs leave a foundational conflict unaddressed: MGMs' exploitative decoding nature is inherently at odds with the exploratory diversity that GRPO presupposes.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026A Survey of Flow Matching in Reinforcement Learning
- Oct 12, 2026Look Back to Move Forward: Learning to Revisit for Agentic Compositional Image Generation
- Oct 12, 2026AUGUstus: Autonomous Driving VLA Model with Unified Generation and Understanding
- Oct 11, 2026AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
- Oct 11, 2026RSIPO: Recursive Self-Improving Policy Optimization for Agentic Reinforcement Learning via Skill-Policy Co-Evolution
- Oct 10, 2026Structural Chain-of-Thought: Reasoning with Executable Computational Graphs