Enhanced Masked Diffusion Unlearning
Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens with bidirectional context.
ProofPaper ↗
Key points
- Masked Diffusion Unlearning (MDU) is the first publicly available unlearning method customized for these models: it tunes the model $pθ$ to match a frozen prompt-masked anchor $p{anchor}$ on the forget set by minimizing the anchor-last divergence $KL(pθ\Vert p{anchor})$.
- First, random answer masking can leave key forget tokens visible to the anchor even when the question is fully masked.
- Second, at a memorized (near-one-hot) prediction, the anchor-last logit gradient vanishes while the anchor-first divergence $KL(p{anchor} \Vert pθ)$ logit gradient does not.
- Backed by theoretical and empirical analyses, we propose Enhanced Masked Diffusion Unlearning (EMDU): anchor-first KL divergence toward a frozen anchor whose entire answer span is masked.
Sources (1)
- [1]Enhanced Masked Diffusion UnlearningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 06:09 PM
Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens with bidirectional context.
Masked Diffusion Unlearning (MDU) is the first publicly available unlearning method customized for these models: it tunes the model $p_θ$ to match a frozen prompt-masked anchor $p_{anchor}$ on the forget set by minimizing the anchor-last divergence $KL(p_θ\Vert p_{anchor})$.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 17, 2026Who Gets Unmasked? Sampler-Induced Bias in Diffusion Language Models
- Oct 17, 2026llama.cpp has come a long way
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026axolotl-ai-cloud/axolotl v0.21.0
- Oct 16, 2026Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design
- Oct 15, 2026EgoP$^{2}$C: Internalizing Plan-Perceive-Compose for Spatial Reasoning in Egocentric Video
