Learning to Couple: Context-Adaptive Joint Denoising for Diffusion Language Models

Key points

• Decoding heuristics reduce the resulting errors by controlling which tokens are committed, but leave the underlying distribution factorized.1
• Joint modeling addresses the source of the problem, yet existing methods attach separately learned layers to frozen denoisers, leaving the backbone unadapted to the joint objective and the coupling parameters fixed across contexts.1
• We propose Codpula, a context-parameterized denoiser combining a Transformer with a tractable probabilistic inference layer.1
• We jointly optimize both components under a conditional denoising objective using a dual-flow multiplicative update that preserves valid probabilistic parameters.1

Sources (1)

• [1]Learning to Couple: Context-Adaptive Joint Denoising for Diffusion Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 03:22 AM

  Diffusion language models accelerate generation by predicting multiple tokens in parallel, but factorized denoisers treat simultaneous predictions as conditionally independent.

  Decoding heuristics reduce the resulting errors by controlling which tokens are committed, but leave the underlying distribution factorized.

Extractive summary: sentences quoted from the sources.