Anchors Enhance Controllability and Reasoning of Diffusion Language Models

Key points

• DLMs' bidirectional attention allows control tokens to guide generation from arbitrary positions in the output sequence, but effective guidance depends on the compatibility between their content and placement.1
• To this end, we propose AnchorDLM, a framework in which an anchor generator produces an anchor consisting of a short output fragment and its position for each query.1
• We train the anchor generator with reinforcement learning (RL) to jointly optimize anchor content and placement, using rewards computed from the DLM's generated outputs based on answer correctness and constraint satisfaction.1
• Experiments on code generation and mathematical reasoning across five DLM backbones show that AnchorDLM improves controllability and further yields gains in reasoning performance across diverse benchmarks.1

Sources (1)

• [1]Anchors Enhance Controllability and Reasoning of Diffusion Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 06:14 PM

  Diffusion language models (DLMs) offer new opportunities for controllable text generation.

  DLMs' bidirectional attention allows control tokens to guide generation from arbitrary positions in the output sequence, but effective guidance depends on the compatibility between their content and placement.

Extractive summary: sentences quoted from the sources.