ReLaDrift: Recurrent Language Generation via Drifting for Test-Time Scaling

Key points

• Drifting Models generate a sample with a single function evaluation, making them attractive for fast parallel language generation, but their one-step formulation does not support iterative refinement.1
• We train this recurrence by sampling the number of rounds and applying the drifting loss only at the final round, stopping gradients through preceding rounds so that only one round requires backpropagation.1
• Unlike the one-step Drifting Model, whose accuracy does not improve when iterated, ReLaDrift enables a trade-off between test-time computation and generation quality, as diffusion and flow language models do.1
• On Sudoku Hard and GSM8K, it achieves competitive or superior performance compared with these iterative-generation approaches across a range of inference budgets.1

Sources (1)

• [1]ReLaDrift: Recurrent Language Generation via Drifting for Test-Time Scaling
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 09:55 AM

  We propose Recurrent Language Generation via Drifting (ReLaDrift), which uses a shared Drifting Model to generate an initial draft from noise and iteratively refine it in a continuous latent space.

  Drifting Models generate a sample with a single function evaluation, making them attractive for fast parallel language generation, but their one-step formulation does not support iterative refinement.

Extractive summary: sentences quoted from the sources.