BCT: Bayes-Consistent Tempering for Flow Language Models
These mismatches are reflected empirically: lowering temperature improves GSM8K performance much less reliably in flow LMs than in discrete diffusion LMs. We introduce Bayes-Consistent Tempering (BCT), which makes temperature explicit in both the distributions and dynamics of the generative flow.
ProofPaper ↗
Key points
- Temperature scaling is typically applied as a post-hoc transformation of model probabilities.
- In posterior-induced flow language models, this creates two mismatches.
- Its learned realization trains the denoiser on tempered intermediate states and tempers the endpoint distribution using the corresponding tempered dynamics.
- Across hyperspherical flow LMs with two denoiser architectures and a von Mises--Fisher flow LM, BCT improves GSM8K generation across broad temperature ranges and achieves the strongest tempering performance.
Sources (1)
- [1]BCT: Bayes-Consistent Tempering for Flow Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:03 AM
These mismatches are reflected empirically: lowering temperature improves GSM8K performance much less reliably in flow LMs than in discrete diffusion LMs. We introduce Bayes-Consistent Tempering (BCT), which makes temperature explicit in both the distributions and dynamics of the generative flow.
Temperature scaling is typically applied as a post-hoc transformation of model probabilities.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 19, 2026Parallel Language Models via Variational Decoding
- Oct 19, 2026TorchNeuron: running HF models on AWS Trainium is easier than ever
- Oct 18, 2026Dual-Slot: From Serial Loops to Parallel Slots in Language Models
- Oct 18, 2026P2N: Reusing Deep Representations for Greater Effective Depth
- Oct 18, 2026VLA-Rolling: Fewer Denoising Steps, Better Control via Adaptive Action Reuse