Unrolled Flow Models for Reasoning
Flow matching enables language generation in few steps, but whether additional integration steps improve reasoning remains unclear.
ProofPaper ↗
Key points
- We prove that a flow parameterized by a two-layer Transformer can solve graph reachability, with the required number of integration steps increasing with the target's distance from the root.
- Yet, standard flow language models can fail to benefit from additional steps on reasoning tasks.
- To address this, we instead train through the model's own latent rollout over a randomly sampled subinterval of [0, 1], decoding only at the endpoint.
- Together, these results establish a theoretical basis for reasoning with flows and show how rollout training, stable latent dynamics, and rollout selection help realize this capacity in practice.
Sources (1)
- [1]Unrolled Flow Models for ReasoningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 09:45 AM
Flow matching enables language generation in few steps, but whether additional integration steps improve reasoning remains unclear.
We prove that a flow parameterized by a two-layer Transformer can solve graph reachability, with the required number of integration steps increasing with the target's distance from the root.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 7, 2026MeshCarve: Artisan Mesh Generation with Flow Matching in Compact Latent Spaces
- Oct 7, 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning
- Oct 6, 2026openbmb/MiniCPM-V-4.7-35B-A3B-gguf
- Oct 6, 2026EmbeddingGemma 2: an open, lightweight multimodal embedding model
- Oct 5, 2026LiquidAI/d1-omni-600M
- Oct 5, 2026MC-Sparse: Deconstructing and Closing the Dense-Sparse Attention Gap in Diffusion Transformers
