PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking
We introduce Progress-Aligned Recurrence (PAR), which trains the prediction at iteration k to match the state after the first k operations and rescales recurrent states to their first-iteration norms.
ProofPaper ↗
Key points
- Looped language models can run more iterations at test time, but additional iterations need not help them solve longer problems.
- For sequential state tracking, final-answer supervision does not specify how far each iteration should advance, and repeated updates can change the scale of the recurrent state.
- On swap-S5, a Qwen2.5-0.5B model trained on at most 32 moves achieves 95.5\% mean full-state accuracy at 512 moves, or 16times the training length, compared with 55.7\% reported for RLT and 0.8\% for explicit chain of thought on the same backbone.
- A learned completion readout selects when to answer without a supplied stopping depth.
Sources (2)
- [1]PAR: Progress-Aligned Recurrence for Length Generalization in State TrackingHugging Face Daily Papers · Oct 20, 12:00 AM
We introduce Progress-Aligned Recurrence (PAR), which trains the prediction at iteration k to match the state after the first k operations and rescales recurrent states to their first-iteration norms.
Looped language models can run more iterations at test time, but additional iterations need not help them solve longer problems.
- [2]PAR: Progress-Aligned Recurrence for Length Generalization in State TrackingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 04:54 AM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Sparse RL Relaxation for LLM Reasoning
- Oct 19, 2026Internalizing Chain-of-thought into Looped Transformers
- Oct 19, 2026Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode
- Oct 19, 2026TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
- Oct 19, 2026Competing Learning in Hybrid Architectures
- Oct 18, 2026Think Visually Across Space and Time: Grounded Reasoning for Video Segmentation