Sparse RL Relaxation for LLM Reasoning
Recent work has explored continuous representations in place of discrete tokens during Chain-of-Thought (CoT) reasoning, often referred to as soft CoT.
ProofPaper ↗
Key points
- This is motivated by the idea that a mixture of token embeddings can represent a superposition of multiple reasoning possibilities within a single step.
- Existing work has explored reinforcement learning (RL)-style training for soft CoT.
- In this paper, we propose a sparse relaxation method for RL (called SpaRLax), which leverages the differentiability of soft-CoT reasoning over time steps and makes training more stable and effective.
- Further analyses validate the effective use of soft-token superposition in our approach, and also demonstrate the contribution of adaptive sparsity in our approach.
Sources (1)
- [1]Sparse RL Relaxation for LLM ReasoningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 07:50 PM
Recent work has explored continuous representations in place of discrete tokens during Chain-of-Thought (CoT) reasoning, often referred to as soft CoT.
This is motivated by the idea that a mixture of token embeddings can represent a superposition of multiple reasoning possibilities within a single step.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 18, 2026Outcome-Anchored Credit Reallocation for Agentic Knowledge Graph Question Answering
- Oct 18, 2026Calibration-Free Early Stopping for Chain-of-Thought Reasoning via E-processes
- Oct 18, 2026Think Visually Across Space and Time: Grounded Reasoning for Video Segmentation
- Oct 17, 2026ProbePO: Sparse Probes, Redistributed Credit for Critic-Free LLM Reinforcement Learning