Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling

Key points

• Recent reward-free power-sampling methods define the sequence-level power target $πα(y\mid x)\propto p0(y\mid x)^α$, where $α>1$, which increases the relative probability of completions that are already likely under the base model.1
• At the target level, full-sequence sharpening can concentrate probability on a few exact reasoning--answer trajectories, suppressing alternative paths that support the same answer.1
• To address these challenges, we propose Decoupled Marginal Sharpening (DMS), which separately controls how strongly the model favors particular reasoning paths and how strongly it favors final answers after aggregating support across those paths.1
• At the sampler level, we develop Fully Adapted Annealed SMC, which combines an annealed sequence of power targets with locally fully adapted proposals and predictive ancestor resampling.1

Sources (1)

• [1]Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 04:55 PM

  We study how to improve LLM reasoning at inference time without updating model weights or using external reward models.

  Recent reward-free power-sampling methods define the sequence-level power target $π_α(y\mid x)\propto p_0(y\mid x)^α$, where $α>1$, which increases the relative probability of completions that are already likely under the base model.

Extractive summary: sentences quoted from the sources.