Logit Leakage through Floating-Point Bias Addition
Language model APIs can expose token-specific biases while withholding logits and token probabilities.
ProofPaper ↗
Key points
- Existing bias-based extraction recovers relative logits, leaving a common additive component undetermined in ideal arithmetic.
- We identify a numerical side channel in floating-point bias addition that exposes information about this component through token identities alone.
- We evaluate recovery across 11 public checkpoints from seven model families and extend the local method with adaptive interval search and anchor selection.
- A separate token-only experiment extracts a pretrained small model's complete output-head representation up to an invertible hidden-basis transformation.
Sources (1)
- [1]Logit Leakage through Floating-Point Bias AdditionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 01:44 PM
Language model APIs can expose token-specific biases while withholding logits and token probabilities.
Existing bias-based extraction recovers relative logits, leaving a common additive component undetermined in ideal arithmetic.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026A "free" multi-modal "embedding" model (already) included with your LLM: vLLM plugin + upstream PR
- Oct 12, 2026Qwen3.8-Flash-Next on six 3090s with vLLM: 7 users at 262k, 6.5k t/s prefill, 57 t/s decode solo / 206 t/s aggregate with all 7 generating. Fixes for the upstream included.
- Oct 12, 2026Hashrope: A Persistent Data Structure for Mechanisms of LLM Context Management
- Oct 12, 2026StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions
- Oct 12, 2026DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck
- Oct 11, 2026Taming the AI Hardware Cambrian Explosion — Abdul Dakkak, Modular