Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches

Key points

• Under Rotary Position Embedding (RoPE), gaps left by evicted tokens continue to affect query-key matching, potentially limiting the use of retained evidence.1
• We study post-selection positional geometry on a fixed selector-produced cache, examining how positional organization affects the use of retained information and which coordinate and continuation rules improve answer quality.1
• To reorganize this geometry, we propose RoPERelay, which assigns compact, order-preserving RoPE coordinates, relocates cached key phases through inverse rotation and re-rotation, and synchronizes the current query and subsequent generation.1
• These findings establish positional organization as a distinct optimization dimension for sparse KV caches after content selection.1

Sources (1)

• [1]Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 02:30 AM

  Sparse KV compression reduces long-context inference costs by retaining selected historical states, yet these states typically keep their coordinates from the full history.

  Under Rotary Position Embedding (RoPE), gaps left by evicted tokens continue to affect query-key matching, potentially limiting the use of retained evidence.

Extractive summary: sentences quoted from the sources.