Evolving Semantic Indexes for Generative Retrieval
Generative retrieval (GR) casts search and recommendation as autoregressive generation of discrete semantic identifiers (SIDs).
ProofPaper ↗
Key points
- We introduce a framework that improves these components through LLM-driven evolutionary program search, to our knowledge the first application of this approach to GR.
- Across 21 SID programs on NQ320K, proxy and target Recall@100 show strong rank correlation (Spearman $ρ= 0.90$).
- Guided by this proxy, evolution improves the seed GR on NQ320K, MSMARCO300K, and a multilingual e-commerce dataset under a fixed training recipe.
- A component study shows that proxy reliability depends on what is evolved: improvements transfer reliably for the semantic index but not for the generator's training objective or the full pipeline.
Sources (1)
- [1]Evolving Semantic Indexes for Generative RetrievalarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:27 PM
Generative retrieval (GR) casts search and recommendation as autoregressive generation of discrete semantic identifiers (SIDs).
We introduce a framework that improves these components through LLM-driven evolutionary program search, to our knowledge the first application of this approach to GR.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
- Oct 20, 2026Exa 3.0 Search API Tops Perplexity, Brave and Parallel on 8 Benchmarks
- Oct 20, 2026RAG-NIMBF: A Non-Invasive, Modular Benchmarking Framework for Automated RAG Configuration Analysis
- Oct 20, 2026Toward Agentic AI for the 6G Physical Layer: From Algorithm Discovery to Verified Baseband Silicon
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM