Don't Rewrite the SID Map: Topology Preservation for Generative Retrieval in Dynamic E-Commerce Search

Key points

• In dynamic environments such as e-commerce, new items arrive continuously, requiring periodic updates to the SID-to-Item mapping and model to maintain retrieval timeliness.1
• We study this problem with extensive controlled experiments over 67 days of large-scale e-commerce search logs and obtain three core findings.1
• (1) SID semantic drift causes severe retrieval degradation that may be masked by large beam sizes: mapping updates reduce HitRate by 41.78-62.05% at beam size 1 but only 8.33-10.85% at beam size 1000, as redundant candidate generation conceals the true severity of drift.1
• (3) The SID topology-preserving principle makes even the simplest warm-start strategy highly effective: fixing old-item SIDs, inserting new-items, and warm-starting from the previous model improves old-item HitRate by 10% and new-item HitRate by 109%, validated across two non-overlapping update cycles.1

Sources (1)

• [1]Don't Rewrite the SID Map: Topology Preservation for Generative Retrieval in Dynamic E-Commerce Search
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 10:01 AM

  Generative retrieval assigns each item a discrete semantic identifier (SID) and retrieves items by autoregressively generating SID sequences.

  In dynamic environments such as e-commerce, new items arrive continuously, requiring periodic updates to the SID-to-Item mapping and model to maintain retrieval timeliness.

Extractive summary: sentences quoted from the sources.