Simple Linear Corrections for Compression in Late Interaction
Late-interaction (multi-vector) models represent documents using a set of token-level vectors, providing a middle ground between cross-encoders and dense encoders in terms of quality and cost of embedding.
ProofPaper ↗
Key points
- We investigate whether a lightweight change to the scoring function can recover retrieval quality after document-vector compression.
- For each combination of encoder, compressor, and vector budget, we learn a global linear correction by distilling the uncompressed model's score distributions over candidate documents.
- We evaluate the method with ColBERTv2 and Jina v4 on MS MARCO and BEIR, and with ColPali2 on the ViDoRe v3 visual document retrieval benchmark.
- We also find that matrices trained with synthetic queries can improve over baseline compression on ViDoRe v3.
Sources (1)
- [1]Simple Linear Corrections for Compression in Late InteractionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 06:08 PM
Late-interaction (multi-vector) models represent documents using a set of token-level vectors, providing a middle ground between cross-encoders and dense encoders in terms of quality and cost of embedding.
We investigate whether a lightweight change to the scoring function can recover retrieval quality after document-vector compression.
Extractive summary: sentences quoted from the sources.