Retrieval Disagreement as a Signal: Diagnosing and Steering Multilingual Dense Embedders

Key points

• Multilingual dense retrievers place all languages in one shared space so that a question finds relevant documents, regardless of which language they are written in.1
• We show that the same question, asked in different languages, retrieves substantially different documents, and this disagreement is not because of weak language understanding.1
• We trace the mechanism: each language keeps its own ranking, but in the shared index, its correct answers are pushed far down, buried under non-relevant documents that are either in the query's own language or in English, depending on the model.1
• Against baselines of language-mean removal, subspace removal, and per-language rotation, it most strongly reduces same-language crowding while remaining competitive on recall, trading off against mean removal, which better preserves ranking quality.1

Sources (1)

• [1]Retrieval Disagreement as a Signal: Diagnosing and Steering Multilingual Dense Embedders
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 03:13 PM

  We introduce disagreement-guided steering: for each language, we contrast the documents that other languages agree on for a question against those that language retrieves idiosyncratically, and take the difference of their representations as a language-specific direction, added to that language's queries at inference.

  Multilingual dense retrievers place all languages in one shared space so that a question finds relevant documents, regardless of which language they are written in.

Extractive summary: sentences quoted from the sources.