Bayesian Test-Time Inference of Task-Aligned Similarity from Weak Interactive Feedback

Key points

• Yet similarity is not unique: lexical, semantic, structural, and stylistic resemblance can point to different candidates, and the task-aligned notion can change across instances.1
• We introduce BASIS (Bayesian Axis Selection via Interactive Signals), a training-free Bayesian method that uses weak test-time feedback to infer which criterion in a supplied, task-specific library governs the current decision.1
• In a finite noisy model, we characterize when the target-criterion pair is recoverable and show why fixed-criterion and fixed-mixture rules can fail under criterion shift.1
• Across concept recovery, reasoning exemplar selection, and response refinement, BASIS improves six-probe concept-recovery Success@1 from 56.0% to 68.1%, reasoning accuracy from 72.3% to 73.1%, and refinement win rate from 61.4% to 63.8% over tuned static mixtures.1

Sources (1)

• [1]Bayesian Test-Time Inference of Task-Aligned Similarity from Weak Interactive Feedback
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 04:19 PM

  Language model systems repeatedly make test-time decisions by comparing similarity: they retrieve exemplars, rank candidates, and revise responses.

  Yet similarity is not unique: lexical, semantic, structural, and stylistic resemblance can point to different candidates, and the task-aligned notion can change across instances.

Extractive summary: sentences quoted from the sources.