Prism-1: Position- and Rival-Independent Scoring for System One Models

Key points

• Web systems increasingly build them by reading decisions from language models.1
• Two axioms of rational choice are natural requirements for such decisions, because in web systems the listing and the option set are determined by other components and can change or be manipulated: the order in which the options are listed should not matter, and adding an option should not change the relative odds of the others (Luce's choice axiom).1
• We test the standard reads, namely single-choice question (SCQ) prompting, with shuffled-order training, order averaging or sorted options, and invariant set architectures, and find that each violates at least one axiom: one shuffle changes 12-20% of SCQ answers, sorting only freezes an arbitrary convention, and a well-chosen added option steers 4-34% of decisions.1
• We implement this as Prism-1 (Position- and Rival-Independent Scoring for System One Models): every option is read in its own block at shared positions, with the model's usual left-to-right attention, in one forward pass.1

Sources (1)

• [1]Prism-1: Position- and Rival-Independent Scoring for System One Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 03:28 PM

  Given a state and a set of options, such as intents, tools or interface elements, System One decision models return a distribution over the options in one forward pass, without generated reasoning.

  Web systems increasingly build them by reading decisions from language models.

Extractive summary: sentences quoted from the sources.