Literal Lock-In: How Test Language Shifts Literal Attraction in Typed Decision Models on Bengali Idioms

Key points

• Decision models, non-generative scorers that report a probability over caller-supplied options and emit no text, are increasingly deployed as routers, classifiers and guardrails.1
• Dropping an idiom's own literal gloss into the option list draws every arm toward it, taking 65% to 96% of each arm's remaining Bengali errors.1
• A model-by-language matrix over one shared item set shows how far this depends on the test language: every arm is 1.9 to 6.4 times more captured in Bengali than in English on the same paired entries, the commercial model's apparent resistance is a property of its test language, and capability does not order the failure.1
• On a semantics-free option list, where accuracy is at chance, the best-calibrated arm still reports mean confidence 0.82.1

Sources (1)

• [1]Literal Lock-In: How Test Language Shifts Literal Attraction in Typed Decision Models on Bengali Idioms
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 06:19 PM

  We audit three: an encoder scorer, a decoder scorer and a closed API, entirely through interventions on their input, using expert-annotated Bengali idioms carrying a gold figurative and a gold literal gloss.

  Decision models, non-generative scorers that report a probability over caller-supplied options and emit no text, are increasingly deployed as routers, classifiers and guardrails.

Extractive summary: sentences quoted from the sources.