Read the Room Before Your Model Generates

Key points

• Large language models (LLMs) generally rely on sampling temperature to regulate the amount of randomness in their outputs.1
• Mainstream methods for selecting temperature typically involve fixed, heuristic global settings and context or token-level calibration, rather than a geometrically interpretable representation of the communicative scenario.1
• Scenario-Actor extracts scenario-relevant structures from Transformer hidden states and determines the temperature from scenario embedding, a structured geometric representation of the communicative setting.1
• Empirical results show that on Llama-3.1-8B-Instruct, Scenario-Actor achieves strong goal-completion performance in interactive social scenarios, significant gains on open-ended instruction following, and competitive performance on multi-turn conversation compared to multiple baselines.1

Sources (1)

• [1]Read the Room Before Your Model Generates
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 05:51 PM

  In this work, we introduce a simple and interpretable temperature selection framework, the Scenario-Actor, which adapts decoding to the inferred situational characteristics.

  Large language models (LLMs) generally rely on sampling temperature to regulate the amount of randomness in their outputs.

Extractive summary: sentences quoted from the sources.