Amortizing Sequential Experimental Design in Reasoning LLMs

Key points

• Despite their extensive pretraining knowledge and strong reasoning capabilities, large language models (LLMs) struggle to adaptively gather information in experimental design tasks.1
• Our approach specifies the experimental generative model independently of the LLM's own beliefs, allowing us to optimize expected information gain under a model of our choosing.1
• We investigate the design choices for amortization, including alternative generative-model constructions, reward estimation, hypothesis-space specification, multi-step objectives, and computational costs relative to training- and inference-time BED baselines.1
• We further demonstrate positive transfer from Twenty Questions training to Medi-Q, a real-world clinical dialogue benchmark.1

Sources (1)

• [1]Amortizing Sequential Experimental Design in Reasoning LLMs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 07:15 PM

  We study how to amortize sequential Bayesian experimental design (BED) into an LLM by explicitly training the reasoning that guides action selection using information-gain rewards.

  Despite their extensive pretraining knowledge and strong reasoning capabilities, large language models (LLMs) struggle to adaptively gather information in experimental design tasks.

Extractive summary: sentences quoted from the sources.