Bayesian Model Discovery with Language Models: Closing the Loop between Hypotheses and Experiments

Key points

• Scientific model discovery requires proposing candidate models of an unknown system and designing experiments that discriminate among them.1
• However, LLM-based approaches typically rely on in-context learning to predict outcomes and select experiments, lacking explicit model evaluation to systematically guide hypothesis search, while Bayesian approaches enable principled model comparison but are confined to a predefined candidate set which results in limited search spaces.1
• Specifically, an LLM proposes model families, while a numerical engine fits their parameters, weighs them by model evidence, and averages their predictions.1
• Across four scientific discovery benchmarks, BMD achieves lowest final prediction error than in-context learning and LLM-based scientific agents while recovering exact functional forms more often, with ablations supporting numerical feedback and the joint objective.1

Sources (1)

• [1]Bayesian Model Discovery with Language Models: Closing the Loop between Hypotheses and Experiments
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 03:58 AM

  To this end, we introduce BMD, which closes the loop between LLM-driven hypothesis generation and Bayesian experimental design.

  Scientific model discovery requires proposing candidate models of an unknown system and designing experiments that discriminate among them.

Extractive summary: sentences quoted from the sources.