Evaluating Inquiry Learning in LLM-Based Agents: Hypothesizing, Reasoning, and Exploration

Key points

• Scientific inquiry involves iterative cycles of exploration, hypothesis generation, and reasoning over observations to uncover relationships in complex systems.1
• Large language models (LLMs) have demonstrated strong capabilities in several components of this process, including inductive reasoning, hypothesis generation, and data analysis.1
• We extend the Countdown task to a mathematical environment where an agent probes the system by manipulating variables, collecting observations, and inferring relationships between them.1
• Our findings highlight current limitations of LLMs in strategic exploration and iterative hypothesis refinement, and show how simple guidance strategies developed for human learners in inquiry learning can support more effective inquiry behavior.1

Sources (1)

• [1]Evaluating Inquiry Learning in LLM-Based Agents: Hypothesizing, Reasoning, and Exploration
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 09:52 AM

  In this work, we introduce a framework for evaluating LLMs' inquiry capabilities in a unified, interactive setting.

  Scientific inquiry involves iterative cycles of exploration, hypothesis generation, and reasoning over observations to uncover relationships in complex systems.

Extractive summary: sentences quoted from the sources.