Choosing LLMs for Strategic Agent Interactions
Game-theoretic benchmarks claim that large language model (LLM) strategic decision-making varies across game types, interaction structures, and capability dimensions.
ProofPaper ↗
Key points
- We find that the evaluation signal is game-specific, and that extracting a reliable signal requires a much higher number of seed runs $N$ compared to traditional benchmarking setups.
- Evaluating across 5 games and 10 LLMs, we show a model's strategic competence varies significantly depending on the game setup and the opponent faced.
- Building on this, we implement a suite of selector methods and find that approaches that rely on query embeddings or LLM self-reflection do not consistently capture the underlying strategic signal.
- Finally, we show that simple non-stationary and adversarial bandit variants fail to outperform a stationary bandit baseline, pointing to a gap in how current methods handle a shifting opponent in strategic settings.
Sources (2)
- [1]Choosing LLMs for Strategic Agent InteractionsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 01:21 PM
Game-theoretic benchmarks claim that large language model (LLM) strategic decision-making varies across game types, interaction structures, and capability dimensions.
We find that the evaluation signal is game-specific, and that extracting a reliable signal requires a much higher number of seed runs $N$ compared to traditional benchmarking setups.
- [2]Choosing LLMs for Strategic Agent InteractionsHugging Face Daily Papers · Oct 13, 12:00 AM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026LATCH: LLM-Assisted Timing Closure Harness for Post-Route FPGA Optimisation
- Oct 12, 2026AuthGateBench: Benchmarking LLMs as Authorization Gate
- Oct 11, 2026Large language model-generated patterns for epidemiological modeling: a multi-benchmark evaluation framework
- Oct 11, 2026Make Every Expansion Count: Learning to Search Efficiently in LLM Code Repair
- Oct 10, 2026Conflict-Aware Autonomous Resource Orchestration for 5G Edge Slicing Using Multi-Agent Systems and LLMs
- Oct 10, 2026Beyond Outcome Feedback: Trajectory-Quality-Guided Experience-based Learning