CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds
We introduce CivWorldBench, a benchmark built on a multi-agent strategy game engine with recorded internal states, controlled observations, and paired intervention rollouts.
ProofPaper ↗
Key points
- The growing use of language models for forecasting motivates a broader evaluation of their world-model capabilities.
- Stored trajectories make intermediate predictions verifiable, while agent-play experiments test whether world-model capabilities support decision-making.
- Agent-play experiments identify useful assisted plans and motivate evaluating decision-relevant capabilities beyond association forecasting.
- These findings identify hidden-state inference, intermediate-state prediction, and effect calibration as priorities for world-model training.
Sources (1)
- [1]CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox WorldsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 09:16 AM
We introduce CivWorldBench, a benchmark built on a multi-agent strategy game engine with recorded internal states, controlled observations, and paired intervention rollouts.
The growing use of language models for forecasting motivates a broader evaluation of their world-model capabilities.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models
- Oct 12, 2026RAR: Rate-Adaptive Rollout for Time-Consistent Interactive World Models
- Oct 12, 2026Esports World Model Dataset
- Oct 12, 2026Multics: Generative Multi-Player World Model with Hierarchical Communication
- Oct 12, 2026Relational Learning as World Modeling
- Oct 11, 2026ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs