CrashGym: Measuring the Decision Value of Experimentation for LLM Agents in Counterfactual Market Crashes

Key points

• Language-model agents are increasingly asked to run experiments, yet benchmarks rarely measure whether those experiments improve the decisions they are meant to inform.1
• Before committing, the agent may query intervention experiments in a laboratory realization of the market.1
• The simulator implements each amplification channel---persistent price impact, liquidity withdrawal, margin-driven liquidation, and an evolving strategy mix---as an explicit mechanism that can be switched on in stages.1
• CrashGym thus measures the value of experimentation as avoided decision loss and lets simulator detail be judged by the decisions it supports.1

Sources (1)

• [1]CrashGym: Measuring the Decision Value of Experimentation for LLM Agents in Counterfactual Market Crashes
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 05:51 AM

  We introduce CrashGym, a deterministic, replayable limit-order-book simulator in which an agent must choose between a deleveraging intervention and a liquidity backstop for a market about to suffer a shock-triggered crash.

  Language-model agents are increasingly asked to run experiments, yet benchmarks rarely measure whether those experiments improve the decisions they are meant to inform.

Extractive summary: sentences quoted from the sources.