What Do Foundation Models for Market Microstructure Learn? A Controlled Study with Synthetic Limit Order Books

Key points

• Because the generative process of real markets is unknown, these models can be judged only by the realism of their output, leaving open what they actually learn.1
• We make the question answerable by inverting the setup: we train GPT-style models on order flow from simulators whose latent parameters are known, so that the Bayes-optimal predictor, the generating state, and the exact data entropy serve as references.1
• Measured against them, next-token pretraining on order flow implements amortized Bayesian inference over market regimes.1
• On real NASDAQ data, zero-shot transfer is set by how well the simulator prior covers real flow: an enriched mixture nearly closes the gap to an in-domain baseline, and pretraining beats from-scratch training at every budget across held-out tickers.1

Sources (1)

• [1]What Do Foundation Models for Market Microstructure Learn? A Controlled Study with Synthetic Limit Order Books
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 12:38 PM

  Autoregressive transformers trained on limit order book (LOB) message streams are increasingly proposed as foundation models for market microstructure.

  Because the generative process of real markets is unknown, these models can be judged only by the realism of their output, leaving open what they actually learn.

Extractive summary: sentences quoted from the sources.