Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books
We find that, similarly to games where sequence models build an internal representation of the board, an 8.2M-parameter Mamba-3 state-space model trained autoregressively on the tokenized message stream of eight large-cap US equities learns an internal representation of the market and its dynamics.
ProofPaper ↗
Key points
- Autoregressive foundation models offer state-of-the-art capabilities to simulate the market by generating the message flow of limit order books (LOBs).
- This study therefore asks what these models actually learn about market microstructure.
- In particular, order-flow imbalance, a central driver of short-horizon price moves, is decodable at $R^2 \approx 0.59$, far above controls and baselines.
- The model also generalizes across metrics beyond order-flow imbalance: results hold across traded volume and volatility.
Sources (1)
- [1]Understanding the Inner Workings of a State-Space Foundation Model for Limit Order BooksarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:01 PM
We find that, similarly to games where sequence models build an internal representation of the board, an 8.2M-parameter Mamba-3 state-space model trained autoregressively on the tokenized message stream of eight large-cap US equities learns an internal representation of the market and its dynamics.
Autoregressive foundation models offer state-of-the-art capabilities to simulate the market by generating the message flow of limit order books (LOBs).
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Learning the 4D World through Point Trajectory Forecasting
- Oct 19, 2026Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
- Oct 19, 2026Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
- Oct 19, 2026State World Models: Execute the Rules, Generate the Pixels
- Oct 19, 2026AnyGoal: On World Model Generalization Beyond Expert Rollouts
- Oct 19, 2026Frechet Forcing: Training Long-Horizon Streaming World Models from Short Clips