ResearchResearch paperLarge Language Models · Efficiency & Inference1 source · Oct 6, 2026

Breaking the Space Barrier and its Application to Language Model Inference

Language models are more and more often asked for structured output: JSON that follows a schema, or a tool call with typed arguments.

A small machine, an automaton, enforces the format by forbidding the tokens that would break it.
Graphs in which only a few paths join any two points are a classical object of complexity theory, and our theoretical result settles an open question about them: one can decide whether such a graph connects two points while verifying that it really has few paths, with very little memory.
The constructions behind the proofs become an inference engine: text the format forces is written without running the model, the mask is recomputed on the GPU without any table, recursive formats use a small stack, every output stays valid under a token limit, and independent fields are decoded in parallel and verified.
On one 16 GB Apple M2 Pro with Qwen3.5-2B and 4B, against MLX with llguidance, the standard setup for this hardware, schema-constrained extraction finishes 1.2- 1.3x sooner with the same answers, a grammar costs 3 MB instead of up to 1.5 GB, one server holds sixteen grammars where tables run out of memory, and sixteen tool-calling agents finish 2.5x sooner.

[1]Breaking the Space Barrier and its Application to Language Model Inference
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 09:29 PM
Language models are more and more often asked for structured output: JSON that follows a schema, or a tool call with typed arguments.
A small machine, an automaton, enforces the format by forbidding the tokens that would break it.

Extractive summary: sentences quoted from the sources.

Before this