Two Is Enough: Binary Classification Priors Suffice for Multi-Class In-Context Tabular Prediction

Key points

• TabICLv2 samples the class count of its classification prior during pretraining, and the prevailing configuration caps it at ten.1
• We report a counter-intuitive result: setting this cap directly to two---making every pretrained classification task binary---still yields weights that perform zero-shot K-way classification 2.1
• On the 38 TabArena classification datasets (200 bootstrap rounds, anchored at default RF = 1000), the binary two-stage recipe reaches 1590.1 Elo, above the official default at 1568.0 (+22.1), above TabPFN-v2.6 (1577.1) and RealTabPFN-v2.5 (1521.6).1
• Mechanistically, classification in TabICLv2 is carried not by a fixed-cardinality output head but by an in-context comparison operator matching query-row embeddings against class-conditional embeddings, so the cardinality of the output space is set by the labels present in the context, not by the label space seen during pretraining.1

Sources (1)

• [1]Two Is Enough: Binary Classification Priors Suffice for Multi-Class In-Context Tabular Prediction
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:34 AM

  Tabular foundation models (TFMs) pretrain a transformer on large-scale synthetic priors so that a single forward pass yields zero-shot tabular prediction.

  TabICLv2 samples the class count of its classification prior during pretraining, and the prevailing configuration caps it at ten.

Extractive summary: sentences quoted from the sources.