Beyond the Vote: When Do LLMs Add Value to Retrieval-Based Tabular Classification?
Retrieval-based in-context learning (ICL) enables large language models (LLMs) to predict tabular labels from similar examples, but it remains unclear whether LLM inference improves on direct voting over those same examples.
ProofPaper ↗
Key points
- We address this question by comparing four general-purpose LLMs with distance-weighted nearest-neighbor voting (Distance-KNN) using identical retrieved neighbors across 19 tabular classification datasets.
- More detailed analyses reveal that LLM gains are concentrated where retrieved neighbors exhibit low label consensus and vary across models.
- These gains further depend on the average feature-space distance to the test query: under low label consensus, some LLMs outperform Distance-KNN in close and intermediate neighborhoods, but all fall behind on average in the most distant group.
- These results characterize when LLM inference adds predictive value beyond distance-weighted voting in retrieval-based tabular ICL.
Sources (1)
- [1]Beyond the Vote: When Do LLMs Add Value to Retrieval-Based Tabular Classification?arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 10:06 AM
Retrieval-based in-context learning (ICL) enables large language models (LLMs) to predict tabular labels from similar examples, but it remains unclear whether LLM inference improves on direct voting over those same examples.
We address this question by comparing four general-purpose LLMs with distance-weighted nearest-neighbor voting (Distance-KNN) using identical retrieved neighbors across 19 tabular classification datasets.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 21, 2026Flat-RLVR: Sharpness-Aware Reinforcement Learning with Verifiable Rewards for Reasoning Capacity in LLMs
- Oct 21, 2026A Large-Scale Evaluation of Role-Enhanced Zero-Shot Chain-of-Thought Prompting
- Oct 21, 2026Quant-Arena: Benchmarking LLM Agents for Quantitative Trading Research
- Oct 19, 2026Evaluating Agents on Highly Subjective Tasks
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM