ResearchResearch paperLarge Language Models · Interpretability1 source · Oct 20, 2026

Big Claims, Small Margins: Do Reported GraphLLM Gains Survive Simple Baselines?

LLM-based methods for node classification on text-attributed graphs (TAGs) routinely report state-of-the-art accuracy.

We ask how these results compare with inexpensive baselines run on the same splits.
On the splits of three recent studies, we tune four baseline representatives with validation-only selection and compare them with the highest mean that each paper reports for an LLM-based method in the matched setting.
Correct & Smooth on TF-IDF features trails this reference by a median of 2.3 percentage points, and an MLP on frozen e5-large-v2 embeddings by 2.8 points.
Reading released code next to the papers, we also document evaluation risks: selection of hyperparameters or checkpoints on the test set, labels of test-pool nodes used in training, baseline numbers reused without disclosure, and a text-label misalignment in a released Cora loader.

[1]Big Claims, Small Margins: Do Reported GraphLLM Gains Survive Simple Baselines?
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:48 PM
LLM-based methods for node classification on text-attributed graphs (TAGs) routinely report state-of-the-art accuracy.
We ask how these results compare with inexpensive baselines run on the same splits.

Extractive summary: sentences quoted from the sources.

Before this