Scaling Graph Reasoning in Language Models with Learnable Graph Positional Encodings
Solving reasoning tasks supported on graphs typically requires providing a Language Reasoning Model (LRM) with a serialized description of the graph.
ProofPaper ↗
Key points
- We propose to address this by augmenting LRMs with scalable and permutation equivariant graph positional encodings that modulate attention logits.
- We compute these positional encodings using a GNN followed by a Graph Transformer.
- We jointly fine-tune the positional encoder and the LRM LoRA adapters using optimal solutions generated by graph-theoretic solvers, resulting in a computationally efficient method.
- Evaluations on graph-theoretic benchmarks also show that performance on small graphs is preserved.
Sources (1)
- [1]Scaling Graph Reasoning in Language Models with Learnable Graph Positional EncodingsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 12:39 AM
Solving reasoning tasks supported on graphs typically requires providing a Language Reasoning Model (LRM) with a serialized description of the graph.
We propose to address this by augmenting LRMs with scalable and permutation equivariant graph positional encodings that modulate attention logits.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026TaQFormer: A Treatment-as-Query Transformer with Data-Fusion for Multidimensional Treatment Modeling in Coupon Allocation
- Oct 19, 2026Learning to Couple: Context-Adaptive Joint Denoising for Diffusion Language Models
- Oct 19, 2026TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
- Oct 18, 2026Learning When to Abort in Transformer Planning: A Mechanistic Study
- Oct 18, 2026PACT: NVFP4 Data-Flow LoRA Fine-Tuning on Consumer GPUs
- Oct 17, 2026Dual Task Vector Merging: Leveraging Emotion and ASR Knowledge for Low-Resource Alzheimer's Disease Detection