Where Research Ideation Begins: Forward-Citation Supervision for Actionable Limitation Generation
Starting from a pilot study, we find that as large language models (LLMs) scale from 0.6B to 32B parameters, they become far better at recognizing which limitation later work addressed, but no better at generating one, suggesting that the bottleneck lies in the supervision signal rather than model capacity.
ProofPaper ↗
Key points
- Automated research ideation has advanced rapidly, with multi-agent workflows that generate whole research ideas end to end.
- In this paper, we focus on this first step in the research ideation pipeline.
- We therefore construct ActLim-Bench from forward citations, yielding 18,279 evidence-linked supervision triples with human verification.
- On this signal, we propose a three-stage post-training pipeline: supervised fine-tuning, verifier-filtered rejection-sampling fine-tuning, and reinforcement learning with the verifier as an online dense reward.
Sources (1)
- [1]Where Research Ideation Begins: Forward-Citation Supervision for Actionable Limitation GenerationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 07:14 AM
Starting from a pilot study, we find that as large language models (LLMs) scale from 0.6B to 32B parameters, they become far better at recognizing which limitation later work addressed, but no better at generating one, suggesting that the bottleneck lies in the supervision signal rather than model capacity.
Automated research ideation has advanced rapidly, with multi-agent workflows that generate whole research ideas end to end.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 16, 2026Welcome Beam: a 501B open model you can own, customize, and scale
- Oct 15, 2026Towards Continual Learning: Mitigating Forgetting With Subspace Tuning
- Oct 15, 2026Progress Reward Model for Reinforcement Learning via Large Language Models
- Oct 15, 2026NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
- Oct 15, 2026Reinforcement Learning-driven Diffusion Models: Language, Vision, and Beyond
- Oct 14, 2026Gray-Box Hallucination Detection in LLMs with Token Identities