Language Models are Lazy Poets: Creative Mode Collapse at Scale
We address this gap through an extensive treatment of LLM mode collapse in poetry.
ProofPaper ↗
Key points
- While aligned large language models (LLMs) can successfully complete many creative tasks, their outputs tend to be highly self-similar.
- Though solutions to this problem of "mode collapse" have been proposed, comprehensive large-scale surveys of the phenomenon in specific textual genres are still lacking.
- At million-poem scale, we find that LLM poetic vocabularies and rhyming abilities are substantial, but severely underutilized.
- Further, we find that lexical homogenization differs across models and poetic forms, noting observed rhyming vocabulary is correlated with model size.
Sources (1)
- [1]Language Models are Lazy Poets: Creative Mode Collapse at ScalearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 08:45 PM
We address this gap through an extensive treatment of LLM mode collapse in poetry.
While aligned large language models (LLMs) can successfully complete many creative tasks, their outputs tend to be highly self-similar.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline
- Oct 16, 2026Language Models Learn to Improve Language Models
- Oct 15, 2026Decomposing Brain-LLM Alignment with Interpretable Semantic Representations
- Oct 15, 2026Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
- Oct 15, 2026Training Reasoning Models to Generate Diverse Scientific Ideas