GenOlympiad: Benchmarking Frontier Image Models at Their Capability Boundaries
We introduce GenOlympiad, a benchmark which provides the most challenging text prompts from nine capability axes for stress-testing frontier text-to-image models.
ProofPaper ↗
Key points
- Text-to-image models have made tremendous progress and can produce high-quality images.
- Based on this, we develop an automated evaluation framework which decomposes the prompt into easy and hard questions, and for most hard questions, draws help lines to ground evidence for accurate grading.
- We evaluate 20+ strong text-to-image models, including Nano Banana Pro, GPT-Image-2.5, Seedream-5.0 and so on.
- We report that even the best-performing T2I model (Nano Banana Pro) only achieves a strict score of 23.9 out of 100 and that coding models such as GPT-6 Astra have higher prompt alignment scores (but not as visually pleasing).
Sources (1)
- [1]GenOlympiad: Benchmarking Frontier Image Models at Their Capability BoundariesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 05:59 PM
We introduce GenOlympiad, a benchmark which provides the most challenging text prompts from nine capability axes for stress-testing frontier text-to-image models.
Text-to-image models have made tremendous progress and can produce high-quality images.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026VibeTraining-Bench: Data-Trainer-Harness Co-Design for Iterative System Improvement
- Oct 19, 2026Advancing Coding in Low-Resource Programming Languages
- Oct 19, 2026Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
- Oct 18, 2026Comfy-Org/Fibo-2.0
- Oct 18, 2026Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark: DeepSeek-V4.1-Flash-1017 EXL3 2.9bpw on 2x NVIDIA DGX Spark (GB10) with TensorFold: 2.1-2.3x the vLLM kit's decode speed.
- Oct 17, 2026Blown away by Minimax H3 sometimes. It nailed this one!!!!