CulturalRiddles: A Multicultural Benchmark of Riddles
We propose CulturalRiddles, a benchmark of original, culturally grounded riddles created entirely by humans through a participatory community effort, spanning 62 cultures (45 countries, 51 languages).
ProofPaper ↗
Key points
- As LLMs expand their language coverage, evaluating how well they understand and represent diverse cultures becomes increasingly important.
- However, many benchmarks remain English-centric, with limited cultural coverage and a focus on factual knowledge over its application in real-world cultural contexts.
- These riddles are deeply rooted in their respective cultures and require abductive and compositional reasoning, alongside detailed factual and social knowledge, language skills, and an understanding of culturally specific associations.
- We benchmark 28 LLM configurations using both automatic and large-scale human evaluation.
Sources (2)
- [1]CulturalRiddles: A Multicultural Benchmark of RiddlesHugging Face Daily Papers · Oct 21, 12:00 AM
We propose CulturalRiddles, a benchmark of original, culturally grounded riddles created entirely by humans through a participatory community effort, spanning 62 cultures (45 countries, 51 languages).
As LLMs expand their language coverage, evaluating how well they understand and represent diverse cultures becomes increasingly important.
- [2]CulturalRiddles: A Multicultural Benchmark of RiddlesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 04:20 PM
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 19, 2026Evaluating Agents on Highly Subjective Tasks
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline
- Oct 16, 2026OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners