CREATE: Creativity Evaluation of VLMs using the Visual Remote Associates Test
Vision language models (VLMs) are used in creative tasks such as visual storytelling, design ideation, art interpretation, and advertising.
ProofPaper ↗
Key points
- Yet we lack principled ways to measure the lateral reasoning of these models compared to human potential.
- We introduce CREATE, a semi-automatically constructed benchmark containing 155 visual Remote Associates Test problems.
- We conduct a human study (N=25) to validate and ground the quality of our benchmark, and evaluate 11 VLMs. Our findings show that most VLMs underperform humans on this task.
- Our experiments indicate that the associative ability of VLMs depends on the cue concepts provided in the text and does not universally extend to visual puzzles.
Sources (1)
- [1]CREATE: Creativity Evaluation of VLMs using the Visual Remote Associates TestarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 03:38 PM
Vision language models (VLMs) are used in creative tasks such as visual storytelling, design ideation, art interpretation, and advertising.
Yet we lack principled ways to measure the lateral reasoning of these models compared to human potential.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Physics-Aware Deepfake Detection via Distance-Speech Consistency
- Oct 12, 2026DTA-GT: Direction- and Topology-Aware Graph Transformer for Neural Network Representation Learning
- Oct 12, 2026Auditing Observation-Quality Shortcuts in the Evaluation of Person-Centered Motion for Video Anomaly Detection
- Oct 12, 2026Not Every Cue Is Evidence: Cue Supportability for Forensic Distillation in Face Forgery Detection
- Oct 12, 2026RelLedger: Relational Information Governance for LLM Agent Organizations
- Oct 11, 2026RetryBench: Separating Retry Permission from Recovery Outcomes in GPT Tool Use