Brain-Aligned Multimodal LLMs for Social and Emotional Understanding of Cartoon Movies
Multimodal large language models (MLLMs) offer scalable annotations of social and affective content, but their temporal agreement with human judgments and neural predictive value require further evaluation.
ProofPaper ↗
Key points
- We compared diffusion-based and autoregressive MLLMs with continuous human Theory of Mind (ToM) and affective valence ratings across three short cartoon movies.
- Behavioral analyses assessed temporally aligned Pearson correlations, rating distributions, and the effects of preceding scene descriptions.
- Ratings from each source were separately used in subject-specific regression models to predict regional fMRI responses.
- Model ratings captured aspects of human temporal variation despite systematic distributional differences, with generally stronger alignment for valence than ToM.
Sources (1)
- [1]Brain-Aligned Multimodal LLMs for Social and Emotional Understanding of Cartoon MoviesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 10:22 PM
Multimodal large language models (MLLMs) offer scalable annotations of social and affective content, but their temporal agreement with human judgments and neural predictive value require further evaluation.
We compared diffusion-based and autoregressive MLLMs with continuous human Theory of Mind (ToM) and affective valence ratings across three short cartoon movies.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026A Plausible Explanation Is Enough to Mislead Multimodal Judges
- Oct 15, 2026Anthropic tells investors quarterly revenue up nearly 50% to $16.8bn ahead of IPO
- Oct 15, 2026OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing
- Oct 14, 2026Tree Appreciator
- Oct 14, 2026On the Limits of Representation-Based Test-Time Adaptation for Multimodal LLMs
- Oct 14, 2026Are We Really Measuring Forgetting? Pitfalls in Evaluating Machine Unlearning for Multimodal LLMs