Local Evidence, Global Failure: Diagnosing Speech-Vision Temporal Composition in Audio-Visual Language Models
We introduce Temporal Evidence Decomposition (TED), a training-free diagnostic framework that separates local visual verification from speech-relative aggregation.
ProofPaper ↗
Key points
- Audio-visual large language models (AV-LLMs) can fail at full-video speech-vision decisions even when useful visual evidence is locally recoverable.
- Automatic speech recognition supplies a transcript and utterance interval.
- TED queries audio-free windows with the transcript for Temporal Alignment and Forecasting, or answer candidates for Cross-Modal Temporal Binding, then aggregates evidence relative to that interval.
- Physical-isolation and timing-shortcut controls test alternative explanations.
Sources (1)
- [1]Local Evidence, Global Failure: Diagnosing Speech-Vision Temporal Composition in Audio-Visual Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 03:49 PM
We introduce Temporal Evidence Decomposition (TED), a training-free diagnostic framework that separates local visual verification from speech-relative aggregation.
Audio-visual large language models (AV-LLMs) can fail at full-video speech-vision decisions even when useful visual evidence is locally recoverable.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026Calibration-Free Early Stopping for Chain-of-Thought Reasoning via E-processes
- Oct 18, 2026Acharya 2.0: An AI Tutor for Self-Learning Subjects in Hindi
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026Beyond Large Language Models: A Mental World Model Framework for Mental Modeling, Simulation, Evolution, and Interpretability
- Oct 16, 2026Language Models Learn to Improve Language Models
- Oct 16, 2026On the Tip of the Tongue: First-Token Cueing as a Behavioral Probe of Recall Failures in LLMs