EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric Alignment
To bridge this gap, we propose EVA-Cap, a novel framework that decomposes captions into atomic audiovisual events and transforms unstructured text supervision into event-centric alignment.
ProofPaper ↗
Key points
- Recent advances in Omni Language Models (OLMs) provide a unified solution for audiovisual video captioning.
- Nevertheless, existing approaches predominantly treat captioning as an unstructured text generation process, neglecting the inherent semantic structure of videos and leading to suboptimal fine-grained audiovisual alignment.
- We train EVA-Cap via Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) on a self-curated dataset, EVA-Data, which is built with event-centric quality control.
- Extensive experiments demonstrate that EVA-Cap significantly outperforms existing methods in event-centric evaluation (e.g., +9.0% average gain on ChronusAV), while achieving competitive performance on general audiovisual captioning benchmarks (e.g., +3.3% on WorldSense).
Sources (2)
- [1]EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric AlignmentHugging Face Daily Papers · Oct 15, 12:00 AM
To bridge this gap, we propose EVA-Cap, a novel framework that decomposes captions into atomic audiovisual events and transforms unstructured text supervision into event-centric alignment.
Recent advances in Omni Language Models (OLMs) provide a unified solution for audiovisual video captioning.
- [2]EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric AlignmentarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 04:42 PM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling
- Oct 14, 2026Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning
- Oct 14, 2026Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning
- Oct 14, 2026The Design Space of Tri-Modal Masked Diffusion Models
- Oct 14, 2026VASPO: Self-Distillation with Privileged Value Estimation
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
