DiffGround: Semantic Difference-Guided Zero-Shot Video Question Grounding

Key points

• We propose DiffGround, an action-driven zero-shot framework built upon frozen Multimodal Large Language Models (MLLMs).1
• Our framework first predicts an answer-relevant action cue that transforms the input question into an explicit semantic target for grounding.1
• Finally, we introduce Semantic Diff Refinement, a training-free temporal refinement method that exploits semantic differences between neighboring SigLIP2 embeddings to recover complete action segments from sparse predictions.1
• Extensive experiments on NExT-GQA and STAR show that DiffGround consistently outperforms existing zero-shot and supervised methods, establishing a new state of the art for Video Question Grounding.1

Sources (1)

• [1]DiffGround: Semantic Difference-Guided Zero-Shot Video Question Grounding
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 10:17 PM

  Video Question Grounding (VideoQG) requires both accurate question answering and precise temporal localization.

  We propose DiffGround, an action-driven zero-shot framework built upon frozen Multimodal Large Language Models (MLLMs).

Extractive summary: sentences quoted from the sources.